Github Can Never Support Kernel Development
Vložit
- čas přidán 11. 07. 2024
- The Linux kernel is a very wierd project making use of external bug trackers, a mailing list, and a really odd way of using Git, it's no surprise that it's like this but because of it there's no way for Github to ever be used.
==========Support The Channel==========
► Patreon: brodierobertson.xyz/patreon
► Paypal: brodierobertson.xyz/paypal
► Liberapay: brodierobertson.xyz/liberapay
► Amazon USA: brodierobertson.xyz/amazonusa
==========Resources==========
Github Kernel Development: blog.ffwll.ch/2017/08/github-...
=========Video Platforms==========
🎥 Odysee: brodierobertson.xyz/odysee
🎥 Podcast: techovertea.xyz/youtube
🎮 Gaming: brodierobertson.xyz/gaming
==========Social Media==========
🎤 Discord: brodierobertson.xyz/discord
🐦 Twitter: brodierobertson.xyz/twitter
🌐 Mastodon: brodierobertson.xyz/mastodon
🖥️ GitHub: brodierobertson.xyz/github
==========Credits==========
🎨 Channel Art:
Profile Picture:
/ supercozman_draws
🎵 Ending music
Track: Debris & Jonth - Game Time [NCS Release]
Music provided by NoCopyrightSounds.
Watch: • Debris & Jonth - Game ...
Free Download / Stream: ncs.io/GameTime
#Linux #OpenSource #Github #FOSS
DISCLOSURE: Wherever possible I use referral links, which means if you click one of the links in this video or description and make a purchase I may receive a small commission or other compensation. - Věda a technologie
Just a side note about Discord conversation too... The great thing about the kernel mailing list is that it's archived, indexed and fully searchable/readable. It kinda bums me out how many seemingly important discussions happen on discord, where it can't be searched and it's just gone forever
the fact people use discord still just confuses me, it's got a really closed off client, it has no e2e, it has no mobile web interface, until recently it had no oled dark mode, it overly pushes premium features, etc.
Matrix is even worse on this. I'm not even talking about contributing to a project, I'm talking about DMs. It's pretty much impossible to go back to a link I shared with friends after like a year or so. I can only imagine how awful it is to use it for project collaboration.
In other words: just use forums or mailing lists to discuss important things.
Fully serchable?
@@voidmain7902 Seaching exists in Matrix tho. In unencrypted matrix rooms the server does the indexing for you, in encrypted rooms (mostly DMs) the server cant see the messages and your client needs to do that instead. Element Desktop does support this and lets you search in your DMs. Additionally through the matrix-viewer it is possible to make chats indexable for search engines.
A lot of tech/hobby communities have shifted to discord also. It sucks because it's hidden from search engines.
Linux was born in the early 90s. Usenet and mailing lists. ISPs ran their own servers for each. You had static IPs and people often ran their own servers. Interoperability was a thing, you didn't have fast links so you'd need a local copy. Virtual machines were not common. It was a very different time and the kernel was born and formed in this context. As a result the environment left its imprint on the project. Plus you also have a lot of older developers who take the view of it ain't broke don't fix it. They changed source control to git because other tools didn't work and Torvalds wrote git. I think if the kernel was to change it would have to be something that is self sovereign and also distributed. Relying on one company for hosting and tooling is risky. What the Kernel currently uses is diversified in software, distributed and many times redundant all be it from the early 90s.
Reliance on a company for tooling is also very risky when that company happens to be owned by the same company that tried to sabotage you decade or so ago
@@thegoldenatlas753 indeed. MS seems better behaved but who knows right. They might do something to make Linux better for Azure at the detriment of the rest of us. Gotta keep en separated.
@@vk3fbabwell, consider that they are still a corporation. Yes they behave much better than they did when Bill Gates was the CEO, but still they are not the most friendly bunch.
@@thegoldenatlas753I think it's not a big risk considering there is like a million mirrors of popular repository
The fork thing is actually a huge pain point imo. I don't want to fork and set up new remotes in my clone just to submit one commit. Forgejo/gitea actually support agit, which lets you open a PR in a repo without forking at all.
I also don't think the kernel should move to a platform like github, since it's proprietary. Gitlab is a bit of a mixed bag, though I am absolutely biased since I'm a member of Codeberg.
Please let's not push something that's so important to be managed by microsoft. Nope. Never. Please.
Their QC is appalling, and I don't want such an amazing project being held back by corporate entities.
Great video and article. The way I see it is that something like an OS kernel hard to split up. It's modular but every module needs to integrate with each other, and in the end needs to be built as a whole in order to actually integration test and run it.
Not every large software project has that problem, there are projects which are really big but from which all parts are buildable far more separately. Splitting those up in multiple repos is a lot less of an issue in that case.
A good example of that is projects around programming languages. For example the DotNet and Rust projects are really big, but they have many individual parts which separate naturally well.
The only reason we use Github is for discoverability and because people are used to it. Life would be so much easier if we could just host everything on our own servers.
For context, we (PULP platform) develop microcontroller related IP, all our stuff needs to be integrated and tested on proprietary software under NDA and so on. At the moment we have these super convoluted systems that can fetch PRs and so on into an internal git, combine it with the non-free parts, simulate and so on and then generate reports.
It would be so much simpler if everything was in one place.
If Linux were to move to something newer my money is on sourcehut.
At the end of the day it likely wouldn't lead to the model actually changing
@@BrodieRobertsonbecause the model offered by github actually offers much more cons to the consumer and much more power to the provider. Why should kernel developer move to github or something like github when they have it better? At the same time if they ever get tired of hosting, sourcehut takes the burden off their shoulders.
The entire design of the Github tooling, and it's competitors, more or less duplicates how Subversion, CVS, and even RCS before them, were used (with custom scripts or third party tools filling the role of the Github tools). This workflow predates the Linux kernel, and to say it newer or more modern than the kernel development flow is wrong. Instead, Github implements habits older than most of the developers using it, while the Linux kernel follows an alternative flow developed to meet the kernel projects needs and Linus' preferences.
On the bright side, at least it isnt Gitee, which requires you to login to be able to just get a local copy
Don't hold your breath
4:38 maybe the real engineering problem is that web developers are too much focused on avoiding duplication. Duplicating efforts is fine on engineering, you can build two bridges that both allow cars to pass, its fine, not everything needs to be reused or go throw the same interface, and maybe that's why we still use text files and we should go for email, the lowest common denominator.
I didn't learn about the linux pull request system until much later than github, but honestly I found it way easier to grasp.
The mailing list is agnostic, and version management tools have changed periodically. Also there are lots of git tools just as good or better than GitHub and without it's ties.
i really want a solution to exist where git is properly decentralized on the server instance level, if i host forgejo/gitea and i make a repo public i probably want anyone to be able to contribute even if they don't have an instance themselves and sending around patch files is like the only reasonable way to achieve that
They are working on implementing federation.
Why, I can pull and push from your instance just fine.
@@izerpizer im guessing through something like activitypub protocol?
Kind of what i was thinking. Git is by design meant to be decentralized. You don't always just pull from/push to upstream but maybe to other origins as well. I think that is kind of what the kernel does in the essence of the git tree and github only supports it within its environment by forking a repo on github. Still this creates all the issues mentioned as only the code is mirrored but neither pull requests nor issues.
Anyone is able to contribute on a git repo. Don't get your question. You mean not having to do a git clone? One needs code to patch against, so the code "instance" is needed no matter what.
mmmmm I have github projects with submodules. You can clone with something like git submodule update --remote. The key is all of my submodules are themselves stand alone libraries that have value as individual projects outside of the including projects. This avoids having to tell people "I have this neat module, but it is buried in this other project"
Yeah that's not something the kernel will ever do, because they have clear intentions of not having stable internal APIs.
Parts of the kernel simply cannot stand on their own, the whole thing has its composite version, and it must be maintained as such.
@@aeghohloechu5022 Well, I think personally it is a mistake to try to track all of the drivers in Linux. This is what? Half the kernel, and they could leave it to the hardware manufacturer's to do. Yes, I know the idea is to force the hardware makers to reveal their code, but that's not the only way to do that, and the kernel would be far more manageable without the drivers all embedded.
Put it this way. BSD does not have this requirement, and the system still works.
@@scottfranco1962 if Linux had gone the micro kernel route then drivers would be user space and could each have their own repo etc. But that's opening up old wounds and creating many more issues than it solves.
You can still do a mono-repo. Coming from SVN, CVS, RCS and many older version control systems, this is a head scratcher to me. Do you really want the version management problem in addition to all the other challenges with developing code? Put related code all in the same repo. The fact that you cannot checkout a single directory in git is a real step backwards.
@@vk3fbab I was never for user space drivers, but I am not for kernel space drivers either. The x86 processors were designed with (IIRC) 4 privilege levels, but of course Intel famously f**ked it up, so more commonly its a binary kernel/user space. In any case, the permission model is dated in any case. The modern model is to wrap each driver in hardware access defined by page protection. This is conflated with "user space drivers" but it need not be. All of that is driven by a design error in interrupt handing by Intel, but of course now we are off topic[1].
Besides the rolling of all drivers into kernel space, a sickness that Windows shares, there is the fact that Linux depends on named linkages, essentially making driver installation driven by a run time linker. This in turn is driven by Linux/Linus's preference for keeping driver models fluid. Why establish a vector model if we aren't going to also keep the driver model stable?
In any case, I would not preach for the so called "user space driver" model. I have seen that implemented by so many corporations in a messed up manner that I am not talking about it anymore (polling instead of interrupts, to mention the biggest issue).
[1] could have an endless thread on that, but note that changing the paging tree root register has nothing to do with the permission level.
It seems like many of these issues are solvable through bots that keep stuff in sync between multiple repos.
I have stay out of issues or pull requests because I always find myself pulling a Linus.
We use submodules in a really big project with no issues.
However for why they don't use new stuff, the answer is simple: they don't have the time or will for the effort of switching everything. And to add on that Linus wants to remain the gran papa without bothering with everything, just his own repo.
If I see a project that redirects you to discord or matrix, that project is dead to me. Chat platforms are not good for maintenance
I have been interested in Linux kernel development for a while.
Is there an interesting part that doesn't require you to be an experienced professor to work on?
Just read the documentation, everything is explained. Essentially you'll have to mail a patch to the respective maintainer of the section you're working on.
@@TheClonerx I can understand that part, but such a patch should be fixing or adding *something* and i don't know where i can get those "something"'s. For example, is there a list of things you can try to work on, or do you need to find those things yourself?
It's probably documented, but i was just lazy and asked the chat.
@blinking_dodo I am curious as well, but some of the projects I followed were mostly corporate funded.
For example, our cloud infra needs to control hardware usage, let's build cgroup!
@@blinking_dodo that's not how kernel development works.
You don't find things to work to as good first issue or something dumb.
You work on the things you NEED. Whether it be a company requirement, your hardware, or something is terribly fucked on your specific system configuration and the printer is on fire.
@@aeghohloechu5022 So if you don't need something solved yourself, you can't involve yourself with kernel development?
Then who is fixing the bugs that get reported by people who don't supply fixes?
does GitLab or Codeburg have the same issues?
Yes
I wonder if they could consider such tricky stuff (like multiproject merge requests) while GitLab is already tackling the federation issue. Maybe we can discuss and expand it given it's far more open in structure than GitHub
Sourcehut! Sourcehut! Sourcehut!
One thing I don't get is why they don't just program a git web interface themself with the monotree and any other stuff they might want? I mean git was born that way in the first place, the linux fundation clearly has the programming power to do it and they would benefit from that.
why fix what isnt broken? the current systems worked just fine for the past 30 years
@@VioletRMbecause dealing with mailing lists is a major pain point for new contributors, even if they're already experienced developers.
They're also just shit in general - simple things like finding out if a patch has been merged, and which repository it's now in, are unnecessarily difficult. Suggesting changes, viewing a whole patchset as a single big patch, having CI integrated and automated, linking other patchsets and so on are entirely trivial to do on Gitlab but complicated or impossible with mailing lists.
Github is now microsoft. Which would mean to basically setting oneself up what happened with bitkeeper in the first place.
This is the ultimate answer to today's imo stupid interview question "What's the difference between Git Flow and GitHub Flow"
It really isn't
I can imagine Microsoft has these features internally and doesn't want to put it on Github for 'competitor's' to use lol
All of these issues are fair enough. Then again, it is insufferably difficult to track and find issues as a technically-minded end-user. Trying to report and follow issues for the DRM drivers for my AMD GPU is pain, and took me nearly a week to stumble on the DRM gitlab repo.
the real question for me is: dear github, if i want to use you as a pure read only mirror, why cant we disable pull request. sound so easy to me yet pull request are enabled on default enforcing additional work to the repo owner to deflect wrongly addressed pull requests
you getting ahead of yourself was really weird lol
This feels like people not understand it is more UI issues. Putting things into a tree chart of sorts could be better esp if you could custom label stuff in a good way. It seems to me is if these issues were brought up to various git sites, it could be fixed. But mailing lists the way they are confuse me way too much.....I couldn't imagine working like that at all. Like it properly supports stuff in github or any other git based system, and I don't think they understand what it supports and doesn't 😅
before i even start to watch this video, i will say AGREE!
So is today the day I shill for Git alternatives like Fossil & Pijul ?
pijul site is broken
But Github does one thing right... You have a nice way to respond to people in the gaming space that tells you "git gud"...
There is a separate package for it that makes it easy to "git gud" and kill the poor meme :D
Hmm... Out of curiosity. What is that package for?)
github isn't the only option... some alternatives could even be open source?
codeberg
It's not just about GitHub though. More like the "modern git workflow" aka the "limited size corpo" way that doesn't quite work when you got thousands of subsystems maintained by people all over the world and contributed by people coming in and out all over the world.
@@voidmain7902 they have decades of historical data on how kernel development works, i bet currently ppl are running scripts at the cmdln... they could be using a modern html5 interface, that still uses email on the backend and internally things can also be tracked using msgids.
You have codeberg, gitlab, gitea are all foss, but they all share a common workflow, it's not a platform specific problem, is more like a workflow problem, for most of the cases his doesn't matter and may actually be better, but for something like the kernel it doesn't work so well
gitblit, never mentioned anywhere but a good option.
monolith =/= monorepo
NixOS video WHEN???? WHEN!!!!
do you have a mailing list or blog with just links to the source articles? i literally never watch your videos just click on them, scroll down to the sources and read the interesting articles. sorry 😬
it would be cool if people went back to using email more often. I think it could be adapted to work like a chat app really easily.
Email has the disadvantage of being neutered to hell and back in the name of countering spam.
It also has no way to anonymize the parties.
It already does - it is called Delta Chat. It has noob-friendly, E2E encryption and all the goodies associated with Signal and co. The only thing it can not do is encryption of the email headers, as that would break delivery of the email. If your use case is that sensitive, only then you would need stronger tools.
mf, last week I was just googling about why do they use mailing list. are you stalking on me or something
Probably because you'll keep questioning every time when debugging something related to Linux. You'll end up in a main listing not providing you any clear answers or just being confusing.
GitHub is dead to me anyway, since the stupid 2FA thing. Lets not move Kernel development to a Microsoft owned platform.
what stupid 2FA thing that they require it you know the thing every website should do by default
@@bigpod I refuse to install any spyware on my phone, or give GitHub my phone number. So just stopped using it.
There are open source otp tools
@@bigpod freedom is having the option, not being forced to tie an account to a physical phone
@@lorenzo42p you still have a choice you dobt have to tie a phone phone is just easiest, you can still use security keys but from security perspective 2fa is kinda a must have if you ask me
Moved off of Github after the Microsoft aquasition
cause what you did is important
Personally, I think its easier to just not argue with Linus, arguing with him instantly gets you roasts so strong, even Gordon Ramsay would want to learn from him
I still do not understand the fundamental problem the kernel community has with GitHub. I don't see what the kernel community is doing that can't be done on GitHub or in addition to GitHub.
Regarding monorepos. You started out with a monorepo and presented a fragmented repo approach as an improvement. Why? You pointed out a lot of problems resulting from it. Isn't it a step back? I believe, modular projects usually move in the opposite direction, from a fragmented approach to a monorepo. It simplifies a plethora of maintenance tasks.
probably Microsoft bad decentralised foss good 💯💯😊😊😊, Ms is big tech but probably the best platform because they are big tech. imagine any modern company using a mailing list for maintaining software. there's a reason why crappy win10 is still better than Linux because ms actually knows that laptop manufacturers and app devs are more important than the actual product
watch video
correction, GIT sucks in many ways. But because linus made it, everybody keeps using it like its not trash
This is Full Garbage, change Linux to an true Micro kernel with Plugins. And make sure that the main Linux Micro kernel never change API and if they make any Changes to an API, just create a new API and make sure that older Plugins still work with the old API. This will make the System more Stable and easier to develop.
Hurd durd
Is this copypasta? If not, use Hird.