Can someone with more understanding of the Linux Kernel arch/design describe why these drivers are in the kernel tree? Are these not part of a HAL or are they more "integrated" then that?
Linux does not use a stable API to talk to drivers, so drivers are intended to be part of the kernel tree so they can change with the rest of the kernel evolving.
Eh, that's not exactly true. Most of the API is pretty standard and stable. They don't have a fixed ABI, which is how closed source OSes handle driver compatibility.
There is a significant difference between having a supported defined interface boundary, and a semi-stable interface that doesn't necessarily guarantee backwards comparability to prior interfaces.
> There is a significant difference between having a supported defined interface boundary
Which is literally the delineating mark between an API and and ABI.
A stable ABI means your binary interface will remain the same and your compiled drivers should work, despite any changes on the kernel level. A stable API means you'll probably have to recompile your drivers when the kernel changes, but that they'll require little (if any) modifications.
A stable API means you will definitely not have to make any modifications, and Linux provides no such guarantees. "Pretty" standard and stable is not the same at all.
So, by that logic, there has never been a stable driver API in the history of OSes. Windows 9x, Windows 2000/XP, Windows Vista/7/10 all had different graphic driver models and were incompatible. Same with various releases of OS X/macOS.
Of course I’m not implying that the driver interfaces have a guarantee or will not change, especially with the rolling release model of Linux. However, you can be pretty certain within a major point release that there won’t be significant breaking changes. Thus the “pretty stable”. It’s not a pro nor a con, it’s just the style that works for that open source project.
It's a matter of degree I guess. My understanding is that the kernel pretty much reserves the right to change these APIs as they feel fit, without much regard for the consequences for out-of-tree drivers, even if in practice the interface has settled and isn't in need of changing much currently.
