Whatever happened to the GNU HURD?
Oct. 29th, 2010 06:51 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
liam_on_linuxIt's been 20 years now since the GNU microkernel Unix, the HURD, was announced. So where is it?
Bear in mind in the following that when I speak of an OS I am talking about the core OS - the kernel and essential services. Not the shell or the filesystem or user tools like ``ls'' or ``more'' or ``vi'' or anything, and certainly nothing to do with relatively trivial outer layers such as graphical user interfaces.
The GNU HURD was a very ambitious project: to build a complete, UNIX-compatible OS on a microkernel basis. Microkernels ("µkernel" for short) are very hard to make work, but it has been done - QNX, Chorus, Amoeba and others are all technically microkernels, for instance. Not are really Unices, though, although QNX sort of superficially resembles one enough for developers to feel some familiarity.
Contrary to popular belief and Apple & NeXT's strongly-promoted message, neither Mac OS X nor NeXTstep before it were technically microkernels.
The idea of a microkernel is that only a tiny piece of code runs in the processor's (or processors') Ring 0; the rest of the OS is composed of small pieces of userspace code, running in Ring 2 or 3 (using x86-32 rings for reference here). The modules, called servers or daemons, all communicate with each other to work together as a complete OS.
The theory is that because the OS is very modular, it is easier to work on, more reliable and more robust. If a server dies, it can be restarted and the rest of the kernel will not be affected.
The first microkernel to get much real-world recognition was Mach, designed at Carnegie-Mellon University in the USA. Mach didn't get to a reasonable level of completion until Mach 3.0 but the earlier versions spawned a host of projects.
One was OSF/1 by Digital Equipment Corporation, which became Compaq Tru64 and was killed by HP. Another was MkLinux for the original Motorola 680x0-based Apple Macintoshes.
And one was NeXTstep, which became Mac OS X.
To make their new OS viable, NeXT wanted to make something Unix-compatible, so they took a huge chunk of the kernel of BSD (not FreeBSD or NetBSD or OpenBSD, this is before them) and built it directly into the Mach kernel as a sort of "Unix compatibility later". This means that the kernel is no longer "micro" at all - it has a honking great monolithic lump of old Unix code bolted onto it.
The result is called Xnu, and whereas it may not be the most elegant solution, it certainly works. It's now the best-selling Unix ever, estimated to outnumber, in both installed systems and number of users, every other commercial Unix and Unix clone put together.
GNU took the Mach kernel as well as the basis for HURD, but it tried to do things properly, the pure microkernel way. When Linus Torvalds wrote his kernel, he didn't expect it to compete with HURD - it was meant to be a small quick hack. As he famously said in news:comp.os.minix: "I'm doing a (free) operating system (just a
hobby, won't be big and professional like gnu) for 386(486) AT clones."
That was the first announcement of what became and was later named Linux. (It had been called "Freax".)
The thing is, the GNU developers found that writing a microkernel-based Unix is very very very hard.
When Linus made his post, Prof. Andy Tanenbaum, a very respected academic in OS research and the author of Minix, the OS that begat Linux, said Linux was obsolete: the direction of the future was clearly microkernels. (And indeed Tanenbaum has produced several µkernel OSs himself.)
Technically, at least in a theoretical sense, he was right, but Torvalds' practicality in deciding to implement a simple, classical, one-big-monolithic-lump-of-code type of old-fashioned Unix kernel has been proved correct - Linux is now mature, sophisticated and highly usable. In some terms, proprietary Unixes such as HP/UX or AIX or Solaris might do some things better, but Linux is doing very well.
Meantime, 20y later, the GNU team have not made all that much progress with HURD.
About 4y ago, it has got to a stage where it looked like it could be used for simple stuff. Debian built a version of the Debian GNU distro around the new HURD kernel instead of the Linux kernel. (There are also Debians built around all the BSD kernels.)
An announcement was made on Slashdot. The infant OS was hosting its own website. It immediately collapsed under the onslaught.
Soon after this, the team decided that things had moved on since the creation of Mach in the early 1980s and moved HURD onto a new, more sophisticated and mature µkernel: L4. This involved a very big backward step, so arguably, HURD is less complete and ready now than it was then. Also, things have moved on since L4 and it too has successors, and research projects are looking at them as possible HURD bases, too.
Personally, I think that one day, Linux is just going to prove to be too big and too complex to maintain and develop efficiently any more. Some developers might move on to a successor, something more modular. I'd have liked to see kernel 2.6 named 3.0 when it was released - there was no planned 2.8 or work-in-progress 2.7; indeed the old stable-and-development-kernels-in-tandem model has been discarded.
But perhaps some day a modular Linux 3.0 might be started.
There are other directions - for instance, the chaps that originally developed Unix (of which Linux is really just a re-implementation) went on to further refine and develop their ideas in Plan 9, which later developed into Inferno. These are networked OSs, with resource sharing between nodes an integral OS concept, rather than something bolted on later as it is with Linux, Unix, Windows and so on.
I do not understand the technical details but the structure of Plan 9 apparently makes the whole concept of µkernels irrelevant - it is functionally divided into pieces already, just not alone the same division of privileged tiny kernel + user-space servers as µkernel OSs. I'd like to see development on Plan 9 picked up and the enhancements of modern Unixes, such as Linux, brought to it. That could be something very special.
Also, given that other µkernel research OSs have made it to complete, functional condition, such as Minix 3, then perhaps the HURD design is flawed and they should drop it and move on to HURD 2 or something. I don't know enough about the minutiæ.
But HURD isn't finished, maybe never well be, but it's been a very interesting project all the same. And whereas maybe µkernels are not the right way to go, I think the evidence from Minix 3 and Ameoba and QNX and so on is that they can be made to work, and perhaps that is how things will in fact go one day.
Bear in mind in the following that when I speak of an OS I am talking about the core OS - the kernel and essential services. Not the shell or the filesystem or user tools like ``ls'' or ``more'' or ``vi'' or anything, and certainly nothing to do with relatively trivial outer layers such as graphical user interfaces.
The GNU HURD was a very ambitious project: to build a complete, UNIX-compatible OS on a microkernel basis. Microkernels ("µkernel" for short) are very hard to make work, but it has been done - QNX, Chorus, Amoeba and others are all technically microkernels, for instance. Not are really Unices, though, although QNX sort of superficially resembles one enough for developers to feel some familiarity.
Contrary to popular belief and Apple & NeXT's strongly-promoted message, neither Mac OS X nor NeXTstep before it were technically microkernels.
The idea of a microkernel is that only a tiny piece of code runs in the processor's (or processors') Ring 0; the rest of the OS is composed of small pieces of userspace code, running in Ring 2 or 3 (using x86-32 rings for reference here). The modules, called servers or daemons, all communicate with each other to work together as a complete OS.
The theory is that because the OS is very modular, it is easier to work on, more reliable and more robust. If a server dies, it can be restarted and the rest of the kernel will not be affected.
The first microkernel to get much real-world recognition was Mach, designed at Carnegie-Mellon University in the USA. Mach didn't get to a reasonable level of completion until Mach 3.0 but the earlier versions spawned a host of projects.
One was OSF/1 by Digital Equipment Corporation, which became Compaq Tru64 and was killed by HP. Another was MkLinux for the original Motorola 680x0-based Apple Macintoshes.
And one was NeXTstep, which became Mac OS X.
To make their new OS viable, NeXT wanted to make something Unix-compatible, so they took a huge chunk of the kernel of BSD (not FreeBSD or NetBSD or OpenBSD, this is before them) and built it directly into the Mach kernel as a sort of "Unix compatibility later". This means that the kernel is no longer "micro" at all - it has a honking great monolithic lump of old Unix code bolted onto it.
The result is called Xnu, and whereas it may not be the most elegant solution, it certainly works. It's now the best-selling Unix ever, estimated to outnumber, in both installed systems and number of users, every other commercial Unix and Unix clone put together.
GNU took the Mach kernel as well as the basis for HURD, but it tried to do things properly, the pure microkernel way. When Linus Torvalds wrote his kernel, he didn't expect it to compete with HURD - it was meant to be a small quick hack. As he famously said in news:comp.os.minix: "I'm doing a (free) operating system (just a
hobby, won't be big and professional like gnu) for 386(486) AT clones."
That was the first announcement of what became and was later named Linux. (It had been called "Freax".)
The thing is, the GNU developers found that writing a microkernel-based Unix is very very very hard.
When Linus made his post, Prof. Andy Tanenbaum, a very respected academic in OS research and the author of Minix, the OS that begat Linux, said Linux was obsolete: the direction of the future was clearly microkernels. (And indeed Tanenbaum has produced several µkernel OSs himself.)
Technically, at least in a theoretical sense, he was right, but Torvalds' practicality in deciding to implement a simple, classical, one-big-monolithic-lump-of-code type of old-fashioned Unix kernel has been proved correct - Linux is now mature, sophisticated and highly usable. In some terms, proprietary Unixes such as HP/UX or AIX or Solaris might do some things better, but Linux is doing very well.
Meantime, 20y later, the GNU team have not made all that much progress with HURD.
About 4y ago, it has got to a stage where it looked like it could be used for simple stuff. Debian built a version of the Debian GNU distro around the new HURD kernel instead of the Linux kernel. (There are also Debians built around all the BSD kernels.)
An announcement was made on Slashdot. The infant OS was hosting its own website. It immediately collapsed under the onslaught.
Soon after this, the team decided that things had moved on since the creation of Mach in the early 1980s and moved HURD onto a new, more sophisticated and mature µkernel: L4. This involved a very big backward step, so arguably, HURD is less complete and ready now than it was then. Also, things have moved on since L4 and it too has successors, and research projects are looking at them as possible HURD bases, too.
Personally, I think that one day, Linux is just going to prove to be too big and too complex to maintain and develop efficiently any more. Some developers might move on to a successor, something more modular. I'd have liked to see kernel 2.6 named 3.0 when it was released - there was no planned 2.8 or work-in-progress 2.7; indeed the old stable-and-development-kernels-in-tandem model has been discarded.
But perhaps some day a modular Linux 3.0 might be started.
There are other directions - for instance, the chaps that originally developed Unix (of which Linux is really just a re-implementation) went on to further refine and develop their ideas in Plan 9, which later developed into Inferno. These are networked OSs, with resource sharing between nodes an integral OS concept, rather than something bolted on later as it is with Linux, Unix, Windows and so on.
I do not understand the technical details but the structure of Plan 9 apparently makes the whole concept of µkernels irrelevant - it is functionally divided into pieces already, just not alone the same division of privileged tiny kernel + user-space servers as µkernel OSs. I'd like to see development on Plan 9 picked up and the enhancements of modern Unixes, such as Linux, brought to it. That could be something very special.
Also, given that other µkernel research OSs have made it to complete, functional condition, such as Minix 3, then perhaps the HURD design is flawed and they should drop it and move on to HURD 2 or something. I don't know enough about the minutiæ.
But HURD isn't finished, maybe never well be, but it's been a very interesting project all the same. And whereas maybe µkernels are not the right way to go, I think the evidence from Minix 3 and Ameoba and QNX and so on is that they can be made to work, and perhaps that is how things will in fact go one day.
