Liam_on_Linux (![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
liam_on_linux) wrote2013-10-19 07:04 pm
liam_on_linux) wrote2013-10-19 07:04 pm
It's not about the programming language.
The only reason why the language should intrude into the discussion is the side-question of whether certain language facilitate or hinder certain types of work.
Look at it from a different angle. For many a working programmer (and writer, believe me), the WWW is a massive distraction. A necessary one, but one that is separate.
What do you need to do, if you are co-working on a significant project? Edit code, obviously. Save it, compile it, run it, probe it with debuggers. Navigate your filesystem, load and view other files, move stuff between them. Occasionally, read and write email, or IRC, or (more historically) newsgroups, to discuss what you're doing. Possibly retrieve files from remote servers or put them there.
The point being, Emacs has extensions to do all this, so that you can do it all in a consistent fashion in a consistent (if horrible) UI, so that you can spend your entire day inside a single Emacs session and never leave and thus mentally never have to change gear.
Emacs, for some of its fans, is their entire OS. Their computer runs something that lets them launch Emacs and has an accessory function of browsing the web.
I don't do this - I don't speak Emacs at all - but I know people who do and really like it.
Well, turn that inside out. Consider an OS whose sole purpose is to do this: a Lisp interpreter running on the metal, which runs Emacs, and inside that, you have all the other functions. No distinction between "OS" and "apps", or between different apps. The OS runs the HLL you're writing natively, so there's no interpreter or compiler or linker. Everything you see is drawn by your editor and is live code that is executing in the environment - you can, if you wish, tweak the email function or the file manager to your taste, or if you want, go grab a new one off the Internet and plug it in instead.
The environment is modular and extensible by design, so if you want a spreadsheet function, you grab a file off a remote server onto your machine and suddenly you can now manipulate grids of numbers. Want charting? Grab a charting module. No binaries, no installing, no libraries - it's all one big (multithreaded?) program.
The online help is another program - you can use a shell to search its database, or edit it, or view it, as you wish.
That's a half-assed crappy description of the LispM environment at my current meagre level of understanding. Everything's live, it's all right there to be inspected and called upon.
Compare with the Unix model: lots of tiny VMs, some big, some small, calling static compiled binaries which link to thousands of other static compiled binaries. Much quicker, but everything is walled off from everything else by design. They're all written in different languages, all configured by different text files in different formats in different places. Yes, there's a `wc` command to count words in a plain text file, but it can't do it on a Word file, and if you install LibreOffice, it has a different word count function, and if you install Abiword, it has a different one too. You can't use Abiword's functions in LibreOffice or vice versa - they're separate binaries. Both use Gtk so they look alike, unlike KWrite which reduplicates all that functionality on a different set of foundation libraries - all 3 in C++ but that makes no difference at all.
If you install GNUstep and get Ink, it can't interoperate with them or share functions, and it's written in Objective-C.
Other bits of your OS are in Ocaml and Perl and Python and Ruby and shell and Tcl and who knows what, some interpreted, some compiled.
And while there are elaborate systems of protocols for GNOME apps to communicate with different GNOME apps, they don't talk to KDE apps, and neither talk to GNUstep apps. All have to drop down to a very basic level of folders with text files in, because that's the highest abstraction that they share.
And that structure of text files is different from the ones that Windows and Mac OS use. It's even a bit different to the different ones that FreeBSD and NetBSD and OpenBSD and DragonflyBSD use. So different that you can't even use maildir on Windows, because Windows' filesystem conventions are too different and prohibit it, so no cross-platform mail client like Thunderbird can use maildir because it's incompatible with the dominant OS.
It's a really simple, clear level of abstraction, but it's /old/ and very very limited - so each different implementation of it does it differently.
And now that we have rich GUI layers on top, they've reinvented it all again, multiple different times, so now, there are umpteen different desktops and they barely talk to each other.
It's simple and it's efficient and there was a time when that was all-important, so once these simple OSes could do the fancy stuff that LispMs and Amigas and Macs etc. could do, they out-competed them.
Then they got into a bunfight of trying to outcompete each other, and Linux came along and swept all that away with Free, open code and just outcompeted the hell out of everyone. All the proprietary ones are now slowly dying: vestiges live on on servers, nothing else.
Windows is going to go the same way next. Apple will follow once the free stuff is good enough.
And what we're left with is an arcane agglomeration of weird crusty old Unix stuff:
<<
“The Jolitzes believed that from its origins as a series of quick, if elegant, hacks, Unix had hardened into a series of unquestioned rituals for getting things done. Many of these rituals were fossils — work-arounds for hardware that no longer existed. “It’s amazing how much we were still tied to the past,” Lynne says. “The physical machines had been hauled away, but elements of the operating systems are still being implemented in the same way.” She believes today’s Linux and BSD developers are carrying on that unhelpful tradition.” “It’s like they’re doing incantations,” she says. “They’re repeating what they’ve been taught, and they don’t know what it means.”
>>
"Unix, the Living Anachronism": http://www.loper-os.org/?p=34
The people who /invented/ Unix moved on, 30y ago - but the inertia of Unix was already too strong, and Unix 2 (called Plan 9) and Unix 3 (Inferno) failed to attract any momentum.
In the mid-1980s, another trendy idea nearly changed things: microkernels, carving Unix up into manageable chunks. There have been 3 notable efforts that escaped academia and obscurity.
#1: Mach
Widely adopted at the time; only one version prospered, NeXTstep.
Not really a microkernel any more: to achieve Unix compatiblity, they built most of the BSD kernel into their microkernel as a "Unix server", meaning the result is just a slightly odd Unix.
#2: QNX
Unix-like, proprietary, hugely commercially successful in embedded systems but minimal impact on the wider market. Nearly happened 3 times over: [1] it was nearly the basis of the next-generation Amiga, then [2a] it was the basis of RIM's doomed tablet, and now [2b] it's the basis of Blackberry Inc's really rather good smartphone - but just not Good Enough to successfully attack the entrenched market leaders Apple and Android.
#3: Minix 3
A real live working FOSS microkernel Unix from the guy who originally pointed out that "Linux is obsolete".
Minix 3 is what the GNU Hurd tried and failed to be, only it's finished and it works fine.
Very clever, but too late to make a real difference. Part of me would like to see Minix 3 replace Linux in the fullness of time, when the kernel becomes just unmaintanable - but unfortunately, tools are advancing to keep it viable to manage such an obscenely huge monolithic bolus of C.
So monolithic Unix has lumbered on, a very good metaphorical approximation of Frankenstein's Monster: a stitched-together patchwork of different bits, ugly but so big and strong that it is thriving.
The next best thing to The Son of Unix was BeOS. A brave attempt, to keep the notions and concepts of Unix (like Posix and the shell and so on), couple them with the open hackability and rich media support of the Amiga, and the graphical attractiveness & simplicity of the Mac. It was brilliant: fast, sleek, gorgeous, friendly and easy. I loved it.
But it too failed, partly because MS stabbed it in the back (the Hitachi bundling deal and strong-arm tactics over OEM licensing, a dirty little episode now mostly forgotten) but mostly because although it was good, it wasn't good /enough./
If Apple had bought it, Apple would be a fading memory now.
What RIM, sorry, Blackberry is doing with QNX now seems to me to be a very good parallel to what Be tried and failed to do. It's very good, but it's not so astoundingly good as to be compelling.
*Nothing* is strong enough to bring down Torvalds' Monster, but the Monster *is* strong enough to bring down the other shambling behemoths: Windows NT and The Son of Mach, Apple OS X and iOS.
We're heading for another monoculture, but this time, a FOSS one. It's an improvement, I guess, but not much of one.
And I don't think anyone can outdo Linux by trying to be better at being a Unix-type system. I don't think it's possible. Haiku is a FOSS BeOS, a better small sleek media-friendly GUI desktop OS - but Linux is good enough at that. OS X is better at being a smooth, easy, friendly, desktop Unix - but Ubuntu actually brings you most of that, for free, but as per /In The Beginning was the command line/, they literally can't give it away.
<<
These are not old-fashioned, cast-iron Soviet tanks; these are more like the M1 tanks of the U.S. Army, made of space-age materials and jammed with sophisticated technology from one end to the other. But they are better than Army tanks. They've been modified in such a way that they never, ever break down, are light and maneuverable enough to use on ordinary streets, and use no more fuel than a subcompact car. These tanks are being cranked out, on the spot, at a terrific pace, and a vast number of them are lined up along the edge of the road with keys in the ignition. Anyone who wants can simply climb into one and drive it away for free.
[...]
The group giving away the free tanks only stays alive because it is staffed by volunteers, who are lined up at the edge of the street with bullhorns, trying to draw customers' attention to this incredible situation.
>>
http://pauillac.inria.fr/~weis/info/commandline.html
Windows is trying desperately to jump over the fence and be a pretty, smoother, more integrated Android but with Apple's device-plus-online-store model. It's failing at both.
In the form of Android, Linux is out-iOS-ing Apple and out-fondleslabbling Microsoft.
It's the Thing, it's the Blob. Whatever you do, it can just copy you and do it well enough. It is Legion and it spawns Pod People at a vast rate.
But it's not pretty or elegant or small or streamlined. It's a huge Lovecraftian agglomeration with eyes and mouths and other distressing moist orifices all over, but the cheap commodity hardware has come up to meet it and now it runs well enough to be pleasant on hardware that costs £25 for the main board and £40 for a whole tablet.
So if you can't take it on at its own game, is there any other way to take it on? What does it not do? What's it bad at?
Well, it's very big and complex and arcane. Something smaller and simpler that doesn't do lots of different things, but does one thing really well, that might work.
It's composed of lots of different types of system - a modern desktop Linux contains bits written in literally dozens of different speciality languages, all very good at their one narrow purpose and quite poor at other things, some compiled, some interpreted.
So that might be a clue: you might want to use a language that is versatile and excels at mutating on the fly to make little sub-languages that are good for their narrow task. Domain-specific languages, in other words.
It's a Unix, meaning that it's a system that excels at juggling lots of isolated processes, all sealed away from one another to variable degrees. That's how it works.
So perhaps, a response might be a system that does the reverse: that keeps everything inside one process, written in one language, and shares code as much as possible.
Linux is the ultimate embodiment of the config-file based system: where all state is kept in myriads of little plain-text files, which are read at bootup or when processes are launched and determine how everything hangs together.
And yet, at the moment, I'm refurbishing some 1980s classic 680x0 Macs, which are usable multitasking GUI computers, with rich apps and are rudimentarily WWW-capable, which don't have a single configuration file on the entire OS. Not one, anywhere. Nothing. Lots of metadata, yes, but the OS manages a database of it, invisibly. Developers need to handle it, but users cannot even see that it is there.
I'm not saying this is a better way, but it certainly shows that something that is utterly integral to Windows, Unix and OS X in all their forms is not actually necessary. You can do without it and the result is better for the user.
But the Mac has a filesystem, binaries and data files, programs and libraries, and it doesn't multitask well. The Newton dispensed with those concepts as well and worked very well without them - in a far smoother, more attractive way than the iPad does today.
I cannot point to something and say "look, this is what we should do." But I can point to various elements of various systems and say "look, they did this stuff better than anything we have today."
But when I do, I get pooh-poohed. There is this entrenched belief that we live in the best of all possible worlds: that modern computers are wonderful and just getting better.
And they really are not. The emperor really is stark naked, but nobody will listen.
Look at it from a different angle. For many a working programmer (and writer, believe me), the WWW is a massive distraction. A necessary one, but one that is separate.
What do you need to do, if you are co-working on a significant project? Edit code, obviously. Save it, compile it, run it, probe it with debuggers. Navigate your filesystem, load and view other files, move stuff between them. Occasionally, read and write email, or IRC, or (more historically) newsgroups, to discuss what you're doing. Possibly retrieve files from remote servers or put them there.
The point being, Emacs has extensions to do all this, so that you can do it all in a consistent fashion in a consistent (if horrible) UI, so that you can spend your entire day inside a single Emacs session and never leave and thus mentally never have to change gear.
Emacs, for some of its fans, is their entire OS. Their computer runs something that lets them launch Emacs and has an accessory function of browsing the web.
I don't do this - I don't speak Emacs at all - but I know people who do and really like it.
Well, turn that inside out. Consider an OS whose sole purpose is to do this: a Lisp interpreter running on the metal, which runs Emacs, and inside that, you have all the other functions. No distinction between "OS" and "apps", or between different apps. The OS runs the HLL you're writing natively, so there's no interpreter or compiler or linker. Everything you see is drawn by your editor and is live code that is executing in the environment - you can, if you wish, tweak the email function or the file manager to your taste, or if you want, go grab a new one off the Internet and plug it in instead.
The environment is modular and extensible by design, so if you want a spreadsheet function, you grab a file off a remote server onto your machine and suddenly you can now manipulate grids of numbers. Want charting? Grab a charting module. No binaries, no installing, no libraries - it's all one big (multithreaded?) program.
The online help is another program - you can use a shell to search its database, or edit it, or view it, as you wish.
That's a half-assed crappy description of the LispM environment at my current meagre level of understanding. Everything's live, it's all right there to be inspected and called upon.
Compare with the Unix model: lots of tiny VMs, some big, some small, calling static compiled binaries which link to thousands of other static compiled binaries. Much quicker, but everything is walled off from everything else by design. They're all written in different languages, all configured by different text files in different formats in different places. Yes, there's a `wc` command to count words in a plain text file, but it can't do it on a Word file, and if you install LibreOffice, it has a different word count function, and if you install Abiword, it has a different one too. You can't use Abiword's functions in LibreOffice or vice versa - they're separate binaries. Both use Gtk so they look alike, unlike KWrite which reduplicates all that functionality on a different set of foundation libraries - all 3 in C++ but that makes no difference at all.
If you install GNUstep and get Ink, it can't interoperate with them or share functions, and it's written in Objective-C.
Other bits of your OS are in Ocaml and Perl and Python and Ruby and shell and Tcl and who knows what, some interpreted, some compiled.
And while there are elaborate systems of protocols for GNOME apps to communicate with different GNOME apps, they don't talk to KDE apps, and neither talk to GNUstep apps. All have to drop down to a very basic level of folders with text files in, because that's the highest abstraction that they share.
And that structure of text files is different from the ones that Windows and Mac OS use. It's even a bit different to the different ones that FreeBSD and NetBSD and OpenBSD and DragonflyBSD use. So different that you can't even use maildir on Windows, because Windows' filesystem conventions are too different and prohibit it, so no cross-platform mail client like Thunderbird can use maildir because it's incompatible with the dominant OS.
It's a really simple, clear level of abstraction, but it's /old/ and very very limited - so each different implementation of it does it differently.
And now that we have rich GUI layers on top, they've reinvented it all again, multiple different times, so now, there are umpteen different desktops and they barely talk to each other.
It's simple and it's efficient and there was a time when that was all-important, so once these simple OSes could do the fancy stuff that LispMs and Amigas and Macs etc. could do, they out-competed them.
Then they got into a bunfight of trying to outcompete each other, and Linux came along and swept all that away with Free, open code and just outcompeted the hell out of everyone. All the proprietary ones are now slowly dying: vestiges live on on servers, nothing else.
Windows is going to go the same way next. Apple will follow once the free stuff is good enough.
And what we're left with is an arcane agglomeration of weird crusty old Unix stuff:
<<
“The Jolitzes believed that from its origins as a series of quick, if elegant, hacks, Unix had hardened into a series of unquestioned rituals for getting things done. Many of these rituals were fossils — work-arounds for hardware that no longer existed. “It’s amazing how much we were still tied to the past,” Lynne says. “The physical machines had been hauled away, but elements of the operating systems are still being implemented in the same way.” She believes today’s Linux and BSD developers are carrying on that unhelpful tradition.” “It’s like they’re doing incantations,” she says. “They’re repeating what they’ve been taught, and they don’t know what it means.”
>>
"Unix, the Living Anachronism": http://www.loper-os.org/?p=34
The people who /invented/ Unix moved on, 30y ago - but the inertia of Unix was already too strong, and Unix 2 (called Plan 9) and Unix 3 (Inferno) failed to attract any momentum.
In the mid-1980s, another trendy idea nearly changed things: microkernels, carving Unix up into manageable chunks. There have been 3 notable efforts that escaped academia and obscurity.
#1: Mach
Widely adopted at the time; only one version prospered, NeXTstep.
Not really a microkernel any more: to achieve Unix compatiblity, they built most of the BSD kernel into their microkernel as a "Unix server", meaning the result is just a slightly odd Unix.
#2: QNX
Unix-like, proprietary, hugely commercially successful in embedded systems but minimal impact on the wider market. Nearly happened 3 times over: [1] it was nearly the basis of the next-generation Amiga, then [2a] it was the basis of RIM's doomed tablet, and now [2b] it's the basis of Blackberry Inc's really rather good smartphone - but just not Good Enough to successfully attack the entrenched market leaders Apple and Android.
#3: Minix 3
A real live working FOSS microkernel Unix from the guy who originally pointed out that "Linux is obsolete".
Minix 3 is what the GNU Hurd tried and failed to be, only it's finished and it works fine.
Very clever, but too late to make a real difference. Part of me would like to see Minix 3 replace Linux in the fullness of time, when the kernel becomes just unmaintanable - but unfortunately, tools are advancing to keep it viable to manage such an obscenely huge monolithic bolus of C.
So monolithic Unix has lumbered on, a very good metaphorical approximation of Frankenstein's Monster: a stitched-together patchwork of different bits, ugly but so big and strong that it is thriving.
The next best thing to The Son of Unix was BeOS. A brave attempt, to keep the notions and concepts of Unix (like Posix and the shell and so on), couple them with the open hackability and rich media support of the Amiga, and the graphical attractiveness & simplicity of the Mac. It was brilliant: fast, sleek, gorgeous, friendly and easy. I loved it.
But it too failed, partly because MS stabbed it in the back (the Hitachi bundling deal and strong-arm tactics over OEM licensing, a dirty little episode now mostly forgotten) but mostly because although it was good, it wasn't good /enough./
If Apple had bought it, Apple would be a fading memory now.
What RIM, sorry, Blackberry is doing with QNX now seems to me to be a very good parallel to what Be tried and failed to do. It's very good, but it's not so astoundingly good as to be compelling.
*Nothing* is strong enough to bring down Torvalds' Monster, but the Monster *is* strong enough to bring down the other shambling behemoths: Windows NT and The Son of Mach, Apple OS X and iOS.
We're heading for another monoculture, but this time, a FOSS one. It's an improvement, I guess, but not much of one.
And I don't think anyone can outdo Linux by trying to be better at being a Unix-type system. I don't think it's possible. Haiku is a FOSS BeOS, a better small sleek media-friendly GUI desktop OS - but Linux is good enough at that. OS X is better at being a smooth, easy, friendly, desktop Unix - but Ubuntu actually brings you most of that, for free, but as per /In The Beginning was the command line/, they literally can't give it away.
<<
These are not old-fashioned, cast-iron Soviet tanks; these are more like the M1 tanks of the U.S. Army, made of space-age materials and jammed with sophisticated technology from one end to the other. But they are better than Army tanks. They've been modified in such a way that they never, ever break down, are light and maneuverable enough to use on ordinary streets, and use no more fuel than a subcompact car. These tanks are being cranked out, on the spot, at a terrific pace, and a vast number of them are lined up along the edge of the road with keys in the ignition. Anyone who wants can simply climb into one and drive it away for free.
[...]
The group giving away the free tanks only stays alive because it is staffed by volunteers, who are lined up at the edge of the street with bullhorns, trying to draw customers' attention to this incredible situation.
>>
http://pauillac.inria.fr/~weis/info/commandline.html
Windows is trying desperately to jump over the fence and be a pretty, smoother, more integrated Android but with Apple's device-plus-online-store model. It's failing at both.
In the form of Android, Linux is out-iOS-ing Apple and out-fondleslabbling Microsoft.
It's the Thing, it's the Blob. Whatever you do, it can just copy you and do it well enough. It is Legion and it spawns Pod People at a vast rate.
But it's not pretty or elegant or small or streamlined. It's a huge Lovecraftian agglomeration with eyes and mouths and other distressing moist orifices all over, but the cheap commodity hardware has come up to meet it and now it runs well enough to be pleasant on hardware that costs £25 for the main board and £40 for a whole tablet.
So if you can't take it on at its own game, is there any other way to take it on? What does it not do? What's it bad at?
Well, it's very big and complex and arcane. Something smaller and simpler that doesn't do lots of different things, but does one thing really well, that might work.
It's composed of lots of different types of system - a modern desktop Linux contains bits written in literally dozens of different speciality languages, all very good at their one narrow purpose and quite poor at other things, some compiled, some interpreted.
So that might be a clue: you might want to use a language that is versatile and excels at mutating on the fly to make little sub-languages that are good for their narrow task. Domain-specific languages, in other words.
It's a Unix, meaning that it's a system that excels at juggling lots of isolated processes, all sealed away from one another to variable degrees. That's how it works.
So perhaps, a response might be a system that does the reverse: that keeps everything inside one process, written in one language, and shares code as much as possible.
Linux is the ultimate embodiment of the config-file based system: where all state is kept in myriads of little plain-text files, which are read at bootup or when processes are launched and determine how everything hangs together.
And yet, at the moment, I'm refurbishing some 1980s classic 680x0 Macs, which are usable multitasking GUI computers, with rich apps and are rudimentarily WWW-capable, which don't have a single configuration file on the entire OS. Not one, anywhere. Nothing. Lots of metadata, yes, but the OS manages a database of it, invisibly. Developers need to handle it, but users cannot even see that it is there.
I'm not saying this is a better way, but it certainly shows that something that is utterly integral to Windows, Unix and OS X in all their forms is not actually necessary. You can do without it and the result is better for the user.
But the Mac has a filesystem, binaries and data files, programs and libraries, and it doesn't multitask well. The Newton dispensed with those concepts as well and worked very well without them - in a far smoother, more attractive way than the iPad does today.
I cannot point to something and say "look, this is what we should do." But I can point to various elements of various systems and say "look, they did this stuff better than anything we have today."
But when I do, I get pooh-poohed. There is this entrenched belief that we live in the best of all possible worlds: that modern computers are wonderful and just getting better.
And they really are not. The emperor really is stark naked, but nobody will listen.