Liam on Linux

Is it me or are Ubuntu's dependencies fundamentally broken in places?

I don't much like Evolution as an email client -- too like Outlook, of which I'm not a fan -- but I can't simply remove it all, because components ranging as deep as the GNOME Panel itself depend on it. Why? I can understand why one or 2 components might, such as the "About Me" applet -- although I don't like it - but why does half of the GNOME desktop need one email client?

Yet, on the other hand, I've just been installing an Asus Eee 900 for a friend. I installed Xubuntu, as it's rather lighter-weight than full-on Ubuntu and he will never need the full desktop. Then the special Array.org kernel. Then I added all the restricted components -- and there's another thing, which I'll get back to -- and checked them out. All fine.

Finally I added the Netbook Remix launcher.

But no, it doesn't work. This iconic launcher had no icons and no menu bar. Why? Because it uses GNOME for them. But does it depend on GNOME? No.

So I had to start over, wipe the machine and install "proper" Ubuntu. Three hours' work down the drain.

(Yes, I could have done "apt-get install ubuntu-desktop" and sucked in all the GNOME stuff... But that would probably have filled up the 4GB SSD of the Eee PC and even if it didn't, left me to clean up all the now-redundant Xfce stuff, Abiword, etc. Cleaner to start afresh.)

So what is going on? Why does a shell that needs GNOME and GNOME components such as the go-home panel applet and maximus and so on not depend on GNOME, whereas I can't remove an email client I don't want from my GNOME system because large chunks of GNOME depend on one particular desktop application?

And speaking of components and restricted ones, it is a real pain in the neck to have to manually add in ubuntu-restricted-extras on every single Ubuntu box I install. They are not "optional" if you want to use the Web; Flash, Java and so on are pretty much mandatory. So, come to that, are the w32codecs from the Medibuntu repository.

I am not American; I am not in the USA; I never plan to live in the USA. Yet as far as I know, I have to jump through hoops installing this stuff because they can't be included by United States laws. These do not apply to me. Other distros can be freely downloaded in Europe which do include these elements.

It's not just me. Ubuntu was founded and funded by a South African who lives in Britain and the company behind it is in incorporated in the Isle of Man in the British Isles. I am fortunate enough to have met Mr Shuttleworth and a fair few of the Ubuntu developers. Many of them, too, are European.

None of us are restricted by US law. Nor are Canadians or Mexicans or Uruguayans or Brazilians, all of whom are also American.

But for this one country, to meet its restrictions, Ubuntu is delivered crippled by the removal of all these essential components.

I am not some radical Stallmanite; I don't care about an all-Free-with-a-capital-F distribution. If I did, I'd be using Debian or GnewSense or something. Ubuntu is not a crusading Free distro, it's a no-nonsense get-the-job-done distro. I use it because it is easier and less cluttered than Suse or Mandriva and is not some rolling alpha-test like Fedora. (I used to. I spent five or six years fighting with RPM and got very tired of it indeed.) Oh, and unlike Xandros, it's fairly current and is updated regularly and is small-f free.

No, I am pragmatic; where the Free alternative is good enough, I'll use it. When it isn't, or if I need something non-Free, I'll use that instead.

Ubuntu needs those codecs, plugins and modules.

Can we not have an uncrippled version of Ubuntu for non-USA-citizens to download and use, without hassle overcoming US restrictions? It's even being sold in the shops now, yet even that version, as far as I know, doesn't include all the proprietary-content players.

It's gradually being realised that TCO is more than H/W + S/W. Virtualisation and thin clients are currently very popular: VDI is the Next Big Thing, Virtual Desktop Infrastructure - you run graphical terminals everywhere and lots of virtual XP machines on a few big servers, with the display being remotely used from said terminals. Easier than WinFrame or Terminal Server, more flexible - they don't all need to run the same image, people can have conflicting versions of apps and "their own" machine which can follow them around a network. They can remotely suspend it & come back to all their windows where they were and so on.

But this misses a number of points.

#1 You still need to buy all the Windows licences. Expensive.

#2 You need some seriously BFO servers to run hundreds of VDI instances. Expensive.

#3 If the server goes down, hundreds of users are screwed. Downtime -> expensive.

#4 The "thin clients" don't do much - no local processing to speak of - but they are each a capable computer in their own right, with local RAM and CPU and graphics and sound and some kind of OS, possibly booting from Flash. Not very expensive, but still a significant cost, and of course, it's inefficient to chuck away hundreds or thousands of PCs and replace 'em all.

What's the point of it all, then?

Well, it's not saving upfront costs. It's reducing maintenance costs. Your techies work in a datacentre, looking after a relatively few servers running lots of instances of near-identical Windows system images. They don't need to go into the field - terminals are interchangable and cheap enough that branches can carry a few spares. Cheap largely-unskilled techs can go out in the field and swap boxes and that's largely it, once the network's built.

This is actually taking off. Lots of big vendors pushing it hard: MS, VMware, Citrix, Parallels on the S/W front, Wyse etc. on the terminals and all the big server vendors are dribbling at the prospect.

But it's an awful lot of work just to reduce management costs and it's got snags of its own.

So. Alternatives? Using our favourite FOSS OS?

Linux desktops

Rolling out Linux onto all your desktops is not a big help. Yes, it saves licence costs, but each machine takes an hour or so to provision, there's still loads of local state, and you have to retrain all your techs to install and support hundreds of Linux installations. Nightmare. Very hard sell.

Linux thin clients or terminals or X terminals

Using Linux as the thin client OS - well, fine, but you still have loads of thin clients, with the concomitant server load. Doesn't solve anything, just maybe makes the thin clients a bit cheaper. You could roll up a bootable-CD thin client distro that didn't require any installation at all, but it doesn't help the problem of all those clients and all those big expensive servers.

Turning PCs into diskless workstations, only, er, with disks

Now, for years, I've been interested in stateless OSs for some modern kind of network workstation. Diskless workstations were a good idea once and I tend to think they could make more sense than "thin clients" which aren't thin, they're just graphical terminals. Terminals dump the entire computation (+ RAM + storage) load onto the server. RAM and storage are relatively cheap; it seems to me to make more sense to make the best possible use of local resources as well as reducing management. X.11 is no help here, it's just an alternative to MS RDP or Citrix ICA or whatever.

The ideal, ISTM, would be to have an identical, stateless OS image run out to all your PCs, something with no local config files at all. PCs would be totally interchangable; if they were buggered, they could be easily replaced.

Now companies like Progeny worked on this for ages, trying to separate out the local config from the global config and produce modular Linuxes that could keep just their important state on the server. It's a tricky job; AFAIK it never got done properly.

But ISTM that there is another way, using something that's already been done. Live CDs.

Live CDs are effectively stateless. They boot on completely unknown hardware, detect it on the fly on the first pass, and get you to a usable graphical desktop complete with apps and a network connection.

But they're slow, 'cos they run from CD, and CDs are slow.

Now with some LiveCD distros, like Knoppix, you can copy the CD contents to HD and it becomes a "normal" distro. But that's no good - that's local state again.

Other CD distros, like Puppy, can copy a write-only image of the CD to HD, boot off that via some kind of loader, and keep their state in a separate little file. That's more like it.

So what you need, ISTM, is a distro that works like Puppy - boots off an image on the HD, but as it boots, according to MAC address (say), it gets access to a bit of system-specific config data on the server. Screen resolution, stuff like that. This is a small package of data, and the tech to do that from local storage is known from several liveCDs. It just needs extending to work over a LAN. It broadcasts a request for a server and if it can "see" a server that knows that machine, it gets handed its local state info. This can be cached locally, outside the system image, for offline operation.

Then when the user logs in, they get a network home directory, so their desktop, their files, their settings etc. follow them around the network. Again, this can be cached locally. This is much the same as Windows' "roaming profiles", only 'cos Linux is a bit better designed, you don't sync a folder full of temporary files and the browser cache every time.

Now if you had a network of hundreds of machines all running off local images of the CD, that's an improvement, 'cos now you have a uniform stateless OS. However, when it comes round to upgrading them all, you're screwed again, because you need to update all those machines. Even if you write some snazzy utility to copy the image down off the network, a few hundred PCs loading a half-gig image is going to cripple your network and it will be disastrous over a slow link to a remote site.

So what you need is a pre-boot environment. Not something techie like PXE which not all PCs support, but you're going to need some kind of bootloader anyway.

So, you have a Linux system that loads a Linux system.

The first-stage loader boots up, gets an IP, connects to a specific disk image server and checks if the local image is current. If it isn't, it rsyncs down the latest one. (It might even make sense to keep a backup copy, in case of corruption or a lost connection during the sync. Grandfather-father-son. We're probably only talking about half a gig a pop and I don't think you can buy a HD smaller than 80GB these days. Even if you boot from SSD, 2G to 4G is something like £5 worth of Flash.)

Once the local image is current - and the sync shouldn't take long unless you do a whole-disk-image upgrade; we're probably talking 10s of meg, no more - then the startup kernel KEXECs the "live" kernel in the disk image. No reboot, so it's fast.

With network-bootable workstations, this is inefficient, but it could still work. However, with a small amount of local storage - a few gig - you have space for:

[a] the startup kernel and a simple FS to keep a couple of disk images in. This sort of job is simple enough that a bootable floppy would almost be enough - the system only needs to be a few meg of code. It needs a kernel, network drivers and rsync and not a lot else. This is the tech of the Swiss product Rembo, now bought by IBM and thus gone from the Free world.
http://www.appdeploy.com/tools/detail.asp?id=39
http://www-01.ibm.com/software/tivoli/products/prov-mgr-os-deploy/

[b] a swap partition, alleviating one of the performance problems of LiveCD operation. You could live without this if it got corrupted, lost or whatever.

[c] a small local cache partition containing a backup copy of any node-specific state and any user-specific state. Again, losing this wouldn't be fatal.

For provisioning machines, making spares etc., just leave a stack of bootable CDs of the live image around. Setting up a machine is a matter of turning it on, inserting the CD, pressing Reset or Ctl-Alt-Del if necessary, and waiting. The LiveCD boots as normal. In the background, a task kicks off and looks for an empty local drive. If it finds one, it installs the startup system, copies an image of the CD onto the disk, updates it from the server.

Indeed, you could even set it up so that like WUBI, the Ubuntu installer that copies itself into an existing Windows partition, it doesn't remove any existing data or anything on the hard disk, it just quietly inserts itself into any free space it finds, installs GRUB and makes itself the default with an option to boot back into the old Windows system if desired. Non-destructive reversible provisioning.

Inside the disk image, well, then it's a matter of making a simple generic desktop that looks as much like XP as humanly possible, with Firefox plus addins, an email client, OpenOffice, etc. All the usual stuff. The one extra I can see being really useful would be a bundled Terminal Server client, so that all those expensive-to-maintain PCs become identical thin clients that a trained monkey could set up, becoming completely interchangeable and so on. All the benefits of thin clients, all the remote manageability etc., because in effect, they're all booting off the image on the server - except without sending the whole OS and apps over the wire each time.

Ideally, if the Terminal Server client could work in a mode like Fusion on VMware on Mac OS X - where the remote apps' windows mingle in with local windows, so the users aren't presented with 2 desktops. They have one desktop, but some shortcuts or menu entries, unbeknownst to them, point to remote applications. E.g. the local Linux boxes could all run Outlook remotely to connect to the Exchange Server. Use something like Citrix or Parallels to host that, it would require /massively/ less server load than an entire virtual desktop image, and if you set it up right, you don't need hundreds of Windows licences - you just need lots of Outlook licences on a small number of copies of Windows.

What's the point of all this?

Well, it's a way to sell desktop Linux. This offers a set of (I hope) compelling advantages:

- massively reduced licence costs next to Windows, virtual or otherwise. Remember all those CALs you need even for thin clients.
- repurpose existing PCs into thin clients with a thin, light OS. Instead of proprietary thin clients, hardware replacements and upgrades use dirt-cheap generic hardware. Testing them for suitability just means booting them once from CD or USB.
- no conventional locally-installed OS, no local state, so PCs become interchangeable. In emergency, can just run from CD/USB key. Something goes wrong with a PC? Bung in the CD, reimage it, and user can work while it's happening.
- reduces network load compared to thin clients or network booting
- reduces server load compared to thin clients or virtualisation
- field support requires very little staff training or knowledge
- major reductions in management cost
- much less disruptive migration than replacement of existing kit
- greener: reuses existing machines & gets improved performance
- secure: no need for local antivirus etc. (so further cost savings)

ISTM that this could actually give Linux a persuasive advantage over Windows as a business OS.

So go on, rip the idea to shreds. What have I missed?

The main hard part is trying to sell it to Windows houses as an alternative. That's why I'd suggest part of it is a theme to make it look as much as possible like Windows, either XP or Vista, as desired. e.g.
http://lxp.sourceforge.net/

Is there a Linux version of Emacs with a grown-up 21st-century GUI? Something akin to Aquamacs on OS X:
http://aquamacs.org/

I've tried installing the GNUstep Emacs, but I couldn't get it to start. I've installed the vanilla Ubuntu Emacs 21 for X.11 instead, but it doesn't even seem to know what the scroll wheel on a mouse it. It just beeps at me.

Just curious. Considering (once again) learning a Real Man's Editor.

I am trying to backup a RAID array of some 100GB of stuff onto a FAT32 external hard drive.

I tried

tar -cvfz usbdrive/raid.tgz raid/

but it didn't like it. I also tried with 'j' instead of 'z' to use bzip2 instead of gzip. That created an archive called 'j' in the current directory.

So I dropped the compression switch altogether.

tar -cvf usbdrive/raid.tgz raid/

This worked but I forget one detail. FAT32 has a max file size of about 4G. So when the archive got to 4G, it barfed.

I see there's a parameter to change tapes, but I don't know if that allows me to create multi-segment archives. Can GNU tar do this? Break up the backup into lots of 4G chunks? Ideally, compressing them on the way? If so, how? I'm getting nowhere Googling for info...

dougs feels that Ubuntu's no-root-user model is irksome. I can certainly see his point. He feels that, even if it's a useful move for a workstation, for servers, this is an inappropriate setup. He suggests that we tell people to simply give root a useful password and use that.

I'm aware of some of the arguments against - for example, if you're good and dutiful and always use the sudo command, and don't do that naughty "sudo -s" and simply become root, then sudo gets logged, I believe, whereas what root does, whether you logged in as root or whether you sudo-ed and became root, it doesn't. All you get is an entry telling you that such-and-such became root and there the trail ends.

Thoughts? I'm going to have to go back and do some rewriting to change this, obviously, but then, I need to do some of that anyway.

I did indeed need to reconfig mdadm after building the array. This may be a difference between 6.10 and 6.06, which is what my older main server's running.

For the curious:

( Output of dmesg... )

I am slightly concerned about the initialisation errors on the RAID drives. They seem to be working fine. I thought it might be because I was using plain old IDE cables, but I've replaced them with 80-core UltraIDE cables: no difference. Odd. I've only seen this before on failing drives, but these errors are one-offs, only happening at startup. Otherwise, it seems fine.

Still, for the nonce, this is only a test box. It's not doing anything really important. So I am not too concerned...

Right. Some considerable poking later, I have:
- the server happily booting
- it's apt-getted into currency
- the screen in 132*60 mode (I've disabled Ubuntu's soft fonts - by marking the console-manipulating utility non-executable)
- I have replaced the duff HD in the array with the working one from Nolly's PC (he has a 40GB one in its place, so he's much better off)
- I have created a 4-disk RAID5 array
- formatted it as ext3
- fscked it to check that it seems healthy
- mounted it under /media/raid
- the remaining space on the 3 slightly larger drives partitioned as swap (roughly 40 + 80 + 80 meg. Plenty for a lightly-loaded 128MB RAM machine, I reckon).

Suggestions for:
- a cleaner way of disabling soft fonts
- making eth0 permanently DHCP
- making the RAID autostart, 'cos it currently must be done by hand

...?

I've been having a quick look through Froogle.

It looks like there are some reasonably affordable ATA RAID cards now...

http://froogle.google.co.uk/froogle?q=ide+raid+adaptor+card+-workstation+-tower+-barebone&btnG=Search&hl=en&show=dd&scoring=pd

Highpoint seem to do a range. IME - which is fairly limited, TBH - one of the distinguishing features of a "true" RAID controller is RAID5 support. If it only does 0/1/0+1, it's a firmware controller; if it does 5, it's probably the real deal. Agreed?

Anyone got any personal experience of these Highpoint controllers?

Everything <£50 looks to me like a firmware job... But there's at least 1 £75 card which looks to me like the genuine item:

http://www.scan.co.uk/Products/ProductInfo.asp?WebProductID=83203&source=froogle

...to the rambling buildings of Unseen University and in particular the apartments of Greyhald Spold, currently the oldest wizard on the Disc and determined to keep it that way.

He has just been extremely surprised and upset.

For the last few hours he has been very busy. He may be deaf and a little hard of thinking, but elderly wizards have very well-trained survival instincts, and they know that when a tall figure in a black robe and the latest in agricultural handtools starts looking thoughtfully at you it is time to act fast. The servants have been dismissed. The doorways have been sealed with a paste made from powdered mayflies, and protective octograms have been drawn on the windows. Rare and rather smelly oils have been poured in complex patterns on the floor, in designs which hurt the eyes and suggest the designer was drunk or from some other dimension or, possibly, both; in the very centre of the room is the eightfold octogram of Witholding, surrounded by red and green candles. And in the centre of that is a box made from wood of the curly-fern pine, which grows to a great age, and it is lined with red silk and yet more protective amulets. Because Greyhald Spold knows that Death is looking for him, and has spent many years designing an impregnable hiding place.

He has just set the complicated clockwork of the lock and shut the lid, lying back in the knowledge that here at last is the perfect defence against the most ultimate of all his enemies, although as yet he has not considered the important part that airholes must play in an enterprise of this kind.

And right beside him, very close to his ear, a voice has just said: DARK IN HERE, ISN'T IT?

Quick question.

Does anyone know of a simple text-mode text editor for Linux which resembles MS-DOS Edit or any other menu-driven CUA-style editor? I know about joe and pico, but I'm hoping for something easier...

Me, I use vi, but I don't like it much & only know the basics. Never got my head around EMACS - it's just too unlike anything else.

UPDATE: Sorted. I have found not only the rather pleasing WW, but now, SETEdit, which so far I like very much.

http://setedit.sourceforge.net/

Thanks for various hints & tips over in the other LJ!