![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
(no subject)
May. 14th, 2019 06:07 pmPowerQuest PartitionMagic was one of my favourite pieces of software ever written.
It offered a lot of functionality for disk and partition management on the PC that had previously been considered impossible, or the sole domain of enterprise storage management systems, such as resizing drive partitions on the fly -- i.e. with all their contents intact.
Later, it gained additional functionality, such as the ability to merge 2 (or more) disk partitions into one larger one.
If, for example, you merged drives C, D and E, you ended up with a big drive C which contained subfolders called "\D\" with the full contents of D: and "\E\" with the full contents of E:
It was then up to you to move stuff around to sort it.
However, the thing is this:
When you move from one drive to another drive, including separate partitions, the OS must copy the data from source to destination, then when it's copied, remove the original file... then repeat this for every file. This is unavoidably slow. It applies even on the same physical drive, if there are multiple partitions.
But if you move a file from one folder to another folder in the same partition, on any modern filesystem, the OS can just rename the file from
/data/my/old/file
... to...
/data/my/new/file
The actual contents of "file" don't move. So it's very, very fast.
So cleaning up the folders left by a PQMagic partition merge was quite quick. It was the merge that took hours. It copied as much data as would fit, shrank D: as much as possible by moving the start, enlarged C: and then copied some more... and repeat. This could be a *very* lengthy process.
This kind of thing is the reason that logical volume management systems exist:
https://en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux)
LVM is complicated and hard to understand. If the above article makes little sense, don't blame yourself. For standalone workstations, I recommend avoiding it.
So, there's LVM, then on top of the LVM space, you have partitions. Those are formatted with a filesystem, such as ext4, or older enterprise filesystems from old commercial Unixes, such as JFS (from IBM's AIX and OS/2), or XFS (from SGI IRIX).
https://en.wikipedia.org/wiki/XFS
https://en.wikipedia.org/wiki/JFS_(file_system)
Fedora enables LVM by default which is just one reason I avoid Fedora.
Then to make matters worse, there are filesystems which support "subvolumes" inside a partition, e.g. Btrfs.
https://en.wikipedia.org/wiki/Btrfs
Btrfs is the default FS of SUSE Linux.
Then you have subvolumes inside partitions on top of LVM volumes on top of disks, and personally it all makes my head spin.
*Because* LVM is hard, and its functionality overlaps with partitioning, there are projects that try to merge them.
For Linux, there was EVMS:
http://evms.sourceforge.net/
Unfortunately, it did not catch on, so we have LVM instead.
https://lwn.net/Articles/14816/
https://unix.stackexchange.com/questions/22885/is-there-a-more-modern-or-more-popular-version-of-evms2
RH does not support Btrfs. However, because it wants some of the features of Btrfs, RH is now building its own new combined logical volume manager / partitioner / filesystem, Stratis:
https://stratis-storage.github.io/
Stratis combines an LVM layer with the XFS filesystem.
I have heard comments that Stratis is in effect re-creating a subset of the functionality of EVMS.
This is a very typical Linux development path.
The richest filesystem/volume manager from commercial Unix is ZFS, from Sun (now Oracle) Solaris.
https://en.wikipedia.org/wiki/ZFS
Like JFS and XFS, ZFS is now open source. However, under a licence that is incompatible with the Linux kernel's GPL licence.
So you _can_ compile a Linux kernel with built-in XFS, but it violates the licence.
However, Ubuntu has found a way around this, with ZFS being a loadable module (AIUI) that isn't part of the kernel itself.
(AIUI. IANAL. Clarification welcome.)
Ubuntu Server offers ZFS instead, in place of Btrfs in SUSE or Stratis in Fedora (or XFS in all of them).
ZFS can replace the LVM _and_ also ext4/XFS/JFS, and therefore Stratis too, but neither SUSE nor RH will bundle ZFS because of licence concerns.
Apple _was_ going to bundle ZFS but it too decided the licensing was too tricky and it has developed its own system, APFS. But then Apple no longer is trying to compete in the server market.
https://en.wikipedia.org/wiki/Apple_File_System
Yes, it is confusing. Yes, it is a mess. Yes, there are too many standards.
https://xkcd.com/927/
It offered a lot of functionality for disk and partition management on the PC that had previously been considered impossible, or the sole domain of enterprise storage management systems, such as resizing drive partitions on the fly -- i.e. with all their contents intact.
Later, it gained additional functionality, such as the ability to merge 2 (or more) disk partitions into one larger one.
If, for example, you merged drives C, D and E, you ended up with a big drive C which contained subfolders called "\D\" with the full contents of D: and "\E\" with the full contents of E:
It was then up to you to move stuff around to sort it.
However, the thing is this:
When you move from one drive to another drive, including separate partitions, the OS must copy the data from source to destination, then when it's copied, remove the original file... then repeat this for every file. This is unavoidably slow. It applies even on the same physical drive, if there are multiple partitions.
But if you move a file from one folder to another folder in the same partition, on any modern filesystem, the OS can just rename the file from
/data/my/old/file
... to...
/data/my/new/file
The actual contents of "file" don't move. So it's very, very fast.
So cleaning up the folders left by a PQMagic partition merge was quite quick. It was the merge that took hours. It copied as much data as would fit, shrank D: as much as possible by moving the start, enlarged C: and then copied some more... and repeat. This could be a *very* lengthy process.
This kind of thing is the reason that logical volume management systems exist:
https://en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux)
LVM is complicated and hard to understand. If the above article makes little sense, don't blame yourself. For standalone workstations, I recommend avoiding it.
So, there's LVM, then on top of the LVM space, you have partitions. Those are formatted with a filesystem, such as ext4, or older enterprise filesystems from old commercial Unixes, such as JFS (from IBM's AIX and OS/2), or XFS (from SGI IRIX).
https://en.wikipedia.org/wiki/XFS
https://en.wikipedia.org/wiki/JFS_(file_system)
Fedora enables LVM by default which is just one reason I avoid Fedora.
Then to make matters worse, there are filesystems which support "subvolumes" inside a partition, e.g. Btrfs.
https://en.wikipedia.org/wiki/Btrfs
Btrfs is the default FS of SUSE Linux.
Then you have subvolumes inside partitions on top of LVM volumes on top of disks, and personally it all makes my head spin.
*Because* LVM is hard, and its functionality overlaps with partitioning, there are projects that try to merge them.
For Linux, there was EVMS:
http://evms.sourceforge.net/
Unfortunately, it did not catch on, so we have LVM instead.
https://lwn.net/Articles/14816/
https://unix.stackexchange.com/questions/22885/is-there-a-more-modern-or-more-popular-version-of-evms2
RH does not support Btrfs. However, because it wants some of the features of Btrfs, RH is now building its own new combined logical volume manager / partitioner / filesystem, Stratis:
https://stratis-storage.github.io/
Stratis combines an LVM layer with the XFS filesystem.
I have heard comments that Stratis is in effect re-creating a subset of the functionality of EVMS.
This is a very typical Linux development path.
The richest filesystem/volume manager from commercial Unix is ZFS, from Sun (now Oracle) Solaris.
https://en.wikipedia.org/wiki/ZFS
Like JFS and XFS, ZFS is now open source. However, under a licence that is incompatible with the Linux kernel's GPL licence.
So you _can_ compile a Linux kernel with built-in XFS, but it violates the licence.
However, Ubuntu has found a way around this, with ZFS being a loadable module (AIUI) that isn't part of the kernel itself.
(AIUI. IANAL. Clarification welcome.)
Ubuntu Server offers ZFS instead, in place of Btrfs in SUSE or Stratis in Fedora (or XFS in all of them).
ZFS can replace the LVM _and_ also ext4/XFS/JFS, and therefore Stratis too, but neither SUSE nor RH will bundle ZFS because of licence concerns.
Apple _was_ going to bundle ZFS but it too decided the licensing was too tricky and it has developed its own system, APFS. But then Apple no longer is trying to compete in the server market.
https://en.wikipedia.org/wiki/Apple_File_System
Yes, it is confusing. Yes, it is a mess. Yes, there are too many standards.
https://xkcd.com/927/
