Re: Workarounds for advanced RAID features in GRUB Legacy?

[Tim Woodall]

On Sat, 10 Sep 2005, Leif W wrote:

> > > Right now the cleanest option seems to be to simply put a separate ext2 
> > > partition at the beginning of each disc, have a kernel and initrd there to 
> > > handle the RAID setup, and install grub multiple times, and update 
> > > everything manually.  But I have SATA discs, which are treated like SCSI 
> > > discs, and thus
> >
> > What I've got is a raid1 /boot (md0) ext3 at the start of the disk and
> > then the rest of the disk is md1 which is then partitioned using LVM2.
> > (I'm just using raid1 on two IDE disks here)
>
> Well, even here, let's take a closer look.  An ext3, is that including a 
> separate partition for a journal device?  And a swap partition?  You don't 
No, the first partition on the disks is a raid1 partition with an ext3
filesystem installed to it but to grub it looks like ext2. So when grub
boots and read the menu.lst, initrd and kernel as if the disk was ext2.
The initrd then sets up the raid on md0 and md1.

> want to mirrow swap, and no need to make it RAID 0, as Linux swap is designed 
> to balance the disc utilization.  So we have journal, boot (RAID 1), swap, 
You might want to mirror swap if you are hoping your machine might stay
up in the event of a disk failure.

> and data (root, RAID 1).  That's 4 partitions already. I've got discs that

I can't really see the point in mixing raid1 with anything else on a
pair of disks but even so, I'd have

hda1/hdc1 128Mb /dev/md0 raid1 /boot ext3
hda2/hdc2 xGb   /dev/md1 raid1 LVM2 vg0
hda3/hdc3 yGb   /dev/md2 raid5 LVM2 vg1

And if you want swap then an extra swap partition which can either go in
hda4/hdc4 or have an extended partition.

> are huge, and can easily fit several OS installations, and need to have some, 
> all on the same discs.  But the SATA discs have 16 partition limit that I 
> bump into if everything is a flat level RAID 1.  That's my dilemma.  With 
I'm not sure I understand. Even if you have multiple linux installs,
provided you are prepared to share /boot you shouldn't need any more
partitions for linux.

> WinXP, it requires the DOS partition table, and is very picky about partition 
> types for a clean install.  And is further picky about partitions changing 
> later.  And can't handle SW-RAID for a single partition, only the full disc, 
> only using the BIOS RAID.
I don't run WinXP but this looks like a limitation in WinXP. If WinXP
requires you to raid the entire disk then that is what you are going to
have to do for WinXP. I suspect, however, that if you raid the whole
disk but then in linux treat the end of each disk as non-raid and then
run software raid in linux it will still all work correctly. (for raid1
at least although you definitely want to check that rebuilding the
mirror in windows doesn't break the linux raid)

> > And then my menu.lst looks something like this:
>
> > <snip>
>
> > You still need to install grub into the boot sector on both disks but
> > once that is done then any updates will update both drives (except for
> > updating the boot sector or partition table)
> >
> > If either drive fails then the system will boot from the other one (you
> > might need to unplug the failed disk to make the bios happy)
>
> This looks similar to some other configs that I've seen and I'll try this 
> technique as well.  The other technique I saw suggested using the 'device' 
> command to assign subsequent discs to hd0.  The thing is, I don't want to 
> have to pull out plugs.  I mean, if something fails, granted I need to power 
> down and pull out plugs and swap the drives. But maybe I don't have time at

It's not unusual when a disk fails, expecially IDE disks, for it to
crash the machine hard so you are going to have to reboot. And it's also
not unusual for the bios to crash if there is a failed disk meaning you
can't boot until you unplug the failed disk. (Almost all my experiences
of failed disks have either been bad sectors (which hopefully raid1 will
compensate for) or failed to start after a shutdown (where bios hanging
isn't uncommon) - but it's not like I'm in a datacentre (which would all
be SCSI anyway) with disks failing all the time so my experiences may
not be typical

If you are very lucky the machine will stay up, mdadm will flag the disk
bad and you will get a message. A bit less lucky and the machine will
crash but a power cycle will be sufficient. Worst case, you will have to
unplug the cable from the failed disk in order to get everything up
again. Once you have the new disk you can install inplace of the failed
disk and rebuild the raid (or have another spare)



> the moment, and just need to limp along until I have sufficient time to 
> obtain a disc (save money, purchase, have it delivered, test it before using, 
> and so on).  But bills need to be paid, data needs to be accessed.  I may not 
> want to fiddle with my drives until I have the replacement parts.  Or the 
> other situation, maybe everything is fine, but I goof up a config file.  I 
> don't want to have to be pulling plugs to get back in.  I don't want to cause 
> excess wear on my plugs.  :-D  So, a solution I'm hoping to find will just 
> either try the next device/disc/partition/raid in a list, or I'll just settle 
> for hitting a down arrow to go to the next menu item.

Assuming you say mess up your grub-install on (hd0) then the menu.lst I
included will allow you to select (hd1). And if (hd0) fails to boot it
will automatically failover to (hd1) anyway.

But almost anything else (messing up a config file) is going to render
your machine unbootable regardless. Assuming that the disks are both ok
and raid is working ok then there is always going to be exactly the same
data and config files on each disk. (Of course, I have LinuxOLD-(hd0)
and LinuxOLD-(hd1) in my menu.lst incase a new kernel install fails to
boot and I automatically failover to these too.)

> > All of this is done using stock debian (sarge).
> >
> > I've considered playing with making the entire disk raid1, including the
> > boot sector and partition table but I've never got around to it and I'm
> > not certain it can be made to work (easily).
>
> Yeah I've been looking at this.  Boot on RAID1.  It's fairly well described 
> in the Software RAID HOWTO, but there are some gaps missing, like anything. 
> And the GRUB specific config isn't too in-depth.  For me personally, it just 
> feels like ugly workarounds, not some very elegant solutions.  So now I'm 
> considering the initrd on boot partition method, but might need loopback 
> support and nice to have CramFS support, but ext2 might suffice.
I've got boot on raid1 and then LVM on a separate raid1. What I would
like is to just have a single raid1 which then contained two partitions,
one for /boot and one for LVM. And the initrd is CramFS (default debian
initrd)

What this would mean is that running fdisk /dev/md0 would then update
the partition table on both my disks. But I'm not sure that grub would
install to a raid device. Of course, if you do what I often do which is
to set it all up on just one disk with the other disk "missing" and then
add the second disk to the raid once everything is set up then
everything will be copied nicely and only if you ever need to install a
new grub will there be a problem. (Adding a new disk to replace a failed
disk will also get the partition table and boot sectors copied
correctly. With my current setup you have to remember to create the
partition table (you can't really forget) and grub install to the new
disk (which you can easily forget) otherwise the new disk will not be
bootable and the "pull cable from the failed disk" technique outlined
above won't work.

Tim.


--
God said, "div D = rho, div B = 0, curl E = - @B/@t, curl H = J + @D/@t,"
and there was light.

     http://tjw.hn.org/      http://www.locofungus.btinternet.co.uk/