tech stuff.

Software RAID on OpenBSD 4.8

with 2 comments

jpiasetz has a very good recipe on installing with software raid on OpenBSD 4.6, and so far I’ve had good success doing something very similar with OpenBSD 4.8. The biggest thing I changed was using wd1a rather than sd0a for /altroot. Then it’s easy enough to use daily(8)’s integrated altroot sync to keep altroot up-to-date.

Here’s the config for altroot:

% grep altroot /etc/fstab
/dev/wd1a /altroot ffs xx 0 0
% cat /etc/daily.local
ROOTBACKUP=1

It’s working well so far, but I haven’t tested the actual failure scenario yet.  I find this method much more satisfying than my earlier attempt at using raidframe for software raid (which I subsequently became afraid to upgrade remotely); this has much more of a “just works” feel to it.

Of course, this configuration won’t survive a primary disk failure without downtime, but for this server I prefer the simple usable solution over the complex available one.

Update 2010/03/16:

I created a VM to test a primary disk failure, and OpenBSD booted so smoothly that I think I might want to add some explicit monitoring for disk failure.

To test, I created a VM with two virtual disks, and installed using jpiasetz’s recipe with the following adjustments:

  • Removed the unnecessary -C force flag from bioctl
  • Use a mount point of none for sd0a
  • Removed the dd if=/dev/wd0a of=/dev/wd1a step

After rebooting, the steps to set up daily altroot sync are:

  1. Add altroot to fstab according to daily(8):
    # echo "/dev/wd1a /altroot ffs xx 0 0" >> /etc/fstab
  2. Enable root backup option in /etc/daily.local:
    # echo "ROOTBACKUP=1" >> /etc/daily.local
  3. Add root e-mail alias to catch the e-mailed /etc/daily output. (optional)
  4. Run the daily script, which will use dd to copy wd0a to wd1a:
    sh /etc/daily
  5. Install the boot block on wd1 so it can be used to boot the system in the absence of wd1:
    mount /dev/wd1a /altroot
    /usr/mdec/installboot -v /altroot/boot /usr/mdec/biosboot wd1
    umount /altroot

Then I powered down the VM and removed the wd0 disk entirely. When I booted, the VM came up with softraid0 in the degraded state (of course), but fully functional.

Important things to realize about this configuration:

  • / is not using RAID. The / partition is on wd0a and is copied byte-for-byte to wd1a every night by daily(8).  Disk failure will not be handled transparently, but should be easily recoverable.
  • All other partitions (except swap) use Software RAID
  • By default, OpenBSD will attempt to use both swap partitions on wd0b and wd1b, which may not be what you want.
  • installboot hard codes inode and block numbers and will need to be re-run if the file /boot changes — after an upgrade, for example.

Updated step-by-step guide here.

Advertisements

Written by Lee Verberne

2011/03/06 at 10:56

Posted in BSD

Tagged with

2 Responses

Subscribe to comments with RSS.

  1. I followed the jpiasetz post too, and it worked very well. I killed my second disk by zeroing it out with dd and then rebooting. When it came up, I ran “bioctl -s sd0”, and it showed the disk as offline. I rebuilt the array with
    “bioctl -R /dev/wd1d sd0”
    Worked like a charm – although it took about 24hrs to rebuild for a 500GB IDE disk.
    There is one thing I didn’t understand about the process – I’m glad you brought it up. How does using sd0 for /altroot as suggested even work? If you create a /altroot partition at the beginning of the sd0 disk, wouldn’t that overwrite the / partition you created on the physical wd0 disk? I must be missing something since this seems to work…

    Mitch

    2011/03/15 at 18:05

    • I suggest using wd1a as altroot instead of sd0a. sd0a goes unused in my example. This means that neither root nor altroot is using software raid, but that’s because OpenBSD doesn’t support root on softraid.

      To answer your question, though, the beginning of the sd0 disk and wd0/wd1 disks aren’t aligned. sd0 is wholly contained within the wd0d & wd1d partitions. That is, changes to sd0a will change data in wd0d, not wd0a.

      I’ve updated the post to include more details about what I did and how I tested it.

      verb

      2011/03/16 at 09:42


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: