tech stuff.

OpenBSD Install via IPMI on SuperMicro Server

leave a comment »

This is a step-by-step guide for how I remotely installed OpenBSD 5.0 to my SuperMicro X8SIE-LN4F based server.  This includes setting up IPMI and remote serial console.

  1. Boot the server and set a static IP for the IPMI Configuration in the BIOS setup.  I guess this might not apply to you, but I like it.
  2. Connect to the IPMI web server, upload your SSL certificates, set your admin user passwords, etc.
  3. Constrain IPMI to the Dedicated LAN interface.  The BMC on this server supports two modes of operation.  It has a dedicated 100BaseTX port, but it can also piggy back (with optional VLAN tagging) the first system ethernet port.  By default the system comes configured to do some sort of failover between the two.  It will use the dedicated LAN port if connected and piggy back the system port otherwise, but I’m not sure of the exact failover behavior.  I don’t like this behavior, though, because it increases the risk of having the BMC sneaking onto my front-end network and exposing its relatively insecure IPMIness to a broader audience.You can disable this behavior by setting LAN Interface to Dedicate under Configuration -> Network in the web UI.
  4. Configure console redirection in BIOS.  Launch the Java-based remote console and enter the BIOS Setup by pressing Delete.  This is where we are going to configure console redirection for the install process. Under Advanced -> Remote Access Configuration, set the following:
    • Remote Access = Enabled
    • Serial Port Number = COM3 *
    • Serial Port Mode = 115200 8,n,1
    • Redirection after BIOS POST = Always

    Pay special attention to the Serial Port Number field. It should have an asterisk next to the port name. This asterisk represents the serial port used by the BMC. Redirection after BIOS POST should be set to Always if you’re going to use SOL to do the install.  Otherwise you can set it to Boot Loader.

  5. Install OpenBSD.  It’s easiest to do the initial install via the java remote console and enable the serial console after booting into the OS. If that option isn’t available to you, though, you can do a full SOL install with these steps.  Make sure you set Redirection after BIOS POST to Always.
    1. Connect the Serial-over-LAN via ipmitool. Like this, for example: ipmitool -I lanplus -U admin -a -H 10.0.0.5 sol activate
    2. Insert your OpenBSD CD, set your boot priority and reboot. Stop the bootloader at the boot> prompt by pressing any key. Now we’re going to change the console to the serial port, thusly:
      boot> stty com2 115200
      boot> set tty com2

      Pay attention to the number convention here. We’re using the third serial port, which the AMI BIOS calls COM3, but OpenBSD names com2.
    3. Install OpenBSD as per your preference. If using AMD64, you’ll need to immediately interrupt the bootloader via SOL, set tty pc0, and rely on BIOS Console redirection to perform the following steps.
  6. Add a third serial device to the GENERIC kernel. As of OpenBSD 5.0, the amd64 GENERIC kernel only comes preconfigured for 2 serial ports. You should be able to skip this step with i386. Rather than recompile the kernel, you can just use config(8) to clone an existing serial device. It goes like this:
    # config -o bsd.new -e bsd
    OpenBSD 5.0 (GENERIC.MP) #63: Wed Aug 17 10:14:30 MDT 2011
    deraadt@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
    Enter 'help' for information
    ukc> find com
    196 com* at puc* port -1 flags 0x0
    206 com* at pcmcia* function -1 irq -1 flags 0x0
    218 com0 at isa0 port 0x3f8 size 0 iomem -1 iosiz 0 irq 4 drq -1 drq2 -1 flags 0x0
    219 com1 at isa0 port 0x2f8 size 0 iomem -1 iosiz 0 irq 3 drq -1 drq2 -1 flags 0x0
    ukc> add com2
    Clone Device (DevNo, 'q' or '?') ? 219
    Insert before Device (DevNo, 'q' or '?') ? 220
    220 com2 at isa0 port 0x2f8 size 0 iomem -1 iosiz 0 irq 3 drq -1 drq2 -1 flags 0x0
    ukc> change com2
    220 com2 at isa0 port 0x2f8 size 0 iomem -1 iosiz 0 irq 3 drq -1 drq2 -1 flags 0x0
    change [n] y
    port [0x2f8] ? 0x3e8
    size [0] ?
    iomem [-1] ?
    iosiz [0] ?
    irq [3] ? 5
    drq [-1] ?
    drq2 [-1] ?
    flags [0] ?
    220 com2 changed
    220 com2 at isa0 port 0x3e8 size 0 iomem -1 iosiz 0 irq 5 drq -1 drq2 -1 flags 0x0
    ukc> find com
    196 com* at puc* port -1 flags 0x0
    206 com* at pcmcia* function -1 irq -1 flags 0x0
    218 com0 at isa0 port 0x3f8 size 0 iomem -1 iosiz 0 irq 4 drq -1 drq2 -1 flags 0x0
    219 com1 at isa0 port 0x2f8 size 0 iomem -1 iosiz 0 irq 3 drq -1 drq2 -1 flags 0x0
    220 com2 at isa0 port 0x3e8 size 0 iomem -1 iosiz 0 irq 5 drq -1 drq2 -1 flags 0x0
    ukc> quit
    Saving modified kernel.

    /bsd.new is the modified kernel. You can boot it to test by typing bsd.new at the boot> prompt. When you boot this kernel, OpenBSD will configure all three serial ports and log it. That will look something like this:

    # dmesg | grep com
    com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
    com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
    com2 at isa0 port 0x3e8/8 irq 5: ns16550a, 16 byte fifo

    If everything looks good you can make it your default kernel:

    # ln bsd bsd.gen && mv bsd.new bsd
  7. Switch console to serial device. This is fully documented here, but here are the basic steps for this exercise:
    1. Add the following lines to /etc/boot.conf:
      stty com2 115200
      set tty com2
    2. Enable /dev/tty02 in /etc/ttys:
      # grep tty02 /etc/ttys
      tty02 "/usr/libexec/getty std.115200" vt220 on secure
  8. Disable post-boot BIOS console redirection. From this point on, OpenBSD will handle the serial devices directly, and it will do a better job than the BIOS VGA redirection. To make sure everything runs smoothly, disable the post-boot console redirection in the BIOS by setting Redirection after BIOS POST to Boot Loader in Advanced -> Remote Access Configuration.
Advertisements

Written by Lee Verberne

2012/01/13 at 22:07

Posted in BSD

Tagged with

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: