Difference between revisions of "Diskless Frontend"

From MythTV Official Wiki
Jump to: navigation, search
(Tomato)
m (Added the wikipedia links for some abbreviations.)
Line 9: Line 9:
  
 
* First the machine requires a boot mechanism.  This can be [[PXE]], which is a feature of the network card to boot entirely from the network, a CD, USB key, or even a small hard drive used only to kickstart the boot process.
 
* First the machine requires a boot mechanism.  This can be [[PXE]], which is a feature of the network card to boot entirely from the network, a CD, USB key, or even a small hard drive used only to kickstart the boot process.
* If the machine is booting using PXE, the PXE boot code acquires a [[DHCP]]/[[BOOTP]] address from a DHCP server that contains information on where to find the bootloader.  This is similar to a [[Grub]] or [[LILO]] boot screen normally found on a Linux system.  This PXE bootloader defines where the init filesystem ([[Initrd]]) and the kernel are located and can be downloaded using [[TFTP]]
+
* If the machine is booting using PXE, the PXE boot code acquires a [[DHCP]]/[[BOOTP]] address from a DHCP server that contains information on where to find the bootloader.  This is similar to a [http://en.wikipedia.org/wiki/GRUB Grub] or [http://en.wikipedia.org/wiki/LILO_%28boot_loader%29 LILO] boot screen normally found on a Linux system.  This PXE bootloader defines where the init filesystem ([http://en.wikipedia.org/wiki/Initrd Initrd]) and the kernel are located and can be downloaded using [http://en.wikipedia.org/wiki/Tftp TFTP]
 
* If the machine is booting using a local media method, the bootloader on that media directs the boot environment to the init filesystem and the kernel.   
 
* If the machine is booting using a local media method, the bootloader on that media directs the boot environment to the init filesystem and the kernel.   
* Once the init filesystem and kernel have been downloaded and loaded onto a [[Ramdisk]], the remainder of the filesystem is mounted via [[NFS]] from the server.
+
* Once the init filesystem and kernel have been downloaded and loaded onto a [http://en.wikipedia.org/wiki/Ramdisk Ramdisk], the remainder of the filesystem is mounted via [[NFS]] from the server.
  
 
=== Creating a test system ===
 
=== Creating a test system ===

Revision as of 08:21, 12 October 2012

This page describes the various methods for creating diskless frontends on a variety of Linux distributions.

Introduction

The following How-To will assist you on building a Diskless Frontend for MythTV. The instructions are based on Fedora/Red Hat unless otherwise noted. This how to assumes a working machine that is configured to run MythTV, getting to this point is out of the scope of this doc. This system can be either a physical install on a hard drive or a Virtual Machine in VMware. Although not required, using only RPM's simplifies the process. Especially for drivers such as Nvidia cards.

Understanding the architecture

A diskless boot system is comprised of a couple different component that enables the entire boot process to work.

  • First the machine requires a boot mechanism. This can be PXE, which is a feature of the network card to boot entirely from the network, a CD, USB key, or even a small hard drive used only to kickstart the boot process.
  • If the machine is booting using PXE, the PXE boot code acquires a DHCP/BOOTP address from a DHCP server that contains information on where to find the bootloader. This is similar to a Grub or LILO boot screen normally found on a Linux system. This PXE bootloader defines where the init filesystem (Initrd) and the kernel are located and can be downloaded using TFTP
  • If the machine is booting using a local media method, the bootloader on that media directs the boot environment to the init filesystem and the kernel.
  • Once the init filesystem and kernel have been downloaded and loaded onto a Ramdisk, the remainder of the filesystem is mounted via NFS from the server.

Creating a test system

The VMware Player can provide an excellent system to test PXE booting and the network boot in general. The VMware Player provides a virtual PC environment that also emulates a PXE network adapter. By using this environment, you can quickly test changes to the DHCP, TFTP, and the kernel itself. Once you can get this virtual machine to boot, it is a simple matter to get a physical machine working.

Use this PXE.VMX file as a starting point for a machine you can test with

config.version = "8"
virtualHW.version = "3"
ide0:0.present = "FALSE"
ide0:0.filename = "WindowsXPPro.vmdk"
memsize = "132"
MemAllowAutoScaleDown = "FALSE"
ide1:0.present = "TRUE"
ide1:0.fileName = "auto detect"
ide1:0.deviceType = "cdrom-raw"
ide1:0.autodetect = "TRUE"
floppy0.present = "FALSE"
ethernet0.present = "TRUE"
usb.present = "TRUE"
sound.present = "FALSE"
sound.virtualDev = "es1371"
displayName = "PXE"
guestOS = "linux"
nvram = "PXE.nvram"
MemTrimRate = "-1"
ide0:0.redo = ""
ethernet0.addressType = "generated"
uuid.location = "56 4d aa 67 e4 36 4a 07-33 f5 00 d9 37 a3 6c b2"
uuid.bios = "56 4d aa 67 e4 36 4a 07-33 f5 00 d9 37 a3 6c b2"
ethernet0.generatedAddress = "00:0c:29:a3:6c:b2"
ethernet0.generatedAddressOffset = "0"
tools.syncTime = "TRUE"
ide1:0.startConnected = "FALSE"
uuid.action = "create"
checkpoint.vmState = ""

Configuring the DHCP server

In order for any network boot to function, a DHCP server must be running on the network and support the custom configuration options needed for the network boot. While this could be on any operating system, these instructions refer to the Linux DHCPD, DD-WRT and Tomato implementations.

PXE boot

Linux DHCPd

To support PXE booting, the following lines must be added to your /etc/dhcpd.conf file and the dhcpd service restarted

allow booting;
allow bootp;
class "pxeclients" {
   match if substring(option vendor-class-identifier, 0, 9) = "PXEClient";
   next-server 192.168.1.100;
   filename "linux-install/pxelinux.0";
}

The next-server directive indicates the TFTP server that the system will contact. The filename is the init file that will be loaded to provide the bootloader. This example is specific to the Redhat and Fedora Core implementation and may vary depending on your distribution.

//Alternative dhcp.conf, assume your PXE root is /mnt/pxe, and you assign your box a static IP of 192.168.1.60:

#option domain-name "domain";
default-lease-time 600;
max-lease-time 7200;
ddns-update-style ad-hoc;
subnet 192.168.1.0 netmask 255.255.255.0 {
   range 192.168.1.60 192.168.1.60;
   }
  option option-150 code 150 = text ; 
   host mythtoo { 
       hardware ethernet 00:14:2A:B3:E4:1C; 
       option option-150 "/mnt/pxe/boot/grub.lst"; 
       option routers 192.168.1.1;
       next-server 192.168.1.90;
       filename "/mnt/pxe/boot/pxegrub"; 
       filename "/mnt/pxe/pxelinux.0";
   }


Important.png Note: This configuration gives the option to use both pxegrub AND pxelinux, whichever one your box picks up first. I show this option to give a wider group of people the ability to boot, assuming they have the appropriate files.

In order to mount the NFS root partition, the DHCP server will also need to provide a root path and IP address to the booted system

option root-path "/diskless/i386/RH9/";

This option directs the DHCP client to the root filesystem. This example is specific to the Redhat and Fedora Core implementation. Please refer to your distribution documentation for details.

group {
       host vmware {
               hardware ethernet 00:0c:29:a3:6c:b2;
               fixed-address 192.168.1.250;
       }
}

These options provide a fixed IP to the diskless client based on the mac address. Note that this example definition matches the VMware player defined mac address used during testing.

DD-WRT

If you use a home router as a DHCP server DD-WRT can handle the PXE options as well.

Navigate to Admin>Services.

  1. Under DHCP Server create Static Leases to keep the IP's the same. (MAC's are in the format of XX:XX:XX:XX:XX:XX)
  2. Under DNSmasq, enable DNSmasq and Disable Local DNS
  3. Add the following for Additional DNSMasq options (not DHCPd options) where IP is the IP of your TFTP server and PATH_TO_PXE is the path to your pxelinux.0 file:
      dhcp-boot=/PATH_TO_PXE/pxelinux.0,,IP
Tomato

Similar to DD-WRT, Tomato custom firmware can also be configured to handle PXE booting. By adding the line:

  dhcp-boot=/PATH_TO_PXE/pxelinux.0,,IP

to Advanced -> DHCP/DNS -> Dnsmasq Custom Configuration.

TFTP server

The TFTP server for your distribution is required for PXE booting and needs to be installed below are instructions for Fedora/Red Hat.

The TFTP service is part of the xinetd installation. Enable it by editing your /etc/xinetd.d/tftp file

service tftp
{
        disable = no
        socket_type             = dgram
        protocol                = udp
        wait                    = yes
        user                    = root
        server                  = /usr/sbin/in.tftpd
        server_args             = -s /tftpboot -v
        per_source              = 11
        cps                     = 100 2
        flags                   = IPv4
}

Set the disable option to no. I also added the -v option so that it provides additional verbose logging to /var/log/messages which can help with troubleshooting problems.

Create the Filesystem

Redhat and the subsequent Fedora Core releases provide a fairly straightforward method of creating a diskless boot environment that can support both PXE and Netboot or Etherboot environments.

These releases provide a redhat|system-config-netboot script which does most of the heavy lifting. It creates a PXE boot image as well as kernel and a snapshot folder for each netboot system. The snapshot folder stores all the host specific files such as /dev, some parts of /etc and /var. The main root filesystem is mounted read only to provide sharing between multiple systems. It uses a pivot-root function to overlay the host specific files and folders on top of the read only portions of the filesystem.

The initrd this script created has all the modules needed for netboot. You can use the stock Fedora or Red Hat kernels.

Creating the root filesystem

Creating the root filesystem is basically a matter of copying a working installation that you want to make your client boot to. This is mostly likely a hard disk installation on the hardware you will be booting from. I have also built an OS in a Virtual Machine then copied the files from the VM to the Server.

Start by making the directories

#> mkdir -p /diskless/i386/F9/root/
#> mkdir /diskless/i386/F9/snapshot/

Next, copy the entire filesystem (exclude /proc/ and /sys/ directories) into the new location

#> rsync -a -e ssh \
   --exclude='/proc/*' \
   --exclude='/sys/*' \
   installed-system.example.com:/ \
   /diskless/i386/F9/root/

Exporting the filesystems

Using NFS, export the root and snapshot filesystems so that they can be accessed by the diskless system

vi /etc/exports
/diskless/i386/RH9/root/     *(ro,sync,no_root_squash)
/diskless/i386/RH9/snapshot/ *(rw,sync,no_root_squash)

You can replace the * with a hostname, an IP address or range, or domain name (*.myhomedomain.org) to limit access to the host(s) specified if desired. Make sure to include the system that is running the system-config-netboot, else you will get an error.

Creating the boot images

The system-config-netboot package provides the scripts necessary to create a diskless configuration. It requires the installation of the busybox and busybox-anaconda packages. My target and source system was Fedora 8.

  • If you already have a boot structure in place, be aware that this script will blindly overwrite your previous pxelinux.cfg/default file. Take a backup of the entire directory BEFORE starting the script.
  • Running this script will first ask you to create a diskless configuration from an installed kernel. If you built a kernel with the options outlined earlier and installed it on the server then choose that kernel.
  • The linux-install folder under your /tftpboot directory will be populated with the kernel specific files including the initrd and the kernel itself. It will also create the snapshot folders.
  • Next, click New on the next window to appear to create a new host identifier which populates the snapshot folder with information specific to this diskless host that will be booting from the root partition created earlier.

Gentoo specific note: If you blindly emerge "tftp" you will emerge hpa-tftpd. I found that I needed to edit the /etc/conf.d/in.tftpd file and add the -l (listen) option to get the tftpd server to start working.


Alternate Configuration

The following alternate configuration is also useful when the root-path option is present in DHCP configuration, and thus not required in the append section.

label linux
    kernel bzImage-2.6.19.diskless
    append root=/dev/nfs ip=dhcp
  • bzImage-2.6.19.diskless is the kernel image to load which must be present in /tftboot

Important.png Note: In this scenario no initial ramdisk (initrd) is required but the kernel must have the following options built in (not as modules):

  • The driver for the booting NIC
  • CONFIG_IP_PNP
  • CONFIG_IP_PNP_DHCP
  • CONFIG_NFS_FS
  • CONFIG_NFS_V3
  • CONFIG_NFS_V3_ACL
  • CONFIG_ROOT_NFS

And the root-path DHCP option specified as something like

option root-path "192.168.0.1:/netboot/frontend";
  • 192.168.0.1 is the IP address of your NFS server.
  • /netboot/frontend is the export that will become root on the remote server.


Configuring a netboot system with Local Storage

If you are using a CD, USB key, or other media configure your boot loader appropriately. Copy the initrd.img and vmlinuz files from the /tftpboot/linux-install/RH9 folder to your installation media. In my work I happened to use a small local hard disk to jumpstart the process and had my /etc/grub.conf file configured as

title Netboot
        root (hd0,0)
        kernel /vmlinuz-netboot ro root=/dev/ram init=disklessrc ramdisk_size=60000  
NFSROOT=192.168.1.100:/usr/local/media/diskless/i386/RH9 SNAPSHOT=unichrome ip=dhcp
        initrd (hd0,0)/initrd-netboot

The (hd0,0) in the initrd line was very important because once the kernel started, it didn't understand where to find the initrd file unless the specific devices was provided. This is different than the PXE method which knows where both reside.


Upgrading a netboot System

On the NFS server

  1. chroot /diskless/x86/fc6/root
  2. mv /dev/log /dev/log.tmp
  3. yum upgrade
  4. mv /dev/log.tmp /dev/log

Follow the kernel upgrade steps if you installed a new kernel

Upgrading a netboot Kernel

  • make backup of tftpboot
  • chroot into new image and install the kernel via yum upgrade kernel. Make sure to update any modules you want in the init time(ie, ethernet, disk, required files for bootup). Things like nvidia should be ok to update without a new initrd, since they're loaded from the nfs.
  • run system-config-netboot
  • answer no to any errors. else you will corrupt current boot.
  • Goto configure
    • properties for the image
    • hit next through the prompts
    • make sure to select the new kernel in the dialog w/ the kernels in it
    • hit apply
    • follow steps from install section regarding updating the initrd file.

You shouldn't have to update/recreate the pxelinux.cfg files, since they reference files that were updated by the above steps

Distribution specific instructions

Mac OS X

Please refer to the Diskless Mac-Mini Howto for complete details on this environment




Gentoo Specific with configuration file examples

DHCP Server

The default /etc/dhcp/dhcpd.conf from emerge dchp proved to be more data than necessary for the simple needs of DHCP I need for *only* the PXE machines on my network (I have a DHCP server on my router that does the DHCP for the entire network, I don't need a second one ;) ). So, I junked the original:

mv /etc/dhcp/dhcpd.conf /etc/dhcp/dhcpd.conf.orig

And created this new one, I'll try to break down the sections below the code box:

#option domain-name "domain";
default-lease-time 600;
max-lease-time 7200;
ddns-update-style ad-hoc;
subnet 192.168.1.0 netmask 255.255.255.0 {
   range 192.168.1.60 192.168.1.60;
   }
  option option-150 code 150 = text ; 
   host mythtoo { 
       hardware ethernet 00:14:2A:B3:E4:1C; 
       option option-150 "/mnt/pxe/boot/grub.lst"; 
       option routers 192.168.1.1;
       next-server 192.168.1.90;
       filename "/mnt/pxe/boot/pxegrub"; 
       filename "/mnt/pxe/pxelinux.0";
    }

This example assumes you do not need the DHCP functionality for your network from this DHCP server, it's only DHCP'ing for your PXE boxes, hence the comment on the domain-name option. The subnet and netmask will be particular to your network, this one assumes you have a simple 192.168.1.1 based network (192.168.1.0/24). I gave a range of 1 IP rather than specifying fixed-address in the 'host mythtoo {' section; why? Because I'm eccentric, or maybe it was an issue with obtaining a valid /etc/resolv.conf ;) The other options should be pretty self-explanatory:

  • hardware ethernet is the MAC address of the client (the diskless machine)
  • option option-150 was my life saving option that i uncovered somewhere online, it was necessary for pxegrub to boot.
  • option routers was where I specified what routers were on my network, after I did this I was able to access the internet from the PXE box, otherwise I was restricted to the internal network (which, for a mythbox might be great).
  • next-server is the IP of the server where your TFTP server lives. This happens to be the same server I have everything on, but it seems necessary to spell it out anyway.
  • filename This is where you specify your bootloader, pxelinux.0 for PXELinux obviously, and pxegrub for Grub :D Notice the full path, not a chroot'd path.

TFTP Server

Default emerge of "tftp" pulls in HPA-TFTPD, and is called by in.tftpd. All that was not exactly easy to find if you don't read your emerge output. From there, the default /etc/conf.d/in.tftpd doesn't work, I had to modify it to look like:

# /etc/init.d/in.tftpd
# Path to server files from
INTFTPD_PATH="/mnt/pxe"
# For more options, see tftpd(8)
INTFTPD_OPTS="-s -l ${INTFTPD_PATH}"


Important.png Note: The addition of the "-l" option, this enables tftpd to listen (why doesn't it listen by default?). To actually get my PXE client to boot, I had to /etc/init.d/in.tftpd stop which is very odd, but works for me, YMMV.

PXE Bootloading Options

I had a great time figuring out how the PXELinux boot loader works, but once I did, it was solid. By default it will search a chroot'd / directory (the / of your PXE client as specified in your TFTPD file). So, you will be creating a configuration file. You have a slew of options: You can use the MAC address of the Client, in the form of: 00-01-a1-c1-d1-f1-x1 (note the lowercase) Or you can use the Hex version of your IP: C0A8013C (PXELinux comes with an application to convert an IP to Hex: gethostip) Or any octet short of that (so C0A8013, COA801.. and so on) OR you can just use the word default. I chose all 3 ;) So you create a directory called pxelinux.cfg in your / directory, and inside it you place your configuration files as described above (MAC address, so on), which should contain something that resembles this:

DEFAULT /boot/bzImage-2.6.16 ip=dhcp root=/dev/nfs nfsroot=192.168.1.90:/mnt/pxe

Should be relatively self-explanatory. /boot/bzImage-2.6.16 is the kernel image in the /mnt/pxe/boot directory. nfsroot is the exported directory from my NFS server (my NFS server's IP is 192.168.1.90)

Just about done.

Final points

In your /etc/conf.d/net file, the following entry keeps the settings already obtained and starts eth0, thus enabling other network services :

config_eth0=( "noop" "dhcp" )


I also had to emerge nfs-utils to get it to stop complaining about netmount not working. It may take 3-4 tries of booting to get your RC's straightened out to let you through to a login.

Conclusion

Once the system is booted, there may be various tweaks and settings the need to be modified. Working with a diskless system can be tricky but since you have access to the root filesystem on the server and can use chroot /diskless/i386/RH9 to change into the environment it can lighten the load.