Difference between revisions of "Diskless Frontend"

From MythTV Official Wiki
Jump to: navigation, search
(Understanding the architecture)
m (use proper interwiki syntax)
(45 intermediate revisions by 10 users not shown)
Line 2: Line 2:
  
 
== Introduction ==
 
== Introduction ==
 
+
The following How-To will assist you on building a Diskless Frontend for MythTV.  The instructions are based on Fedora/Red Hat unless otherwise noted.  This how to assumes a working machine that is configured to run MythTV, getting to this point is out of the scope of this doc.  This system can be either a physical install on a hard drive or a Virtual Machine in VMware.  Although not required, using only RPM's simplifies the process.  Especially for drivers such as Nvidia cards.
 +
 
=== Understanding the architecture ===
 
=== Understanding the architecture ===
  
 
A diskless boot system is comprised of a couple different component that enables the entire boot process to work.   
 
A diskless boot system is comprised of a couple different component that enables the entire boot process to work.   
  
* First the machine requires a boot mechanism.  This can be [[PXE]], which is a feature of the network card to boot entirely from the network.  This could also be a boot floppy, CD, USB key, or even a small hard drive used only to kickstart the boot process.
+
* First the machine requires a boot mechanism.  This can be [[PXE]], which is a feature of the network card to boot entirely from the network, a CD, USB key, or even a small hard drive used only to kickstart the boot process.
* If the machine is booting using PXE, the PXE boot code acquires a [[DHCP]]/[[BOOTP]] address from a DHCP server that contains information on where to find the bootloader.  This is similar to a [[Grub]] or [[LILO]] boot screen normally found on a Linux system.  This PXE bootloader defines where the init filesystem ([[Initrd]]) and the kernel are located and can be downloaded using [[TFTP]]
+
* If the machine is booting using PXE, the PXE boot code acquires a [[DHCP]]/[[BOOTP]] address from a DHCP server that contains information on where to find the bootloader.  This is similar to a [[wikipedia:GRUB|Grub]] or [[wikipedia:LILO_%28boot_loader%29|LILO]] boot screen normally found on a Linux system.  This PXE bootloader defines where the init filesystem ([[wikipedia:Initrd|Initrd]]) and the kernel are located and can be downloaded using [[wikipedia:Tftp|TFTP]]
 
* If the machine is booting using a local media method, the bootloader on that media directs the boot environment to the init filesystem and the kernel.   
 
* If the machine is booting using a local media method, the bootloader on that media directs the boot environment to the init filesystem and the kernel.   
* Once the init filesystem and kernel have been downloaded and loaded onto a [[Ramdisk]], the remainder of the filesystem is mounted via [[NFS]] from the server.
+
* Once the init filesystem and kernel have been downloaded and loaded onto a [[wikipedia:Ramdisk|Ramdisk]], the remainder of the filesystem is mounted via [[NFS]] from the server.
 
+
{{Warning box
+
|msg=If you are thinking about diskless and have an Nvidia card, there is a known bug installing their driver on such a system (see [http://www.nvnews.net/vbulletin/showthread.php?t=79389 Installation fails on NFS]). This ''should'' only apply to building the driver on that system - installing from an RPM ''should'' be fine.
+
}}
+
  
 
=== Creating a test system ===
 
=== Creating a test system ===
  
I found that the [http://www.vmware.com/products/player/ VMware Player] can provide an excellent system to test PXE booting and the network boot in general.  The VMware Player provides a virtual PC environment that also emulates a PXE network adapter.  By using this environment, you can quickly test changes to the DHCP, TFTP, and the kernel itself.  Once you can get this virtual machine to boot, it is a simple matter to get a physical machine working.
+
The [http://www.vmware.com/products/player/ VMware Player] can provide an excellent system to test PXE booting and the network boot in general.  The VMware Player provides a virtual PC environment that also emulates a PXE network adapter.  By using this environment, you can quickly test changes to the DHCP, TFTP, and the kernel itself.  Once you can get this virtual machine to boot, it is a simple matter to get a physical machine working.
  
 
Use this PXE.VMX file as a starting point for a machine you can test with
 
Use this PXE.VMX file as a starting point for a machine you can test with
Line 54: Line 51:
 
=== Configuring the DHCP server ===
 
=== Configuring the DHCP server ===
  
In order for any network boot to function, a DHCP server must be running on the network and support the custom configuration options needed for the network boot.  While this could be on any operating system, these instructions refer to the Linux DHCPD implementation.
+
In order for any network boot to function, a DHCP server must be running on the network and support the custom configuration options needed for the network boot.  While this could be on any operating system, these instructions refer to the Linux DHCPD, DD-WRT and Tomato implementations.
  
 
==== PXE boot ====
 
==== PXE boot ====
 
+
=====Linux DHCPd =====
 
To support PXE booting, the following lines must be added to your /etc/dhcpd.conf file and the dhcpd service restarted
 
To support PXE booting, the following lines must be added to your /etc/dhcpd.conf file and the dhcpd service restarted
  
Line 89: Line 86:
 
     }
 
     }
  
[[Image:Important.png|left]] '''Note:''' This configuration gives the option to use both pxegrub AND pxelinux, whichever one your box picks up first.  I show this option to give a wider group of people the ability to boot, assuming they have the appropriate files.<br style="clear:left;"/>
+
{{Note box|This configuration gives the option to use both pxegrub AND pxelinux, whichever one your box picks up first.  I show this option to give a wider group of people the ability to boot, assuming they have the appropriate files.
 
+
}}
==== TFTP server ====
+
 
+
The TFTP server for your distribution is required for PXE booting and needs to be installed.
+
 
+
Gentoo specific note:
+
If you blindly emerge "tftp" you will emerge hpa-tftpd.  I found that I needed to edit the /etc/conf.d/in.tftpd file and add the -l (listen) option to get the tftpd server to start working.
+
 
+
==== DHCP server ====
+
  
 
In order to mount the NFS root partition, the DHCP server will also need to provide a root path and IP address to the booted system
 
In order to mount the NFS root partition, the DHCP server will also need to provide a root path and IP address to the booted system
Line 115: Line 104:
 
These options provide a fixed IP to the diskless client based on the [[mac address]].  Note that this example definition matches the VMware player defined mac address used during testing.
 
These options provide a fixed IP to the diskless client based on the [[mac address]].  Note that this example definition matches the VMware player defined mac address used during testing.
  
==== Building the kernel ====
+
===== DD-WRT =====
  
A diskless kernel requies some options that normal kernels may not include
+
If you use a home router as a DHCP server DD-WRT can handle the PXE options as well.
  
CONFIG_NFS_FS=y
+
Navigate to Admin>Services.
  CONFIG_ROOT_NFS=m (some instructions say this needs to be in the kernel but I had problems with that and didn't need it ultimately)
+
# Under DHCP Server create Static Leases to keep the IP's the same. (MAC's are in the format of XX:XX:XX:XX:XX:XX)
CONFIG_IP_PNP=y
+
# Under DNSmasq, enable DNSmasq and Disable Local DNS
CONFIG_IP_PNP_RARP=y
+
# Add the following for '''Additional DNSMasq options''' (not DHCPd options) where IP is the IP of your TFTP server and PATH_TO_PXE is the path to your pxelinux.0 file:
CONFIG_IP_PNP_BOOTP=y
+
       
CONFIG_IP_PNP_DHCP=y
+
      dhcp-boot=/PATH_TO_PXE/pxelinux.0,,IP
  
Make sure to also include your Ethernet driver in the kernel and not as a module.  For the VMware Player test system, use the AMD PCNET32 driver.
+
===== Tomato =====
  
== Distribution specific instructions ==
+
Similar to DD-WRT, '''Tomato''' custom firmware can also be configured to handle PXE booting. By adding the line:
  
=== Mac OS X ===
+
  dhcp-boot=/PATH_TO_PXE/pxelinux.0,,IP
  
Please refer to the [[Diskless Mac-Mini Howto]] for complete details on this environment
+
to Advanced -> DHCP/DNS -> Dnsmasq Custom Configuration.
  
=== Redhat and Fedora Core ===
+
==== TFTP server ====
 
+
Redhat and the subsequent Fedora Core releases provide a fairly straightfoward method of creating a diskless boot environment that can support both PXE and [[Netboot]] or [[Etherboot]] environments.
+
 
+
These releases provide a redhat|system-config-netboot script which does most of the heavy lifting.  It creates a PXE boot image as well as kernel and a snapshot folder for each netboot system.  The snapshot folder stores all the host specific files such as /dev, some parts of /etc and /var.  The main root filesystem is mounted read only to provide sharing between multiple systems.  It uses a [[pivot-root]] function to overlay the host specific files and folders on top of the read only portions of the filesystem.
+
  
==== TFTP server notes ====
+
The TFTP server for your distribution is required for PXE booting and needs to be installed below are instructions for Fedora/Red Hat.
  
 
The TFTP service is part of the xinetd installation.  Enable it by editing your /etc/xinetd.d/tftp file
 
The TFTP service is part of the xinetd installation.  Enable it by editing your /etc/xinetd.d/tftp file
Line 159: Line 144:
  
 
Set the disable option to no.  I also added the -v option so that it provides additional verbose logging to /var/log/messages which can help with troubleshooting problems.
 
Set the disable option to no.  I also added the -v option so that it provides additional verbose logging to /var/log/messages which can help with troubleshooting problems.
 +
 +
=== Create the Filesystem ===
 +
Redhat and the subsequent Fedora Core releases provide a fairly straightforward method of creating a diskless boot environment that can support both PXE and [[Netboot]] or [[Etherboot]] environments.
 +
 +
These releases provide a redhat|system-config-netboot script which does most of the heavy lifting.  It creates a PXE boot image as well as kernel and a snapshot folder for each netboot system.  The snapshot folder stores all the host specific files such as /dev, some parts of /etc and /var.  The main root filesystem is mounted read only to provide sharing between multiple systems.  It uses a [[pivot-root]] function to overlay the host specific files and folders on top of the read only portions of the filesystem.
 +
 +
The initrd this script created has all the modules needed for netboot. You can use the stock Fedora or Red Hat kernels.
  
 
==== Creating the root filesystem ====
 
==== Creating the root filesystem ====
  
Creating the root filesystem is basically a matter of copying a working installation that you want to make your client boot to.  This is mostly likely a hard disk installation on the hardware you will be booting from.   
+
Creating the root filesystem is basically a matter of copying a working installation that you want to make your client boot to.  This is mostly likely a hard disk installation on the hardware you will be booting from.  I have also built an OS in a Virtual Machine then copied the files from the VM to the Server.
  
 
Start by making the directories
 
Start by making the directories
  
  mkdir -p /diskless/i386/RH9/root/
+
  #> mkdir -p /diskless/i386/F9/root/
  mkdir /diskless/i386/RH9/snapshot/
+
  #> mkdir /diskless/i386/F9/snapshot/
  
 
Next, copy the entire filesystem (exclude /proc/ and /sys/ directories) into the new location
 
Next, copy the entire filesystem (exclude /proc/ and /sys/ directories) into the new location
  
  rsync -a -e ssh --exclude='/proc/*' --exclude='/sys/*' installed-system.example.com:/ /diskless/i386/RH9/root/
+
  #> rsync -a -e ssh \
 +
    --exclude='/proc/*' \
 +
    --exclude='/sys/*' \
 +
    installed-system.example.com:/ \
 +
    /diskless/i386/F9/root/
  
 
==== Exporting the filesystems ====
 
==== Exporting the filesystems ====
  
 
Using NFS, export the root and snapshot filesystems so that they can be accessed by the diskless system
 
Using NFS, export the root and snapshot filesystems so that they can be accessed by the diskless system
 +
 +
vi /etc/exports
  
 
  /diskless/i386/RH9/root/    *(ro,sync,no_root_squash)
 
  /diskless/i386/RH9/root/    *(ro,sync,no_root_squash)
 
  /diskless/i386/RH9/snapshot/ *(rw,sync,no_root_squash)
 
  /diskless/i386/RH9/snapshot/ *(rw,sync,no_root_squash)
  
You can replace the * with a hostname, an IP address or range, or domain name (*.myhomedomain.org) to limit access to the host(s) specified if desired.
+
You can replace the * with a hostname, an IP address or range, or domain name (*.myhomedomain.org) to limit access to the host(s) specified if desired. Make sure to include the system that is running the system-config-netboot, else you will get an error.
  
 
==== Creating the boot images ====
 
==== Creating the boot images ====
  
The system-config-netboot package provides the scripts necessary to create a diskless configuration.  It requires the installation of the busybox and busybox-anaconda packages.  My target and source system was Redhat 9.  The redhat-config-netboot script provided with Redhat Enterprise Linux is massively broken.  The system-config-netboot script provided with Fedora Core 4 had some additional python dependencies that I didn't care to work out so I chose to use the Fedora Core 3 script which worked fine.
+
The system-config-netboot package provides the scripts necessary to create a diskless configuration.  It requires the installation of the busybox and busybox-anaconda packages.  My target and source system was Fedora 8.   
 
+
http://download.fedora.redhat.com/pub/fedora/linux/core/updates/3/i386/system-config-netboot-0.1.30-1_FC3.noarch.rpm
+
 
+
Neither the suggested 0.1.30-1_FC3 nor the current 0.1.40-1.FC5 versions of system-config-netboot worked for me on FC5However, I found a new version 0.1.41-1.FC6 that worked flawlessly.
+
 
+
http://download.fedora.redhat.com/pub/fedora/linux/core/development/source/SRPMS/system-config-netboot-0.1.41-1.FC6.src.rpm
+
  
 
* If you already have a boot structure in place, be aware that this script will blindly overwrite your previous pxelinux.cfg/default file.  '''Take a backup of the entire directory BEFORE starting the script.'''
 
* If you already have a boot structure in place, be aware that this script will blindly overwrite your previous pxelinux.cfg/default file.  '''Take a backup of the entire directory BEFORE starting the script.'''
* Running this script will first ask you to create a diskless configuration from an installed kernel.  If you built a kernel with the options outlined earlier and installed it on the server then choose that kernel.  I found it is easiest to use the same kernel for both the server and the diskless client whenever possible.  It makes buildling kernel modules for the  diskless kernel alot easier because the root filesystem on the diskless machine is read-only and it can make some installers throw fits, like nvidida.
+
* Running this script will first ask you to create a diskless configuration from an installed kernel.  If you built a kernel with the options outlined earlier and installed it on the server then choose that kernel.
 
* The linux-install folder under your /tftpboot directory will be populated with the kernel specific files including the initrd and the kernel itself.  It will also create the snapshot folders.
 
* The linux-install folder under your /tftpboot directory will be populated with the kernel specific files including the initrd and the kernel itself.  It will also create the snapshot folders.
 
* Next, click New on the next window to appear to create a new host identifier which populates the snapshot folder with information specific to this diskless host that will be booting from the root partition created earlier.
 
* Next, click New on the next window to appear to create a new host identifier which populates the snapshot folder with information specific to this diskless host that will be booting from the root partition created earlier.
  
However system-config-netboot-0.1.41-1.FC6 does not work flawlessly on an FC5 or FC6 system. To make it work, you need a couple of changes.
+
Gentoo specific note:
 
+
If you blindly emerge "tftp" you will emerge hpa-tftpdI found that I needed to edit the /etc/conf.d/in.tftpd file and add the -l (listen) option to get the tftpd server to start working.
1) FC5 & FC6: Add "ramdisk_blocksize=1024" to the append line of the boot stanza below.
+
 
+
2) FC5 & FC6: SELinux can cause problems with logging in to the system when it is netbooted.  Disable it by adding "selinux=0" to the append line of the boot stanza below.
+
 
+
3) FC6 only: Include /sbin/mount.nfs in the initrd.img after it is generated. This can be done as follows (as root).
+
<pre>
+
cd to the boot directory. On my system this is /tftpboot/linux-install/mythfe.fc6
+
 
+
#> zcat initrd.img >| /tmp/initrd
+
#> mount -o loop /tmp/initrd /mnt
+
#> cd /mnt
+
#> cp /sbin/mount.nfs sbin/
+
#> cd -
+
#> umount /mnt
+
#> gzip /tmp/initrd
+
#> mv /tmp/initrd.gz initrd.img</pre>
+
 
+
4) FC5 (with newer kernels) & FC6:  Several people have found that the snapshot directory is not being mounted rw using the newer kernels.  The following is a fix involving modifying the disklessrc file which is run after the kernel boots.  It references the same initrd.img file above
+
 
+
<pre>
+
# mkdir -p /tmp/initrd-mount
+
# zcat initrd.img >| /tmp/initrd
+
# mount -o loop /tmp/initrd  /tmp/initrd-mount
+
</pre>
+
Then you must edit /tmp/initrd-mount/disklessrc and make the following change (adding the line in '''bold''').
+
 
+
''...<br>
+
echo Mounting Snapshot directories<br>
+
mount -t nfs $NFS_IP:${NFS_DIR}/snapshot /.snapshot -o rw,nolock &&<br>
+
'''mount -o remount,rw /.snapshot &&'''<br>
+
{<br>
+
...<br>''
+
 
+
Save and exit the file, then...
+
<pre>
+
# umount /tmp/initrd-mount
+
# gzip /tmp/initrd
+
# mv /tmp/initrd.gz /tmp/initrd.img
+
</pre>
+
And finally copy /tmp/initrd.img back over the original.
+
 
+
==== Configuring the PXE bootloader ====
+
 
+
If you are using PXE, edit your /tftpboot/linux-install/pxelinux.cfg/default file to include a new option
+
 
+
label FC5
+
    kernel FC5/vmlinuz
+
    append initrd=FC5/initrd.img init=disklessrc root=/dev/ram0 ramdisk_size=60000
+
NFSROOT=192.168.1.100:/diskless/i386/FC5 SNAPSHOT=unichrome ip=dhcp
+
 
+
* '''label''' is the text you will type in when the PXE bootloader prompt appears
+
* '''kernel''' specifies the directory relative to the /tftpboot/linux-install folder where the kernel lies
+
 
+
'''append''' specifies a series of options to pass to the kernel
+
 
+
* '''initrd''' specifies the directory relative to the /tftpboot/linux-install folder where the init filesystem is
+
* '''init''' is the script that is run when the filesystem boots. This disklessrc file is the script that mounts all the filesystems needed for the diskless client
+
* '''root''' is the device where the init filesystem lives.  This is a ramdisk.
+
* '''ramdisk_size''' is needed to specify the size of your init filesystem.  Mine was over 6MB so I had to provide this line to increase the size of the ramdisk to hold it
+
* '''NFSROOT''' tells the kernel where to mount the root filesystem from.  It will append /root to the end.
+
* '''SNAPSHOT''' tells the disklessrc file which snapshot folder to mount
+
* '''ip''' tells the kernel how to get it's IP address for netbooting
+
  
  
===== Alternate Configuration =====
+
=== Alternate Configuration ===
  
 
The following alternate configuration is also useful when the '''root-path''' option is present in DHCP configuration, and thus not required in the '''append''' section.
 
The following alternate configuration is also useful when the '''root-path''' option is present in DHCP configuration, and thus not required in the '''append''' section.
Line 290: Line 220:
 
* '''/netboot/frontend''' is the export that will become root on the remote server.
 
* '''/netboot/frontend''' is the export that will become root on the remote server.
  
==== Configuring a netboot system ====
 
  
If you are using a CD, USB key, or other media configure your bootloader appropriately.  Copy the initrd.img and vmlinuz files from the /tftpboot/linux-install/RH9 folder to your installation media.  In my work I happened to use a small local hard disk to jumpstart the process and had my /etc/grub.conf file configured as
+
=== Configuring a netboot system with Local Storage ===
 +
 
 +
If you are using a CD, USB key, or other media configure your boot loader appropriately.  Copy the initrd.img and vmlinuz files from the /tftpboot/linux-install/RH9 folder to your installation media.  In my work I happened to use a small local hard disk to jumpstart the process and had my /etc/grub.conf file configured as
  
 
  title Netboot
 
  title Netboot
Line 301: Line 232:
  
 
The (hd0,0) in the initrd line was very important because once the kernel started, it didn't understand where to find the initrd file unless the specific devices was provided.  This is different than the PXE method which knows where both reside.
 
The (hd0,0) in the initrd line was very important because once the kernel started, it didn't understand where to find the initrd file unless the specific devices was provided.  This is different than the PXE method which knows where both reside.
 +
 +
 +
 +
=== Upgrading a netboot System ===
 +
On the NFS server
 +
# chroot /diskless/x86/fc6/root
 +
# mv /dev/log /dev/log.tmp
 +
# yum upgrade
 +
# mv /dev/log.tmp /dev/log
 +
Follow the kernel upgrade steps if you installed a new kernel
 +
 +
=== Upgrading a netboot Kernel ===
 +
 +
* make backup of tftpboot
 +
* chroot into new image and install the kernel via yum upgrade kernel. Make sure to update any modules you want in the init time(ie, ethernet, disk, required files for bootup). Things like nvidia should be ok to update without a new initrd, since they're loaded from the nfs.
 +
* run system-config-netboot
 +
* answer no to any errors. else you will corrupt current boot.
 +
* Goto configure
 +
** properties for the image
 +
** hit next through the prompts
 +
** make sure to select the new kernel in the dialog w/ the kernels in it
 +
** hit apply
 +
** follow steps from install section regarding updating the initrd file.
 +
 +
You shouldn't have to update/recreate the pxelinux.cfg files, since they reference files that were updated by the above steps
 +
 +
== Distribution specific instructions ==
 +
 +
=== Mac OS X ===
 +
 +
Please refer to the [[Diskless Mac-Mini Howto]] for complete details on this environment
 +
 +
 +
 +
 +
  
 
=== Gentoo Specific with configuration file examples ===
 
=== Gentoo Specific with configuration file examples ===
Line 331: Line 298:
 
This example assumes you do not need the DHCP functionality for your network from this DHCP server, it's only DHCP'ing for your PXE boxes, hence the comment on the domain-name option.  The subnet and netmask will be particular to your network, this one assumes you have a simple 192.168.1.1 based network (192.168.1.0/24).  I gave a range of 1 IP rather than specifying fixed-address in the 'host mythtoo {' section; why?  Because I'm eccentric, or maybe it was an issue with obtaining a valid /etc/resolv.conf ;)
 
This example assumes you do not need the DHCP functionality for your network from this DHCP server, it's only DHCP'ing for your PXE boxes, hence the comment on the domain-name option.  The subnet and netmask will be particular to your network, this one assumes you have a simple 192.168.1.1 based network (192.168.1.0/24).  I gave a range of 1 IP rather than specifying fixed-address in the 'host mythtoo {' section; why?  Because I'm eccentric, or maybe it was an issue with obtaining a valid /etc/resolv.conf ;)
 
The other options should be pretty self-explanatory:
 
The other options should be pretty self-explanatory:
hardware ethernet is the MAC address of the client (the diskless machine)
+
* '''hardware ethernet''' is the MAC address of the client (the diskless machine)
option option-150 was my life saving option that i uncovered somewhere online, it was necessary for pxegrub to boot.
+
* '''option option-150''' was my life saving option that i uncovered somewhere online, it was necessary for pxegrub to boot.
option routers was where I specified what routers were on my network, after I did this I was able to access the internet from the PXE box, otherwise I was restricted to the internal network (which, for a mythbox might be great).
+
* '''option routers''' was where I specified what routers were on my network, after I did this I was able to access the internet from the PXE box, otherwise I was restricted to the internal network (which, for a mythbox might be great).
next-server is the IP of the server where your TFTP server lives.  This happens to be the same server I have everything on, but it seems necessary to spell it out anyway.
+
* '''next-server''' is the IP of the server where your TFTP server lives.  This happens to be the same server I have everything on, but it seems necessary to spell it out anyway.
filename  This is where you specify your bootloader, pxelinux.0 for PXELinux obviously, and pxegrub for Grub :D  Notice the full path, not a chroot'd path.
+
* '''filename''' This is where you specify your bootloader, pxelinux.0 for PXELinux obviously, and pxegrub for Grub :D  Notice the full path, not a chroot'd path.
  
 
==== TFTP Server ====
 
==== TFTP Server ====
Line 349: Line 316:
 
  INTFTPD_OPTS="-s -l ${INTFTPD_PATH}"
 
  INTFTPD_OPTS="-s -l ${INTFTPD_PATH}"
  
[[Image:Important.png|left]] '''Note:''' The addition of the "-l" option, this enables tftpd to listen (why doesn't it listen by default?).  To actually get my PXE client to boot, I had to /etc/init.d/in.tftpd stop  which is very odd, but works for me, YMMV.<br style="clear:left;"/>
+
{{Note box|The addition of the "-l" option, this enables tftpd to listen (why doesn't it listen by default?).  To actually get my PXE client to boot, I had to /etc/init.d/in.tftpd stop  which is very odd, but works for me, YMMV.}}
  
 
==== PXE Bootloading Options ====
 
==== PXE Bootloading Options ====

Revision as of 13:20, 12 October 2012

This page describes the various methods for creating diskless frontends on a variety of Linux distributions.

Introduction

The following How-To will assist you on building a Diskless Frontend for MythTV. The instructions are based on Fedora/Red Hat unless otherwise noted. This how to assumes a working machine that is configured to run MythTV, getting to this point is out of the scope of this doc. This system can be either a physical install on a hard drive or a Virtual Machine in VMware. Although not required, using only RPM's simplifies the process. Especially for drivers such as Nvidia cards.

Understanding the architecture

A diskless boot system is comprised of a couple different component that enables the entire boot process to work.

  • First the machine requires a boot mechanism. This can be PXE, which is a feature of the network card to boot entirely from the network, a CD, USB key, or even a small hard drive used only to kickstart the boot process.
  • If the machine is booting using PXE, the PXE boot code acquires a DHCP/BOOTP address from a DHCP server that contains information on where to find the bootloader. This is similar to a Grub or LILO boot screen normally found on a Linux system. This PXE bootloader defines where the init filesystem (Initrd) and the kernel are located and can be downloaded using TFTP
  • If the machine is booting using a local media method, the bootloader on that media directs the boot environment to the init filesystem and the kernel.
  • Once the init filesystem and kernel have been downloaded and loaded onto a Ramdisk, the remainder of the filesystem is mounted via NFS from the server.

Creating a test system

The VMware Player can provide an excellent system to test PXE booting and the network boot in general. The VMware Player provides a virtual PC environment that also emulates a PXE network adapter. By using this environment, you can quickly test changes to the DHCP, TFTP, and the kernel itself. Once you can get this virtual machine to boot, it is a simple matter to get a physical machine working.

Use this PXE.VMX file as a starting point for a machine you can test with

config.version = "8"
virtualHW.version = "3"
ide0:0.present = "FALSE"
ide0:0.filename = "WindowsXPPro.vmdk"
memsize = "132"
MemAllowAutoScaleDown = "FALSE"
ide1:0.present = "TRUE"
ide1:0.fileName = "auto detect"
ide1:0.deviceType = "cdrom-raw"
ide1:0.autodetect = "TRUE"
floppy0.present = "FALSE"
ethernet0.present = "TRUE"
usb.present = "TRUE"
sound.present = "FALSE"
sound.virtualDev = "es1371"
displayName = "PXE"
guestOS = "linux"
nvram = "PXE.nvram"
MemTrimRate = "-1"
ide0:0.redo = ""
ethernet0.addressType = "generated"
uuid.location = "56 4d aa 67 e4 36 4a 07-33 f5 00 d9 37 a3 6c b2"
uuid.bios = "56 4d aa 67 e4 36 4a 07-33 f5 00 d9 37 a3 6c b2"
ethernet0.generatedAddress = "00:0c:29:a3:6c:b2"
ethernet0.generatedAddressOffset = "0"
tools.syncTime = "TRUE"
ide1:0.startConnected = "FALSE"
uuid.action = "create"
checkpoint.vmState = ""

Configuring the DHCP server

In order for any network boot to function, a DHCP server must be running on the network and support the custom configuration options needed for the network boot. While this could be on any operating system, these instructions refer to the Linux DHCPD, DD-WRT and Tomato implementations.

PXE boot

Linux DHCPd

To support PXE booting, the following lines must be added to your /etc/dhcpd.conf file and the dhcpd service restarted

allow booting;
allow bootp;
class "pxeclients" {
   match if substring(option vendor-class-identifier, 0, 9) = "PXEClient";
   next-server 192.168.1.100;
   filename "linux-install/pxelinux.0";
}

The next-server directive indicates the TFTP server that the system will contact. The filename is the init file that will be loaded to provide the bootloader. This example is specific to the Redhat and Fedora Core implementation and may vary depending on your distribution.

//Alternative dhcp.conf, assume your PXE root is /mnt/pxe, and you assign your box a static IP of 192.168.1.60:

#option domain-name "domain";
default-lease-time 600;
max-lease-time 7200;
ddns-update-style ad-hoc;
subnet 192.168.1.0 netmask 255.255.255.0 {
   range 192.168.1.60 192.168.1.60;
   }
  option option-150 code 150 = text ; 
   host mythtoo { 
       hardware ethernet 00:14:2A:B3:E4:1C; 
       option option-150 "/mnt/pxe/boot/grub.lst"; 
       option routers 192.168.1.1;
       next-server 192.168.1.90;
       filename "/mnt/pxe/boot/pxegrub"; 
       filename "/mnt/pxe/pxelinux.0";
   }


Important.png Note: This configuration gives the option to use both pxegrub AND pxelinux, whichever one your box picks up first. I show this option to give a wider group of people the ability to boot, assuming they have the appropriate files.

In order to mount the NFS root partition, the DHCP server will also need to provide a root path and IP address to the booted system

option root-path "/diskless/i386/RH9/";

This option directs the DHCP client to the root filesystem. This example is specific to the Redhat and Fedora Core implementation. Please refer to your distribution documentation for details.

group {
       host vmware {
               hardware ethernet 00:0c:29:a3:6c:b2;
               fixed-address 192.168.1.250;
       }
}

These options provide a fixed IP to the diskless client based on the mac address. Note that this example definition matches the VMware player defined mac address used during testing.

DD-WRT

If you use a home router as a DHCP server DD-WRT can handle the PXE options as well.

Navigate to Admin>Services.

  1. Under DHCP Server create Static Leases to keep the IP's the same. (MAC's are in the format of XX:XX:XX:XX:XX:XX)
  2. Under DNSmasq, enable DNSmasq and Disable Local DNS
  3. Add the following for Additional DNSMasq options (not DHCPd options) where IP is the IP of your TFTP server and PATH_TO_PXE is the path to your pxelinux.0 file:
      dhcp-boot=/PATH_TO_PXE/pxelinux.0,,IP
Tomato

Similar to DD-WRT, Tomato custom firmware can also be configured to handle PXE booting. By adding the line:

  dhcp-boot=/PATH_TO_PXE/pxelinux.0,,IP

to Advanced -> DHCP/DNS -> Dnsmasq Custom Configuration.

TFTP server

The TFTP server for your distribution is required for PXE booting and needs to be installed below are instructions for Fedora/Red Hat.

The TFTP service is part of the xinetd installation. Enable it by editing your /etc/xinetd.d/tftp file

service tftp
{
        disable = no
        socket_type             = dgram
        protocol                = udp
        wait                    = yes
        user                    = root
        server                  = /usr/sbin/in.tftpd
        server_args             = -s /tftpboot -v
        per_source              = 11
        cps                     = 100 2
        flags                   = IPv4
}

Set the disable option to no. I also added the -v option so that it provides additional verbose logging to /var/log/messages which can help with troubleshooting problems.

Create the Filesystem

Redhat and the subsequent Fedora Core releases provide a fairly straightforward method of creating a diskless boot environment that can support both PXE and Netboot or Etherboot environments.

These releases provide a redhat|system-config-netboot script which does most of the heavy lifting. It creates a PXE boot image as well as kernel and a snapshot folder for each netboot system. The snapshot folder stores all the host specific files such as /dev, some parts of /etc and /var. The main root filesystem is mounted read only to provide sharing between multiple systems. It uses a pivot-root function to overlay the host specific files and folders on top of the read only portions of the filesystem.

The initrd this script created has all the modules needed for netboot. You can use the stock Fedora or Red Hat kernels.

Creating the root filesystem

Creating the root filesystem is basically a matter of copying a working installation that you want to make your client boot to. This is mostly likely a hard disk installation on the hardware you will be booting from. I have also built an OS in a Virtual Machine then copied the files from the VM to the Server.

Start by making the directories

#> mkdir -p /diskless/i386/F9/root/
#> mkdir /diskless/i386/F9/snapshot/

Next, copy the entire filesystem (exclude /proc/ and /sys/ directories) into the new location

#> rsync -a -e ssh \
   --exclude='/proc/*' \
   --exclude='/sys/*' \
   installed-system.example.com:/ \
   /diskless/i386/F9/root/

Exporting the filesystems

Using NFS, export the root and snapshot filesystems so that they can be accessed by the diskless system

vi /etc/exports
/diskless/i386/RH9/root/     *(ro,sync,no_root_squash)
/diskless/i386/RH9/snapshot/ *(rw,sync,no_root_squash)

You can replace the * with a hostname, an IP address or range, or domain name (*.myhomedomain.org) to limit access to the host(s) specified if desired. Make sure to include the system that is running the system-config-netboot, else you will get an error.

Creating the boot images

The system-config-netboot package provides the scripts necessary to create a diskless configuration. It requires the installation of the busybox and busybox-anaconda packages. My target and source system was Fedora 8.

  • If you already have a boot structure in place, be aware that this script will blindly overwrite your previous pxelinux.cfg/default file. Take a backup of the entire directory BEFORE starting the script.
  • Running this script will first ask you to create a diskless configuration from an installed kernel. If you built a kernel with the options outlined earlier and installed it on the server then choose that kernel.
  • The linux-install folder under your /tftpboot directory will be populated with the kernel specific files including the initrd and the kernel itself. It will also create the snapshot folders.
  • Next, click New on the next window to appear to create a new host identifier which populates the snapshot folder with information specific to this diskless host that will be booting from the root partition created earlier.

Gentoo specific note: If you blindly emerge "tftp" you will emerge hpa-tftpd. I found that I needed to edit the /etc/conf.d/in.tftpd file and add the -l (listen) option to get the tftpd server to start working.


Alternate Configuration

The following alternate configuration is also useful when the root-path option is present in DHCP configuration, and thus not required in the append section.

label linux
    kernel bzImage-2.6.19.diskless
    append root=/dev/nfs ip=dhcp
  • bzImage-2.6.19.diskless is the kernel image to load which must be present in /tftboot

Important.png Note: In this scenario no initial ramdisk (initrd) is required but the kernel must have the following options built in (not as modules):

  • The driver for the booting NIC
  • CONFIG_IP_PNP
  • CONFIG_IP_PNP_DHCP
  • CONFIG_NFS_FS
  • CONFIG_NFS_V3
  • CONFIG_NFS_V3_ACL
  • CONFIG_ROOT_NFS

And the root-path DHCP option specified as something like

option root-path "192.168.0.1:/netboot/frontend";
  • 192.168.0.1 is the IP address of your NFS server.
  • /netboot/frontend is the export that will become root on the remote server.


Configuring a netboot system with Local Storage

If you are using a CD, USB key, or other media configure your boot loader appropriately. Copy the initrd.img and vmlinuz files from the /tftpboot/linux-install/RH9 folder to your installation media. In my work I happened to use a small local hard disk to jumpstart the process and had my /etc/grub.conf file configured as

title Netboot
        root (hd0,0)
        kernel /vmlinuz-netboot ro root=/dev/ram init=disklessrc ramdisk_size=60000  
NFSROOT=192.168.1.100:/usr/local/media/diskless/i386/RH9 SNAPSHOT=unichrome ip=dhcp
        initrd (hd0,0)/initrd-netboot

The (hd0,0) in the initrd line was very important because once the kernel started, it didn't understand where to find the initrd file unless the specific devices was provided. This is different than the PXE method which knows where both reside.


Upgrading a netboot System

On the NFS server

  1. chroot /diskless/x86/fc6/root
  2. mv /dev/log /dev/log.tmp
  3. yum upgrade
  4. mv /dev/log.tmp /dev/log

Follow the kernel upgrade steps if you installed a new kernel

Upgrading a netboot Kernel

  • make backup of tftpboot
  • chroot into new image and install the kernel via yum upgrade kernel. Make sure to update any modules you want in the init time(ie, ethernet, disk, required files for bootup). Things like nvidia should be ok to update without a new initrd, since they're loaded from the nfs.
  • run system-config-netboot
  • answer no to any errors. else you will corrupt current boot.
  • Goto configure
    • properties for the image
    • hit next through the prompts
    • make sure to select the new kernel in the dialog w/ the kernels in it
    • hit apply
    • follow steps from install section regarding updating the initrd file.

You shouldn't have to update/recreate the pxelinux.cfg files, since they reference files that were updated by the above steps

Distribution specific instructions

Mac OS X

Please refer to the Diskless Mac-Mini Howto for complete details on this environment




Gentoo Specific with configuration file examples

DHCP Server

The default /etc/dhcp/dhcpd.conf from emerge dchp proved to be more data than necessary for the simple needs of DHCP I need for *only* the PXE machines on my network (I have a DHCP server on my router that does the DHCP for the entire network, I don't need a second one ;) ). So, I junked the original:

mv /etc/dhcp/dhcpd.conf /etc/dhcp/dhcpd.conf.orig

And created this new one, I'll try to break down the sections below the code box:

#option domain-name "domain";
default-lease-time 600;
max-lease-time 7200;
ddns-update-style ad-hoc;
subnet 192.168.1.0 netmask 255.255.255.0 {
   range 192.168.1.60 192.168.1.60;
   }
  option option-150 code 150 = text ; 
   host mythtoo { 
       hardware ethernet 00:14:2A:B3:E4:1C; 
       option option-150 "/mnt/pxe/boot/grub.lst"; 
       option routers 192.168.1.1;
       next-server 192.168.1.90;
       filename "/mnt/pxe/boot/pxegrub"; 
       filename "/mnt/pxe/pxelinux.0";
    }

This example assumes you do not need the DHCP functionality for your network from this DHCP server, it's only DHCP'ing for your PXE boxes, hence the comment on the domain-name option. The subnet and netmask will be particular to your network, this one assumes you have a simple 192.168.1.1 based network (192.168.1.0/24). I gave a range of 1 IP rather than specifying fixed-address in the 'host mythtoo {' section; why? Because I'm eccentric, or maybe it was an issue with obtaining a valid /etc/resolv.conf ;) The other options should be pretty self-explanatory:

  • hardware ethernet is the MAC address of the client (the diskless machine)
  • option option-150 was my life saving option that i uncovered somewhere online, it was necessary for pxegrub to boot.
  • option routers was where I specified what routers were on my network, after I did this I was able to access the internet from the PXE box, otherwise I was restricted to the internal network (which, for a mythbox might be great).
  • next-server is the IP of the server where your TFTP server lives. This happens to be the same server I have everything on, but it seems necessary to spell it out anyway.
  • filename This is where you specify your bootloader, pxelinux.0 for PXELinux obviously, and pxegrub for Grub :D Notice the full path, not a chroot'd path.

TFTP Server

Default emerge of "tftp" pulls in HPA-TFTPD, and is called by in.tftpd. All that was not exactly easy to find if you don't read your emerge output. From there, the default /etc/conf.d/in.tftpd doesn't work, I had to modify it to look like:

# /etc/init.d/in.tftpd
# Path to server files from
INTFTPD_PATH="/mnt/pxe"
# For more options, see tftpd(8)
INTFTPD_OPTS="-s -l ${INTFTPD_PATH}"


Important.png Note: The addition of the "-l" option, this enables tftpd to listen (why doesn't it listen by default?). To actually get my PXE client to boot, I had to /etc/init.d/in.tftpd stop which is very odd, but works for me, YMMV.

PXE Bootloading Options

I had a great time figuring out how the PXELinux boot loader works, but once I did, it was solid. By default it will search a chroot'd / directory (the / of your PXE client as specified in your TFTPD file). So, you will be creating a configuration file. You have a slew of options: You can use the MAC address of the Client, in the form of: 00-01-a1-c1-d1-f1-x1 (note the lowercase) Or you can use the Hex version of your IP: C0A8013C (PXELinux comes with an application to convert an IP to Hex: gethostip) Or any octet short of that (so C0A8013, COA801.. and so on) OR you can just use the word default. I chose all 3 ;) So you create a directory called pxelinux.cfg in your / directory, and inside it you place your configuration files as described above (MAC address, so on), which should contain something that resembles this:

DEFAULT /boot/bzImage-2.6.16 ip=dhcp root=/dev/nfs nfsroot=192.168.1.90:/mnt/pxe

Should be relatively self-explanatory. /boot/bzImage-2.6.16 is the kernel image in the /mnt/pxe/boot directory. nfsroot is the exported directory from my NFS server (my NFS server's IP is 192.168.1.90)

Just about done.

Final points

In your /etc/conf.d/net file, the following entry keeps the settings already obtained and starts eth0, thus enabling other network services :

config_eth0=( "noop" "dhcp" )


I also had to emerge nfs-utils to get it to stop complaining about netmount not working. It may take 3-4 tries of booting to get your RC's straightened out to let you through to a login.

Conclusion

Once the system is booted, there may be various tweaks and settings the need to be modified. Working with a diskless system can be tricky but since you have access to the root filesystem on the server and can use chroot /diskless/i386/RH9 to change into the environment it can lighten the load.