Page History

7. What is PXE, DHCP, TFTP and NFS and why are they needed by my linuxRT IOC?

LinuxRT is installed on our system using the Preboot Execution Environment (PXE) method of network booting.

We enable the PXE/network-booting method in the BIOS.

In order to use PXE there is a boot-server that will allow our client system to :
(a) Request an IP address (via DHCP)
(b) Download a kernel (via TFTP)

With both of these services in place any system which supports PXE/network-booting
should be able to gain an IP address, fetch a kernel, and boot without an installed operating system.

PXE uses three distinct network protocols that map to three server processes to perform the installation.
In our case, some of the processes run on lcls-dev1 (LCLSDEV daemon)

Both  DHCP and TFTP services run on the LCLSDEV host 'dhcp3' maintained by SCCS.

(a) Dynamic Host Configuration Protocol (DHCP)

PXE uses DHCP to deliver initial network configuration options to client nodes.
The DHCP server supplies the PXE boot plug-in with
(i) IP address
(ii) TFTP server address
(iii) Stage 1 image boot-loader name from which to download and execute the image.

As the supplied PXE installation environments are non-interactive and will unconditionally reinstall a client machine,
we have the client associate its MAC address with a specific OS installation before starting the PXE boot.

The configuration information, in our case, in addition to IP/MAC address, includes a hostname and a pointer to the Master Starupt script in afs for our IOC.
It has an optional root-path variable pointing to the afs area which hosts the boot image that is served via TFTP.
This can be over-ridden as will be seen later.

When the Linux server is rebooted or power-cycled, PXE will attempt the network booting method first
and as a first step it will contact the DHCP server to retrieve the network configuration information.

Hence, every new linuxRT ioc (host) needs to be added to the DHCP server configuration file in afs.

This file is in /afs/slac/service/dhcp-pxe/dhcpd.conf

Note that the DHCP service running on dhcp3 is intended only for booting embedded devices like our linuxRT servers that are connected to the LCLSDEV and SSRL subnets.

The MAC addresses for such devices must be registered in CANDO and assigned fixed IP addresses.

The IP/MAC address of the primary ethernet that will fetch the linuxRT boot image is defined here.
To add a new host to the DHCP configuration, contact Thuy.

After a new ioc is added to dhcpd.conf, the DHCP service must be restarted.

To restart DHCP Server

From a Unix command line:

remctl dhcp3 dhcp check

remctl dhcp3 dhcp restart

For help:

remctl dhcp3 dhcp help

Currently only Thuy, Ernest and a couple of others have permissions to perform this restart on dhcp3.

Here's is an example - ioc-b34-bd32:

host ioc-b34-bd32 {
# SuperMicro (INTELx86)
#
hardware ethernet 00:25:90:D1:95:1E;
fixed-address 134.79.218.190;
option host-name "ioc-b34-bd32";
if ( substring( option vendor-class-identifier, 0, 5 ) = "udhcp" ) {
filename "/afs/slac/g/lcls/epics/iocCommon/ioc-b34-bd32/startup.cmd";
option root-path "afsnfs1:/afs/slac:/afs/slac";
}
}

To find out more about how our linux server boots up linuxRT, click on the following link:

 How does the Linux Server boot up linuxRT?

(b) Trivial File Transfer Protocol (TFTP)

PXE uses TFTP that defines a simple UDP protocol for delivering files over a network.
PXE delivers kernels and initial bootstrap software to client nodes using TFTP.

tftpboot is mounted in afs area in LCLSDEV on the server dhcp3, which runs both the TFTP an DHCP services:

 /tftpboot -> /afs/slac.stanford.edu/service/dhcp-pxe/tftpboot/

The iocs retrieve the linuxRT boot image from the TFTP server from the following location:

/afs/slac/g/lcls/tftpboot/linuxRT/boot

In this location, there are several linuxRT-x86 bootimages.
These were custom-built by T.Straumann for the various Linux Servers/IPCs that we currently have setup to boot with linuxRT OS.

Of these images, '3.14.12-rt9' is the latest and it has in-built support for the
Broadcom networking ethernet chipset that are used in our dev Poweredge Dell Servers.

(c) Network File System (NFS)

The NFS service is used by the installation kernel to read all of the packages necessary to the installation process.

NFS services run on the LCLSDEV hosts afsnfs1 and afsnfs2  maintained by SCCS.

 This service makes available the boot directory to all linuxRT targets that boot as diskless clients.

All clients have read-only permissions to this directory.

The linuxRT iocs need some additional NFS Mount Points to write their data some where.

surrey04b is an NFS Appliance and is used by iocs for data and have both read and write permissions to the data directory ($IOC_DATA).

In LCLSDEV, we must obtain permissions for the iocs to write to the $IOC_DATA directory.

 Fill out an online form provided by SCCS to obtain permissions for your ioc to write to this directory:

https://www-rt.slac.stanford.edu/rt3/SelfService/Forms/IocNfs.html