Page History

8. How do I start my CPU? Where is my host's statrup.cmd?

There are a few scripts that automate this process.

To begin with, there is the 'ipxe.ini' script in the tftp boot area /afs/slac/g/lcls/tftpboot/linuxRT/boot that PXE will run.

This is where the version of (linuxRT) kernel to run is specified as follows:

set vers 3.2.13-121108

This version number can be over-ridden by a chained, host-specific pxe init script to load an image different from the above:

chain ${hostname}.ipxe ||

 For example, we have defined a script specifically for our ioc cpu-b34-bd32.ipxe which chooses to load the latest linuxRT image:

set vers 3.14.12-rt9_x86_64_glibglibc

 This is also the place to over-ride the 'root-path' option specified in the DHCP configuration file dhcpd.conf.

 For example, we may decide to over-ride the afsnfs1 server and instead choose to get my boot image from lcls-dev3 server:

set extra-args brd.rd_size=524288 ROOTPATH=lcls-dev3:/afs/slac:/afs/slac BOOTFILE=/afs/slac/g/lcls/epics/iocCommon/cpu-b34-bd32/startup.cmd

The 'ipxe.ini' script loads the linuxRT kernel via the TFTP protocol:

kernel --name linux tftp://${next-server}/linuxRT/boot/${vers}/bzImage && initrd tftp://${next-server}/linuxRT/boot/${vers}/rootfs.ext2 || shell
imgargs linux debug idle=halt root=/dev/ram0 console=ttyS0,115200 BOOTIF_MAC=${net0/ mac:hex } ${extra-args} || boot || shell

After linuxRT boot image is downloaded to the target and linuxRT starts to run, additional nfs mounts will be done.

 The afs to nfs translator service makes available the directory structure, to all clients that have mounted this nfs space.

 NFS File Servers for LCLSDEV is primarily lcls-dev3. Alternately, it can use afsnfs1 or afsnfs2 but these will be phased out soon.

 One of the arguments to the kernel process is the location of the BOOTFILE that does the mounting.

The 'filename' argument (which can be over-ridden by the BOOTFILE argument for linuxRT) is as follows:

"/afs/slac/g/lcls/epics/iocCommon/cpu-b34-bd32/startup.cmd"

This script is similar to and modeled after RTEMS startup.cmd. The standard startup.cmd script is a soft link to ../skel/startup_cpu.cmd.

If you use the above startup_cpu.cmd script, you can skip the rest of this section.

The remaining standard host level configuration involves setting up iocConsole (question 9) and hardware specific kernel modules (question 10).

Then system startup continues with IOC application startup (questions 11 through 14)

If non-default behavior is desired, the soft link may be replaced by a different startup.cmd. The standard startup_cpu.cmd sets up the host-wide environment by calling the generic script All/common/linuxRT_startup_cpu.sh.

The linuxRT_startup_cpu.sh script defines standard environment variables such as $HOSTNAME and $T_A, and calls a script for facility specific environment setup, facility/linuxRT_env.sh.

When linuxRT loads and starts, the kernel process is run as "root" user.

 Hence it has permissions to setup the nfs mounts which is done by the following line in linuxRT_startup_cpu.sh:

linuxRT_nfs.cmd

Additional NFS Mount Points for linuxRT pertaing to the ioc data directory $IOC_DATA are mounted as well.

The next line in the linuxRT_startup_cpu.sh file loads the system specific linuxRT kernel modules.

More on kernel modules under question (10).

This can also be done only by the "root" user:

/afs/slac/g/lcls/epics/iocCommon/cpu-b34-bd32/kernel-modules.cmd

Next we must start the caRepeater process:

(http://www.aps.anl.gov/epics/base/R3-14/12-docs/CAref.html#Repeater)

"When several client processes (virtual iocs) run on the same host it is not possible for all of them to directly receive a copy of the server beacon messages when the beacon messages are sent to unicast addresses, or when legacy IP kernels are still in use. To avoid confusion over these restrictions a special UDP server, the CA Repeater, is automatically spawned by the CA client library when it is not found to be running. This program listens for server beacons sent to the UDP port specified in the EPICS_CA_REPEATER_PORT parameter and fans any beacons received out to any CA client program running on the same host that have registered themselves with the CA Repeater. If the CA Repeater is not already running on a workstation, then the "caRepeater" program must be in your path before using the CA client library for the first time."

So we have added the following lines to  linuxRT_startup_cpu.sh to start a caRepeater for all EPICS VIOCs that may be hosted by this CPU:

su $IOC_USER -c $(which caRepeater) &

Once linuxRT_startup_cpu.sh completes system-wide configuration, it is possible to automatically startup one or more EPICS IOCs and detach them using the linux screen program. The standard startup script uses linuxRT_cpu_load_iocs.sh for this. It searches screeniocs for viocs configured to run on this host and starts them.

There are other options to start IOCs when the standard automatic startup is not wanted. Rather than using screeniocs to determine what to start, a custom startup.cmd may call linuxRT_cpu_load_iocs.sh after defining environment variable $LOCAL_IOCS and optionally $IOC_EXECUTABLES to list which IOCs to start.

More directly, we can start another shell such that the ioc user account, usually "laci", can start the IOC process instead of the "root" user:

su laci -c /afs/slac/g/lcls/epics/iocCommon/cpu-b34-bd32/startup-epics-bd32.cmd

or

 su laci -c /afs/slac/g/lcls/epics/iocCommon/vioc-b34-bd32/startup.cmd

 More on startup-epics-bd32.cmd under Question (13).

 Under linuxRT, a few real time processes that need real time scheduler and kernel memory locking features, can be specifically run as such.

 The screen process does not need or have RT priority.

The _MAIN_ thread in epics application process, which is started  doesn't have the RT priority either.

But, other threads which are created by the _MAIN_ thread may need RT priority.

 Kukhee provides some information about this to find out which processes are running with real-time priorities: Command to look up thread priorities