Page History

1. What is 'new' EPICS? 

The 'new' version of EPICS is R3-14-12-34_1-0

4. Where are the EPICS modules for this new base?

/afs/slac/g/lcls/epics/R3-14-12-34_1-0/modules

A handful of EPICS modules have been built for R3-14-12-34_1-0.
Most of them have been cross-compiled for several targets including the following:

linuxRT-x86
linux-x86
linux-x86_64

6. Where and how do I create a new EPICS application?  How do I setup a new IOC to use linuxRT-86 OS? 

(a) Create directories and files as shown below using ioc-b34-bd32 as an example:

 $IOC/ioc-b34-bd32

 $IOC_DATA/ioc-b34-bd32

 $IOC_DATA/ioc-b34-bd32/screenlog.0

 $IOC_DATA/vioc-b34-bd32

 $IOC_DATA/vioc-b34-bd32/autosave

 $IOC_DATA/vioc-b34-bd32/autosave-req

 $IOC_DATA/vioc-b34-bd32/iocInfo

TO DO list: Generate a script to create above directories automatically via a single python or perl script.

$APP/Test (look at this as an example) 

Now create your EPICS application called 'MyTest' as below using standard templates and scripts in a sandbox area called Development.

(b) cd $APP/Test

As MyTest development continues, this is where you will be creating/checking out from CVS, several versions of your MyTest application.

Now create a sandbox area for MyTest.

mkdir Development

(c) cd Development

You are currently in:

/afs/slac/g/lcls/epics/R3-14-12-34_1-0/iocTop/Test/Development

This is the top directory for MyTest and referred as $(TOP).

(d) Using a perl script that automates the process, create MyTest:
makeBaseApp.pl -t slac MyTest

The perl script had populated your application using icdTemplates module that provides standard templates.

These templates are used by ICD to support our style of IOC Application Building for all facilities we support.

The location for icdTemplates module was set as below when you sourced go_epics_3-14-12-3_1-0.bash.

EPICS_MBA_TEMPLATE_TOP=/afs/slac/g/lcls/epics/R3-14-12-34_1-0/modules/icdTemplates/icdTemplates-R1-0-1

If you would like to know more about makeBaseApp.pl, type:

makeBaseApp.pl -help

(e) Now, open the configure/RELEASE file.

(i) Notice how the following line appears towards the beginning:
include $(TOP)/RELEASE_SITE

The above sets the paths for EPICS base, modules, packages, tools etc. to point to version R3-14-12-34_1-0. 

(ii) The file has the following line towards the end of it:

# Location of Linux Kernel Modules:

LINUX_KERNEL_MODULES=$(PACKAGE_SITE_TOP)/linuxKernel_Modules

 (iii) configure/RELEASE file may need some modifications from what was automatically generated: 
 
Change the various module version numbers as needed for your application.
Ensure that the specific versions included in your RELEASE file DO EXIST under the modules directory and are consistent. 

They must also have been built for the linuxRT-x86 target. If something is missing, contact Ernest.

For EVENT module, Kukhee recommends us to use event-R4-1-3 or greater with this base.
 
EVENT_MODULE_VERSION=event-R4-1-3

(f) Open  MyTestApp/src/Makefile and notice that the IOC application is referred to as 'MyTest". 

(g) Now it is time to add your linuxRT-x86 virtual ioc (vioc) startup scripts. vioc-b34-bd32 is used here as an example.

From $(TOP) type the following command to create a boot directory for 'vioc-b34-bd32':

makeBaseApp.pl -i -t slac vioc-b34-bd32

When prompted to choose target architecture, choose linuxRT-x86.

Two files are modified based on this selection.

(1) vioc-b34-my01/st.cmd

(2) vioc-b34-my01/Makefile.

When prompted with 'Application name?'  type 'MyTest'.

This will use the application name MyTest that was created in MyTestApp/src/Makefile as described in step (f).

If you just hit 'Enter' without specifying your application name, then your ioc name vioc-b34-bd32 will be used as the default for the dbd file in the st.cmd

Now iocBoot directory has been created under $(TOP) and underneath it, 'vioc-b34-bd32' has been created.

Open iocBoot/vioc-b34-bd32/Makefile and notice that ARCH is set to linuxRT-x86.
Open iocBoot/vioc-b34-bd32/st.cmd and notice that this script is setup to pick the binary from  linuxRT-x86 directory.

(h) icdTemplates created MyTestApp/src/Makefile. Open this file and notice how it is populated.

It has support for base, iocAdmin, autosave-restore which all IOCs must include.

It also includes support for Till Straumann's Cexp shell. For more information on Cexp, read What is Cexp?

Additionally, the Makefile includes 'system.dbd' - this is mainly for linux-based IOCs.

For linuxRT, 'system' command is useful for changing kernel thread priorities for kernel drivers at run-time. This is explained later under rtPrioritySetup.cmd.

(i) If you have an EVR in your system, uncomment the following lines in MyTestApp/src/Makefile:

# MyTest_DBD += evrSupport.dbd
# MyTest_DBD += devMrfEr.dbd

Ensure the following lines are not commented as well:

MyTest_LIBS += evrSupport
MyTest_LIBS += devMrfEr

....

ifeq ($(T_A),linuxRT-x86)
    MyTest_LIBS += drvMrf
endif

For a PMC EVR230 or PCI-e EVR300 that is installed in your system, the above is all you need.

Replace the macros in the EPICS database files, or add more databases and modify MyTestApp/Db/Makefile as needed.

(j) Now 'make' your application from the top level directory $(TOP) to ensure your changes to  RELEASE files are good:

There should be no build errors.

Having tested this phase, you can now uninstall the binaries and cleanup the make-generated files using the following command:

make clean uninstall

Do the above before committing the files to CVS.

TO DO list: icdTemplates module that provides default templates is a work in progress.

We have to manually make several changes to the  MyTestApp/Db/* files and iocBoot/vioc-b34-bd32/st.cmd.

Follow the example MyTest and create similar database files and st.cmd.

The EVR-related screens have been added for vioc-b34-bd32. 

 You can see them when the vioc-b34-bd32 is running as follows from lcls-dev2:

lclshome -> User Dev Displays.... -> Test IOCs/Dev Event IOCs....

The buttons for the various panels for VIOC:B34:BD32 are towards the bottom of this screen.

For easy access to your IOC's EVR panels in LCLSDEV, add the necessary buttons to this page.

The 'Pattern' button brings up the 'Pattern Diagnostics' screen.

'PMC EVR:B34:BD32' button brings up the Low Level 'Diagnostics' screen for EVR;B34:BD32

'Triggers' button brings up 'Trigger Configuration' setup screen.

(c) Add your application to CVS and commit the source files.

Ernest has created some slides that explains some guidelines for importing new applications into CVS:

cvs-vendor-import.txt

CVS-Importing-Merging.pdf

CVS-At-SLAC.pdf

8. How do I start my IOC? Where is my ioc's statrup.cmd?

There are a few scripts that automate this process.

To begin with, there is the 'ipxe.ini' script in the tftp boot area /afs/slac/g/lcls/tftpboot/linuxRT/boot that PXE will run.

This is where the version of (linuxRT) kernel to run is specified as follows:

set vers 3.2.13-121108

This version number can be over-ridden by a chained, host-specific pxe init script to load an image different from the above:

chain ${hostname}.ipxe ||

For example, we have defined a script specifically for our ioc ioc-b34-bd32.ipxe which chooses to load the latest linuxRT image:

set vers 3.14.12-rt9

This is also the place to over-ride the 'root-path' option specified in the DHCP configuration file dhcpd.conf.

For example, I may decide to over-ride the afsnfs1 server and instead choose to get my boot image from afsnfs2 server:

set extra-args ROOTPATH=afsnfs2:/afs/slac:/afs/slac BOOTFILE=/afs/slac/g/lcls/epics/iocCommon/ioc-b34-bd32/startup.cmd

A few more extra arguments are specified in ioc-b34-bd32. Leave them as they are.

The 'ipxe.ini' script loads the linuxRT kernel via the TFTP protocol:

kernel --name linux tftp://${next-server}/linuxRT/boot/${vers}/bzImage && initrd tftp://${next-server}/linuxRT/boot/${vers}/rootfs.ext2 || shell
imgargs linux debug idle=halt root=/dev/ram0 console=ttyS0,115200 BOOTIF_MAC=${net0/mac:hex} ${extra-args} || boot || shell

After linuxRT boot image is downloaded to the target and linuxRT starts to run, additional nfs mounts will be done.

The afs to nfs translator service makes available the directory structure, to all clients that have mounted this nfs space.
NFS File Servers for LCLSDEV are afsnfs1 and afsnfs2

One of the arguments to the kernel process is the location of the BOOTFILE that does the mounting.

The 'filename' argument (which can be over-ridden by the BOOTFILE argument for linuxRT) is as follows:

"/afs/slac/g/lcls/epics/iocCommon/ioc-b34-bd32/startup.cmd"

This script is similar to and modelled after RTEMS startup.cmd.

When linuxRT loads and start, the kernel process is run as "root" user.
Hence it has permissions to setup the nfs mounts which is done by the following line in startup.cmd:

/afs/slac/g/lcls/epics/iocCommon/All/Dev/linuxRT_nfs.cmd

Additional NFS Mount Points for linuxRT pertaining to the ioc data directory $IOC_DATA are mounted as well.

The next line in the startup.cmd file loads the linuxRT kernel modules.

More on kernel modules under question (10).

This can also be done only by the "root" user:

/afs/slac/g/lcls/epics/iocCommon/ioc-b34-bd32/kernel-modules.cmd

Next we must start the caRepeater process:

(http://www.aps.anl.gov/epics/base/R3-14/12-docs/CAref.html#Repeater)

"When several client processes (virtual iocs) run on the same host it is not possible for all of them to directly receive a copy of the server beacon messages when the beacon messages are sent to unicast addresses, or when legacy IP kernels are still in use. To avoid confusion over these restrictions a special UDP server, the CA Repeater, is automatically spawned by the CA client library when it is not found to be running. This program listens for server beacons sent to the UDP port specified in the EPICS_CA_REPEATER_PORT parameter and fans any beacons received out to any CA client program running on the same host that have registered themselves with the CA Repeater. If the CA Repeater is not already running on a workstation, then the "caRepeater" program must be in your path before using the CA client library for the first time."

So we add the following lines to startup.cmd to start a caRepeater for all EPICS VIOCs that may be hosted by this CPU:

export EPICS_CA_REPEATER_PORT=5067

su laci -c /afs/slac/g/lcls/epics/R3-14-12-34_1-0/base/base-R3-14-12-34_1-0/bin/linuxRT-x86/caRepeater

Finally, it is possible to automatically startup one or more EPICS IOCs right here and detach them using the linux screen program.

We can start another shell such that a user called "laci" can start the IOC process instead of the "root" user:

su laci -c /afs/slac/g/lcls/epics/iocCommon/ioc-b34-bd32/startup-epics-bd32.cmd

More on startup-epics-bd32.cmd under Question (13).

Under linuxRT,  a few real time processes that need real time scheduler and kernel memory locking features, can be specifically run as such.

The screen process does not need or have RT priority.

 The _MAIN_ thread in epics application process, which is started  doesn't have the RT priority either.

But, other threads which are created by the _MAIN_ thread may need RT priority.

Kukhee provides some information about this to find out which processes are running with real-time priorities:

Command to look up thread priorities

11. How do I create a startup script for my ioc?

The third and final script is specifically to setup and start your EPICS ioc.

The 'startupConsole-bd32.cmd' lets you start your EPICS ioc as a foreground process in your host.

This lets you observe the ioc boot process to catch errors and interact with the ioc shell via iocConsole.

This is useful during development for debugging. Once the ioc has been tested fully, you can automatically start and run the process in background.

Under linuxRT, your EPICS ioc must run as a real-time process. It must lock the kernel in memory.

The following command in your 'startupConsole-bd32.cmd' does that:

ulimit -l unlimited

The following line is also needed to run the ioc with real-time priorities:

ulimit -r unlimited

Finally, you will be running your ioc as a specific user called as 'laci' who has permissions to run this ioc:
Setup the permissions for this user 'laci':

umask 002

Now you are ready to start your IOC and have it run as a foreground process.

Create a directory called as 'vioc-b34-bd32' for your ioc as below:
 
mkdir $IOC/ioc-b34-bd32/vioc-b34-bd32

cd $IOC/ioc-b34-bd32/vioc-b34-bd32
 
Set up a soft link to the 'bin' directory of your IOC app that you created in step (6). This is where your executable is:

ln -s  /afs/slac/g/lcls/epics/R3-14-12-34_1-0/iocTop/Test/Development/bin/linuxRT-x86 bin

Create an ASCII text file called 'screenrc' with the following lines in it:

deflog on

logtstamp on

defscrollback 10000

'screenrc' is passed as an argument to the 'screen' process and allows us to customize a few parameters such as number of lines stored in history buffer that we can scroll back for viewing.

In the same directory $IOC/ioc-b34-my01/vioc-b34-my01, add a startup script 'iocStartup.cmd' for vioc-b34-my01:

It setups some shell environment variables used by all iocs, and starts the st.cmd file in your ioc boot directory.

The EPICS environment variables that are set in this script can be used by your application's st.cmd script.

The default st.cmd script generated by the module icdTemplates, expects the following environment values to be defined somewhere.

iocStartup.cmd may be a good place to define it:

epicsEnvSet("IOC_NAME","VIOC:B34:BD32")

epicsEnvSet("IOC_APP","/afs/slac/g/lcls/epics/R3-14-12-34_1-0/iocTop/Test/Development")

epicsEnvSet("IOC_BOOT","${IOC_APP}/iocBoot/vioc-b34-bd32")

epicsEnvSet("IOC_COMMON","/afs/slac/g/lcls/epics/iocCommon")

epicsEnvSet("SYS_FACILITY","SYS0")

In addition to setting these environment variables, this script also executes some scripts common to all EPICS IOCs, such as below:

${IOC_COMMON}/All/Dev/linuxRT_pre_st.cmd

${IOC_COMMON}/All/Dev/linuxRT_post_st.cmd

The IOC application startup occurs between the pre and post scirpts:

cd ${IOC_BOOT}

< st.cmd

12. What are the guidelines for creating App-specific st.cmd?

Work is still in progress regarding standardizing App-specific st.cmd.

The IOC engineer has freedom to be creative with this script during development.

But the following are expected as a minimum when the ioc is installed in production.

That is the reason they are included in the default template provided by the icdTemplates module, as a guideline:

1. 1.  Making available several key paths and variables to your ioc by setting the environment variables.
      These were auto-generated during the application 'make' process and stored the 'envPaths' file under your ioc's boot directory.
      st.cmd should set all these variables using 'envPaths'.
      < envPaths
      
   2. Set the '<TOP>' variable. You could do this via the macro ${IOC_APP} which was defined in 'iocStartup.cmd' as describe before.
      epicsEnvSet(TOP,"${IOC_APP}")
   3. Load Timing-related databases using standard templates.
   4. Load iocAdmin and iocRelease databases using standard templates
   5. Load database for autosave using standard templates and use "makeAutosaveFiles" to autogenerate the PVs to be autoasaved/restored.

While the above is common guideline for all iocs, the following is specific to linuxRT.

You can set up real-time priorities after iocInit() for your driver threads and this can be done with a script such as rtPrioritySetup.cmd.

cd ${IOC_BOOT}

system("/bin/su root -c `pwd`/rtPrioritySetup.cmd")

Look at /afs/slac/g/lcls/epics/R3-14-12-34_1-0/iocTop/Test/Development/iocBoot/vioc-b34-bd32/rtPrioritySetup.cmd as an example.