Versions Compared

Old Version 97

changes.mady.by.user Shantha Condamoor

Saved on

compared with

New Version 98

changes.mady.by.user Shantha Condamoor

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Panel
10. What are kernel modules and how are they loaded in linuxRT?

In the linux world, kernel modules are pieces of code that can be loaded and unloaded into the kernel upon demand.
They extend the functionality of the kernel without the need to reboot the system.
For example, one type of module is the device driver, which allows the kernel to access hardware connected to the system.
Without modules, we would have to build monolithic kernels and add new functionality directly into the kernel image.
Besides having larger kernels, this has the disadvantage of requiring us to rebuild and reboot the kernel every time we want new functionality.

linuxRT too, lets you load and unload kernel modules dynamically.

Now we are ready to load some kernel modules essential to our ioc - like EVR.

In our 'startup.cmd' script, we have the following line which lets us customize and load our kernel modules:

/afs/slac/g/lcls/epics/iocCommon/ioc-b34-bd32/kernel-modules.cmd

The location for kernel modules is specified as an environment variable in linuxRT:

KERNEL_DRIVER_HOME=/afs/slac/g/lcls/package/linuxKernel_Modules

There are several linuxRT drivers in this directory.

The PCI EVR230 driver is here. The following driver version has been built for the latest linuxRT 3.14.12-rt9:

EVR230_PCI_EVR_DRIVER=$KERNEL_DRIVER_HOME/pci_mrfevr_linuxRT/buildroot-2014.08

The PCI Express EVR300 driver built for linuxRT 3.14.12-rt9, is here:

EVR300_PCI_EVR_DRIVER=$KERNEL_DRIVER_HOME/pci_mrfev300_linuxRT/buildroot-2014.08

The kernel drivers are installed (loaded) dynamically as follows:

# Load the MRF EVR230 Kernel Module for timing
insmod $EVR230_PCI_EVR_DRIVER/pci_mrfevr.ko
$EVR230_PCI_EVR_DRIVER/module_load

# Load the MRF EVR300 Kernel Module for timing
insmod $EVR300_PCI_EVR_DRIVER/pci_mrfevr300.ko
$EVR300_PCI_EVR_DRIVER/module_load

There are a couple of things to note:

1. Currently the EVR kernel modules SW has the restriction that if there are both a PMC EVR230 and a PCI-e EVR300 in a linux box,
then the PMC EVR230 MUST BE initialized as card 0 and loaded first. EVR300 must be initialized as card 1.

Additionally, due to hard-coded device names in the module, it is essential to setup the following links:

ln -s /dev/er3a0 /dev/erb0
ln -s /dev/er3a1 /dev/erb1
ln -s /dev/er3a2 /dev/erb2
ln -s /dev/er3a3 /dev/erb3

2. If only one EVR (either PMC EVR230 or PCI EVR300) is installed in your system, then the above restriction does not apply and soft links are not needed.

Take a look at the following script:
/afs/slac/g/lcls/package/linuxKernel_Modules/pci_mrfev300_linuxRT/buildroot-2014.08/module_load

Notice how kernel modules are loaded as device drivers under the /dev/ in linuxRT much like linux.

3. The Broadcom Ethernet NIC driver used to be a separate kernel module and its dirver was loaded dynamically via ;modprobe tg3' in this script.
With the latest linuxRT version 3.14.12-rt9, this step has become unnecessary as the driver has become part of this linuxRT boot image.

4. The SIS digitizers for uTCA, loads their device drivers in 'kernel-modules.cmd':
SIS8300_DRIVER=$KERNEL_DRIVER_HOME/sis8300drv/MAIN_TRUNK

modprobe uio
insmod $SIS8300_DRIVER/sis8300drv.ko

Please note that as of date, SIS8300 has NOT been rebuilt for the latest linuxRT 3.14.12-rt9.
It is currently unsupported on COTS Linux servers.

For convinience, you can setup a soft link to your ioc's home directory in kernel-modules.cmd as below:

ln -s /afs/slac/g/lcls/epics/iocCommon/ioc-b34-pm32              bd32              /home/laci/ioc-b34-pm32bd32

Panel
11. How do I create a startup script for my ioc?

The third and final script is specifically to setup and start your EPICS ioc.

The 'startupConsole-laci-rt.cmd' lets you start your EPICS ioc as a foreground process in your host.

This lets you observe the ioc boot process to catch errors and interact with the ioc shell via iocConsole.

This is useful during development for debugging. Once the ioc has been tested fully, you can automatically start and run the process in background.

Under linuxRT, your EPICS ioc must run as a real-time process. It must lock the kernel in memory.

The following command in your 'startupConsole-laci-rt.cmd' does that:

ulimit -l unlimited

The following line is also needed to run the ioc with real-time priorities:

ulimit -r unlimited

Finally, you will be running your ioc as a specific user called as 'laci' who has permissions to run this ioc:
Setup the permissions for this user 'laci':

umask 002

Now you are ready to start your IOC and have it run as a foreground process.

Create a directory called as 'vioc-b34-bd32' for your ioc as below:
 
mkdir $IOC/ioc-b34-bd32/vioc-b34-bd32

cd $IOC/ioc-b34-bd32/vioc-b34-bd32
 
Set up a soft link to the 'bin' directory of your IOC app that you created in step (6). This is where your executable is:

ln -s  /afs/slac/g/lcls/epics/R3-14-12-3_1-0/iocTop/Test/Development/bin/linuxRT-x86 bin

Create an ASCII text file called 'screenrc' with the following lines in it:

deflog on

logtstamp on

defscrollback 10000

'screenrc' is passed as an argument to the 'screen' process and allows us to customize a few parameters such as number of lines stored in history buffer that we can scroll back for viewing.

In the same directory $IOC/ioc-b34-my01/vioc-b34-my01, add a startup script 'iocStartup.cmd' for vioc-b34-my01:

It setups some shell environment variables used by all iocs, and starts the st.cmd file in your ioc boot directory.

The EPICS environment variables that are set in this script can be used by your application's st.cmd script.

The default st.cmd script generated by the module icdTemplates, expects the following environment values to be defined somewhere.

iocStartup.cmd may be a good place to define it:

epicsEnvSet("IOC_NAME","VIOC:B34:BD32")

epicsEnvSet("IOC_APP","/afs/slac/g/lcls/epics/R3-14-12-3_1-0/iocTop/Test/Development")

epicsEnvSet("IOC_BOOT","${IOC_APP}/iocBoot/vioc-b34-bd32")

epicsEnvSet("IOC_COMMON","/afs/slac/g/lcls/epics/iocCommon")

epicsEnvSet("SYS_FACILITY","SYS0")

In addition to setting these environment variables, this script also executes some scripts common to all EPICS IOCs, such as below:

${IOC_COMMON}/All/Dev/linuxRT_pre_st.cmd

${IOC_COMMON}/All/Dev/linuxRT_post_st.cmd

The IOC application startup occurs between the pre and post scirpts:

cd ${IOC_BOOT}

< st.cmd

...