Correction - the configurations mentioned are relevant to IOC hosts, not to DAQ hosts. For the latter, there is more investigation needed.
This is focused on software versions - additional topics are also discussed in the referenced thread.
From mec-detectors thread on 1 September 2023
...
Almost done.. epics-release crashed with eco_tools/R2.35/epics-release.py complaining not all args converted during string formatting.... Bypassed the epics-release issue and booting the IOC from my dev area - IOC crashes with a DMA API version issue - is that a clash with the o/s build for daq-mec-cam01??
Michael: Default driver loaded seems to be v5.15.2. I'm pretty sure you need v5.16.0. Well, a host usually tries to run /reg/d/iocCommon/hosts/$HOSTNAME/startup.cmd to load drivers... but if that doesn't exist, it runs /reg/d/iocCommon/rhel7-x86_64/common/startup.cmd, which defines versions in kernel-module-dirs.cmd and loads them in kernel-modules.cmd. So looking at the latter, it runs:
lspci_SLAC_pgp=`lspci -d 1a4a:2020 -n`
...
if [ "$lspci_SLAC_pgp" != "" ]; then
...
if [ -n "$SLAC_AES_DRIVER" -a -f $SLAC_AES_DRIVER/pgpcard.ko ];
...
then echo Installing SLAC pgpcard driver: $SLAC_AES_DRIVER
...
rmmod pgpcard
rmmod pgpcardG3
insmod $SLAC_AES_DRIVER/pgpcard.ko cfgRxCount=256 cfgTxCount=64
chmod 666 /dev/pgpcard*
else
echo SLAC pgpcard driver not found: $SLAC_AES_DRIVER/pgpcard.ko
fi
else
echo SLAC pgpcard device not found.
fi
where from the former:
#
...
Select
...
default
...
SLAC_AES_DRIVER
...
if
...
not
...
overridden
...
if
...
[
...
"$SLAC_AES_DRIVER"
...
==
...
""
...
];
...
then
...
if
...
[
...
"$SLAC_AES_VER"
...
==
...
""
...
];
...
then
...
#SLAC_AES_VER=latest
...
SLAC_AES_VER=v5.15.2
...
fi
...
SLAC_AES_DRIVER=$PACKAGE_SITE_TOP/slaclab/aes-stream-drivers/$SLAC_AES_VER/install/`uname
...
-r`/
...
fi
...
export
...
SLAC_AES_DRIVER
...
So... you don't necessarily want to
...
change everyone... but you can create a host-specific startup file, and define SLAC_AES_VERSION SLAC_AES_VER to v5.16.0, and then
...
s
ource the standard rhel7 file.gpcardG3
...
Then either reboot, or manually install the new driver. To manually start... stop all of the PGP IOCs, and then as root run:
# rmmod pgpcard # rmmod pgpcardG3 # insmod /reg/g/pcds/package/slaclab/aes-stream-drivers/v5.16.0/install/`uname -r`/pgpcard.ko cfgRxCount=256 cfgTxCount=64
...
# chmod 666 /dev/pgpcard*
(I've just filled out all of the paths from the kernel*
...
cmd files there. )
Implementation for daq-mec-cam01
peregrin@mec-daq:/reg/d/iocCommon/hosts/daq-mec-cam01$ pwd
/reg/d/iocCommon/hosts/daq-mec-cam01
peregrin@mec-daq:/reg/d/iocCommon/hosts/daq-mec-cam01$ cat startup.cmd
#!/bin/bash
export SLAC_AES_VER=v5.16.0
/reg/d/iocCommon/rhel7-x86_64/common/startup.cmd
peregrin@mec-daq:/reg/d/iocCommon/hosts/daq-mec-cam01$
Investigation
This didn't work for me - the IOC still crashes on a DMA API version mismatch.
Investigation
Does any other host use this scheme?
Evidently not!
peregrin@mec-daq:/reg/d/iocCommon/hosts$ pwd
/reg/d/iocCommon/hosts
peregrin@mec-daq:/reg/d/iocCommon/hosts$ grep AES */startup.cmd
daq-mec-cam01/startup.cmd:export SLAC_AES_VER=v5.16.0
peregrin@mec-daq:/reg/d/iocCommon/hosts$
Does anyone else run an Epix IOC against common/epix/1.5.0?
Yes!
peregrin@mec-daq:/reg/g/pcds/epics/ioc$ pwd
/reg/g/pcds/epics/ioc
peregrin@mec-daq:/reg/g/pcds/epics/ioc$ grep -l "R1.5.0" */epix/*/*cfg
cxi/epix/R1.5.0/ioc-cxi-d51-epix.cfg
det/epix/R1.5.0/ioc-det-epix10ka2m.cfg
det/epix/R1.5.0/ioc-det-epix2.cfg
mfx/epix/R1.5.0/ioc-mfx-epix.cfg
mfx/epix/R1.5.1/ioc-mfx-epix10ka2m.cfg
mfx/epix/R1.5.1/ioc-mfx-epix.cfg
xcs/epix/R1.5.0/ioc-xcs-epix10ka2m.cfg
xcs/epix/R1.5.0/ioc-xcs-epix1.cfg
xcs/epix/R1.5.0/ioc-xcs-epix2.cfg
xcs/epix/R1.5.0/ioc-xcs-epix3.cfg
xcs/epix/R1.5.0/ioc-xcs-epix4.cfg
xcs/epix/R1.5.1/ioc-xcs-epix10ka2m.cfg
xcs/epix/R1.5.1/ioc-xcs-epix1.cfg
xcs/epix/R1.5.1/ioc-xcs-epix2.cfg
xcs/epix/R1.5.1/ioc-xcs-epix3.cfg
xcs/epix/R1.5.1/ioc-xcs-epix4.cfg
xpp/epix/R1.5.0/ioc-xpp-alc-epix.cfg
xpp/epix/R1.5.0/ioc-xpp-detmbl-epix1.cfg
xpp/epix/R1.5.0/ioc-xpp-detmbl-epix2.cfg
xpp/epix/R1.5.0/ioc-xpp-epix10ka2m.cfg
xpp/epix/R1.5.0/ioc-xpp-rob-epix.cfg
xpp/epix/R1.5.0/ioc-xpp-usr-epix-1.cfg
xpp/epix/R1.5.0/ioc-xpp-usr-epix-2.cfg
peregrin@mec-daq:/reg/g/pcds/epics/ioc$
And what DAQ versions are these hutches running?
cxi - 10.7.0-p9.3.4
det - 10.7.1-p9.3.4
mfx - 10.7.0-p9.3.4
xcs - 10.7.1-p9.3.4
xpp - 10.7.1-p9.3.4
Great, so which hosts are these?
However, none of these hosts have directories in /reg/d/iocCommon/hosts/.
(so now what?)
HOST | IOCs | CPU |
---|---|---|
daq-mec-cam01 | ioc-mec-epix-01, 02, 03, 04 | Intel Xeon E5620 |
daq-cxi-pgp04 | ioc-cxi-d51-epix | AMD EPYC 7282 |
daq-det-pgp01 | ioc-det-epix10ka2m | Intel Xeon Bronze 3106 |
daq-det-jungfrau | ioc-mfx-epix | Intel Xeon E5645 |
daq-mfx-pgp01 | ioc-mfx-epix10ka2m | Intel Xeon E5520 |
daq-mfx-pgp02 | ioc-mfx-epix | AMD EPYC 7282 |
daq-xcs-pgp01 | ioc-xcs-epix1, ioc-xcs-epix2, ioc-xcs-epix3, ioc-xcs-epix4 | Intel Xeon E5520 |
daq-xcs-pgp02 | ioc-xcs-epix10ka2m | Intel Xeon E5520 |
daq-xpp-pgp01 | ioc-xpp-rob-epix, ioc-xpp-usr-epix-1, ioc-xpp-usr-epix-2, ioc-xpp-usr-epix-2 | Intel Xeon E5675 |
daq-xpp-pgp02 | ioc-xpp-epix10ka2m | AMD EPYC 7282 |