Correction - the configurations mentioned are relevant to IOC hosts, not to DAQ hosts. For the latter, there is more investigation needed.
This is focused on software versions - additional topics are also discussed in the referenced thread.
From mec-detectors thread on 1 September 2023
...
Michael: Default driver loaded seems to be v5.15.2. I'm pretty sure you need v5.16.0. Well, a host usually tries to run /reg/d/iocCommon/hosts/$HOSTNAME/startup.cmd to load drivers... but if that doesn't exist, it runs /reg/d/iocCommon/rhel7-x86_64/common/startup.cmd, which defines versions in kernel-module-dirs.cmd and loads them in kernel-modules.cmd. So looking at the latter, it runs:
lspci_SLAC_pgp=`lspci -d 1a4a:2020 -n`
...
if [ "$lspci_SLAC_pgp" != "" ]; then
...
if [ -n "$SLAC_AES_DRIVER" -a -f $SLAC_AES_DRIVER/pgpcard.ko ];
...
then echo Installing SLAC pgpcard driver: $SLAC_AES_DRIVER
...
rmmod pgpcard
rmmod pgpcardG3
insmod $SLAC_AES_DRIVER/pgpcard.ko cfgRxCount=256 cfgTxCount=64
chmod 666 /dev/pgpcard*
else
echo SLAC pgpcard driver not found: $SLAC_AES_DRIVER/pgpcard.ko
fi
else
echo SLAC pgpcard device not found.
fi
where from the former:
#
...
Select
...
default
...
SLAC_AES_DRIVER
...
if
...
not
...
overridden
...
if
...
[
...
"$SLAC_AES_DRIVER"
...
==
...
""
...
];
...
then
...
if
...
[
...
"$SLAC_AES_VER"
...
==
...
""
...
];
...
then
...
#SLAC_AES_VER=latest
...
SLAC_AES_VER=v5.15.2
...
fi
...
SLAC_AES_DRIVER=$PACKAGE_SITE_TOP/slaclab/aes-stream-drivers/$SLAC_AES_VER/install/`uname
...
-r`/
...
fi
...
export
...
SLAC_AES_DRIVER
So...
...
you
...
don't
...
necessarily
...
want
...
to
...
change everyone...
...
but
...
you
...
can
...
create
...
a
...
host-specific
...
startup
...
file,
...
and
...
define
...
SLAC_AES_VERSION
...
SLAC_AES_VER to
...
v5.16.0,
...
and
...
then
...
s
ource
...
the
...
standard
...
rhel7
...
file.gpcardG3
Then either reboot, or manually install the new driver. To manually start... stop all of the PGP IOCs, and then as root run:
# rmmod pgpcard # rmmod pgpcardG3 # insmod /reg/g/pcds/package/slaclab/aes-stream-drivers/v5.16.0/install/`uname -r`/pgpcard.ko cfgRxCount=256 cfgTxCount=64 # chmod 666 /dev/pgpcard*
(I've
...
just
...
filled
...
out
...
all
...
of
...
the
...
paths
...
from
...
the
...
kernel*cmd
...
files
...
there.
...
)
Implementation for daq-mec-cam01
peregrin@mec-daq:/reg/d/iocCommon/hosts/daq-mec-cam01$ pwd
/reg/d/iocCommon/hosts/daq-mec-cam01
peregrin@mec-daq:/reg/d/iocCommon/hosts/daq-mec-cam01$ cat startup.cmd
#!/bin/bash
export SLAC_AES_VER=v5.16.0
/reg/d/iocCommon/rhel7-x86_64/common/startup.cmd
peregrin@mec-daq:/reg/d/iocCommon/hosts/daq-mec-cam01$
Investigation
This didn't work for me - the IOC still crashes on a DMA API version mismatch.
Investigation
Does any other host use this scheme?
Evidently not!
peregrin@mec-daq:/reg/d/iocCommon/hosts$ pwd
/reg/d/iocCommon/hosts
peregrin@mec-daq:/reg/d/iocCommon/hosts$ grep AES */startup.cmd
daq-mec-cam01/startup.cmd:export SLAC_AES_VER=v5.16.0
peregrin@mec-daq:/reg/d/iocCommon/hosts$
Does anyone else run an Epix IOC against common/epix/1.5.0?
Yes!
peregrin@mec-daq:/reg/g/pcds/epics/ioc$ pwd
/reg/g/pcds/epics/ioc
peregrin@mec-daq:/reg/g/pcds/epics/ioc$ grep -l "R1.5.0" */epix/*/*cfg
cxi/epix/R1.5.0/ioc-cxi-d51-epix.cfg
det/epix/R1.5.0/ioc-det-epix10ka2m.cfg
det/epix/R1.5.0/ioc-det-epix2.cfg
mfx/epix/R1.5.0/ioc-mfx-epix.cfg
mfx/epix/R1.5.1/ioc-mfx-epix10ka2m.cfg
mfx/epix/R1.5.1/ioc-mfx-epix.cfg
xcs/epix/R1.5.0/ioc-xcs-epix10ka2m.cfg
xcs/epix/R1.5.0/ioc-xcs-epix1.cfg
xcs/epix/R1.5.0/ioc-xcs-epix2.cfg
xcs/epix/R1.5.0/ioc-xcs-epix3.cfg
xcs/epix/R1.5.0/ioc-xcs-epix4.cfg
xcs/epix/R1.5.1/ioc-xcs-epix10ka2m.cfg
xcs/epix/R1.5.1/ioc-xcs-epix1.cfg
xcs/epix/R1.5.1/ioc-xcs-epix2.cfg
xcs/epix/R1.5.1/ioc-xcs-epix3.cfg
xcs/epix/R1.5.1/ioc-xcs-epix4.cfg
xpp/epix/R1.5.0/ioc-xpp-alc-epix.cfg
xpp/epix/R1.5.0/ioc-xpp-detmbl-epix1.cfg
xpp/epix/R1.5.0/ioc-xpp-detmbl-epix2.cfg
xpp/epix/R1.5.0/ioc-xpp-epix10ka2m.cfg
xpp/epix/R1.5.0/ioc-xpp-rob-epix.cfg
xpp/epix/R1.5.0/ioc-xpp-usr-epix-1.cfg
xpp/epix/R1.5.0/ioc-xpp-usr-epix-2.cfg
peregrin@mec-daq:/reg/g/pcds/epics/ioc$
And what DAQ versions are these hutches running?
cxi - 10.7.0-p9.3.4
det - 10.7.1-p9.3.4
mfx - 10.7.0-p9.3.4
xcs - 10.7.1-p9.3.4
xpp - 10.7.1-p9.3.4
Great, so which hosts are these?
However, none of these hosts have directories in /reg/d/iocCommon/hosts/.
(so now what?)
HOST | IOCs | CPU |
---|---|---|
daq-mec-cam01 | ioc-mec-epix-01, 02, 03, 04 | Intel Xeon E5620 |
daq-cxi-pgp04 | ioc-cxi-d51-epix | AMD EPYC 7282 |
daq-det-pgp01 | ioc-det-epix10ka2m | Intel Xeon Bronze 3106 |
daq-det-jungfrau | ioc-mfx-epix | Intel Xeon E5645 |
daq-mfx-pgp01 | ioc-mfx-epix10ka2m | Intel Xeon E5520 |
daq-mfx-pgp02 | ioc-mfx-epix | AMD EPYC 7282 |
daq-xcs-pgp01 | ioc-xcs-epix1, ioc-xcs-epix2, ioc-xcs-epix3, ioc-xcs-epix4 | Intel Xeon E5520 |
daq-xcs-pgp02 | ioc-xcs-epix10ka2m | Intel Xeon E5520 |
daq-xpp-pgp01 | ioc-xpp-rob-epix, ioc-xpp-usr-epix-1, ioc-xpp-usr-epix-2, ioc-xpp-usr-epix-2 | Intel Xeon E5675 |
daq-xpp-pgp02 | ioc-xpp-epix10ka2m | AMD EPYC 7282 |
Then either reboot, or manually install the new driver. To manually start... stop all of the PGP IOCs, and then as root run:
# rmmod pgpcard
# rmmod pgpcardG3
# insmod /reg/g/pcds/package/slaclab/aes-stream-drivers/v5.16.0/install/`uname -r`/pgpcard.ko cfgRxCount=256 cfgTxCount=64
# chmod 666 /dev/pgpcard*
...