...

• user "systemctl list-unit-files" to see if tdetsim.service or kcu.service is enabled
• associated .service files are in /usr/lib/systemd/system/
• to see if events are flowing from the hardware to the software: "cat /proc/datadev_0" and watch "Tot Buffer Use" counter under "Read Buffers"
• if you see the error "rogue.GeneralError: AxiStreamDma::AxiStreamDma: General Error: Failed to map dma buffers. Increase vm map limit: sysctl -w vm.max_map_count=262144".  This can be caused by that parameter being too low, but it can also be caused by loading the driver with CfgMode=1 in tdetsim.service (we use CfgMode=2).  This CfgMode parameter has to do with the way memory gets cached.
  • make sure the tdetsim.service is the same as another working node
  • make sure that the appropriate service is enabled (see first point)
  • check that the driver in /usr/local/sbin/datadev.ko is the same as a working node
  • check that /etc/sysctl.conf has the correct value of vm.max_map_count
• we have also seen this error when the datadev.ko buffer sizes and counts were configured incorrectly in tdetsim.service.

An example tdetsim.service for an opal camera:

drp-neh-cmp012:~$ more /usr/lib/systemd/system/tdetsim.service
[Unit]
Description=SimCam Device Manager
Requires=multi-user.target
After=multi-user.target

[Service]
Type=simple
ExecStart=/usr/local/sbin/kcuSim -s -d /dev/datadev_0
ExecStartPost=/sbin/insmod /usr/local/sbin/datadev.ko cfgTxCount=4 cfgRxCount=40
96 cfgSize=0x210000 cfgMode=0x2
ExecStop=
KillMode=process
IgnoreSIGPIPE=no
StandardOutput=syslog
StandardError=inherit

[Install]
WantedBy=multi-user.target

HSD

• look at configured parameters using (for example) "hsdpva DAQ:LAB2:HSD:DEV06_3D:A"
• for kcu firmware that is built to use both QSFP links, the naming of the qsfp's is swapped.  i.e. the qsfp that is normally called /dev/datadev_0 is now called /dev/datadev_1
• HSD is not configured to do anything (Check the HSD config tab for no channels enabled)
• if hsd timing frames are not being received at 926kHz (status here), click TxLink Reset in XPM window.  Typically when this is an issue the receiving rate is ~20kHz.
• The HSD configuration partition number does not match platform number in .cnf file (Check the HSD "base" tab)
• also check that HEADERCNTL0 is incrementing in "status" tab of HSD cfg window.
• in hsd window timepausecnt is number of clocks we are dead
• in hsd expert window "full threshold(events)" sets threshold for hsd deadtime
• in hsd status window "fex free events" and "raw free events" are the current free events.
• in hsd status window "write fifo count" is number of timing headers waiting for HSD data to associate.
• "readcntsum" on hsd timing tab goes up when we send a transition OR L1Accepts. "trigcntsum" counts L1Accepts only.
• check kcuStatus for "locPause" non-zero (a low level pgp FIFO being full).  If this happens then:  configure hsd, clear readout, reboot drp node with KCU

...