Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

All 3 IRQs we typically see in SRCF are listed so that the irqbalance file can be used on all SRCF machines without modification.  Note that if another device is added to the PCI bus, the IRQ values may be different.


Timeout Waiting for Buffer

Error messages may arise from rogue that look similar to:

Code Block
1711665935.215109:pyrogue.axi.AxiStreamDma: AxiStreamDma::acceptReq: Timeout waiting for outbound buffer after 1.0 seconds! May be caused by outbound back pressure.

These arise from a select  which times out if no buffer is available. (Code is available here - as of 2024/04/01: https://github.com/slaclab/rogue/blob/de67a57927285d6df9f1a688a8187800ea80fcd8/src/rogue/hardware/axi/AxiStreamDma.cpp#L296C4-L297C4). There are (at least) a couple of scenarios where this may occur.

  • Per Larry - "This can happen is the software is not able to keep up with the DMA rate. This message means the DMA is out of buffers because software receiving buffers is not able to keep up"
  • Write buffers are stuck in hardware

Write buffers in hardware

This led to the above error when trying to run the high-rate encoder (hrencoder). The Root rogue object could be instantiated but as soon as a register access was attempted (read or write) the software would enter a loop with continual timeout. The DAQ software would continue to run, printing the error every second.

Running cat /proc/datadev_0  showed that most Read buffers were in User (desired situation); however, the write buffers were stuck in hardware (problematic). Due to this, it was not possible to access registers.

Code Block
-------------- Write Buffers ---------------
         Buffer Count : 4
          Buffer Size : 8192
          Buffer Mode : 2
      Buffers In User : 0
        Buffers In Hw : 4       # <-- THESE
  Buffers In Pre-Hw Q : 0
  Buffers In Sw Queue : 0       # <-- SHOULD BE HERE
      Missing Buffers : 0
       Min Buffer Use : 150982
       Max Buffer Use : 150983
       Avg Buffer Use : 150982
       Tot Buffer Use : 603929

The solution was to restart the kcu.service  service (sudo systemctl restart kcu.service ). Alternatively the kernel modules can be removed and reloaded directly.

Code Block
sudo rmmod /usr/local/sbin/datadev.ko
sudo insmod /usr/local/sbin/datadev.ko <Parameters...>


HSD

Location in Chassis and Repair Instructions

...