Page History
...
Example of two-level batching event builder (BEB) in the FEB and KCU:
FEB segment a batch"a" event-build for event 1 with two detector segments a,b and a timestamp t1: t1,d1a,d1b event-built into "f1a"
FEB segment b batch"b" event-build for event 1 with two detector segments c,d and a timestamp t1: t1,d1c,d1d event-built into "f1b"
KCU batch event-build for event 1: f1a,f1b (static size for HR, UHR, but variable for sparkPIX). Note that this is a partial event-build because other detector segments will be connected to other KCUs. Existing LCLS software does that high-level event-build.
Note: will always get a packet from sparkpix from empty payload
July 1, 2024
- we think gpu can write to kcu, but Jeremy will try to confirm that this is really the case. we think kcu to gpu is broken.
- Mudit will work on TDet firmware
- Jeremy and TID will use rdsrv419
- LCLS should find another machine where we can control the root-complex topology better. lab3 daq-tst-dev06?
- could be that the CPU datadev driver is incompatible with the GPU datadev driver
- should probably find a different machine. a fee-alcove machine?
- chris and gabriel on vacation July 6-20
- Jeremy unavailable July 8-10
July 8, 2024
- Jeremy and CPO are on vacation
- Chris and I have set up
daq-tst-dev06
in Lab 3 for testing with a GPU & KCU- The KCU appears as
/dev/datagpu_1
- After some debugging to get the software to recognize
/dev/datagpu_1
(as opposed to_0
) theinterCardGui
comes up and shows sensible values test_dma
also runs, but theAxiVersion.ScratchPad
register does not go from0x0
to0xdeadbeef
- Neither does
test_dma
see any DMAs
- Neither does
- The KCU appears as
- Larry urges to move axi-pcie-devel forward to v4.1.0 rather than working with v4.0.0
- Mudit has created v4.2.0 (CPU/GPU DMA switch?) but it has not been tested with hardware yet
- It is unclear whether Jeremy has gotten a baseline working system yet
- Ryan agreed that my hacks of
test_dma.cu
and_Root.py
to target/dev/datagpu_1
should be sufficient - Ryan would like us all to work on one machine and get confidence in it before we branch out to different machines
- I suggested that we could pull out the current
/dev/datagpu_0
from dev06 so that the KCU of interest becomes _0- Ryan & Larry suggested pullling the other ones out as well
- Ryan suggests setting
iommu=no
in the BIOS as well as on the command line- I later found that there is no iommu parameter in dev06's BIOS
- There's no experience with multiple KCU cards and the datagpu driver - it is untested
- Larry is aware of some issue with the usual datadev driver and multiple KCUs
- Work on rdsrv419
- A GPU is installed
- CUDA is installed
- A KCU is installed and has PCIe slot 03:00
Overview
Content Tools