Testing KCU/GPU interaction

The axi-pcie-devel package is supplied with a test application to verify KCU/GPU interaction.  It is comprised of the interCardGui application and the test_dma program.  These are used with the InterCardTest firmware that should be loaded on the KCU.  The interCardGui brings up a devGui-like GUI that provides access to the registers provided by the InterCardTest firmware on the KCU.

Running the test

After building and installing the datagpu driver using comp_and_load_drivers.sh, go to the gpu directory in a axi-pcie-devel check-out to launch the interCardGui:

cd ~/git/axi-pcie-devel/software/gpu/
../scripts/interCardGui

There was an issue launching it from another directory at one point, but this may have been fixed.  The GUI connects with the /dev/datagpu_0 device by default, but another may be select by using the --dev option.  The following GUI should come up:

Depending on the state of the system, it may be desirable to reset the firmware state by opening the AxiPcieCore.AxiVersion blocks and clicking on UserRst to clear out any previous state:

From another prompt, start the test_dma program:

cd ~/git/axi-pcie-devel/software/gpu/
sudo ./bin/test_dma

This program also interacts with /dev/datagpu_0 by default.  Another device may be selected using the -d option.  The program will print some stuff and then pause:

(rogue_v6.1.3) claus@drp-srcf-gpu001:gpu$ sudo ./bin/test_dma
[sudo] password for claus:
Total devices 1
Selected device: NVIDIA RTX A5000
Global memory: 24026 MB
64-bit Memory Address support
Setting write pointer: 0x7f9885600000 - 65536
Setting read pointer: 0x7f9885610000 - 65536
Done with pointers
Mapping FPGA registers
swFpgaRegs = 0x7f98a36bd000
Enabling IO memory for FPGA registers
Mapping write start register
Mapping read start register
Mapped FPGA registers
Create stream
write memory
Trigger write
Wait memory value

At this point it is useful to click Read All at the bottom of the InterCardGui to verify that various registers look reasonable.