Link to the slide deck explaining the ATCA Timing Crossbar: ATCA_Timing_Crossbar.pdf
In order to test the timing signal distribution in an HPS ATCA crate, follow the procedure described below depending on the type of timing system you are using (Normal Conducting (NC) vs Super Conducting (SC)):
Slot 2 FW image
You don't need to load any MPS Link Node firmware into the carrier in slot 2 for this test. The default image "AmcCarrierKU040LoopbackTester" should be enough for this test.
The Timing/MPS RTM has two fiber inputs for timing, labeled EVR0 and EVR1. By convention, SLAC adopted the connection of the Normal Conducting (NC) timing fiber in EVR0 and the Super Conducting (SC) timing fiber in EVR1. The instructions below assume that this convention is in use.
The RF Interlock RTM has only one input for the timing fiber. The input corresponds to EVR0 in the instructions below and either the NC timing fiber or the SC timing fiber can be connected to EVR0.
Normal Conducting (NC) Timing
Login to the supervisory CPU, go to the timing application tool, and source the environment:
# In Prod, $ ssh mcclogin $ ssh softegr@lcls-srv01 [softegr@lcls-srv01 ~]$ ssh laci@<CPU_NODENAME> [laci@<CPU_NODENAME>]$ cd /usr/local/lcls/package/timing/tpg/R1.4.0/buildroot-2019.08-x86_64/bin # In Dev, $ ssh lcls-dev3 [lcls-dev3 ~]$ ssh laci@<CPU_NODENAME> [laci@<CPU_NODENAME>]$ cd /afs/slac/g/lcls/package/timing/tpg/R1.4.0/buildroot-2019.08-x86_64/bin
Configure the timing crossbar in the MPS Link Node carrier (slot 2) to receive the timing signal from the RTM input, and distribute it via the backplane (Here change <SLOT2_IP> by the IP address of the carrier in slot 2)
[laci@<CPU_NODENAME>]$ ./hps_peek -h Usage: ./hps_peek [options] Options: -a <ip address, dotted notation> -y <yaml file>[,<path to timing>] -d Dump raw timing stream -D Dump framed timing -s Dump stats -S <secs> Dump stats over seconds -t <value> Set timingMode/clkSel -P <value> set polarity -X <output>,<value> set xbar output -x ignore crossbar -r reset FPGA -R reload FPGA[laci@<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -t 0 [laci@<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -X 1,0 [laci@<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -X 2,0
"-t 0" indicates NC timing mode and "-t 1" specifies SC timing mode.
Check that the MPS Link Node carrier (slot 2) is locked to the timing system. In this output:
The carrier is locked to the timing system if the "Link" status says "Up",
you should check that both "OUT[1 (FPGA)]" and "OUT[2 ( BP)]" point to "EVR0". This is the timing source for this carrier.
[lcls laci@<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -s buildStamp AmcCarrierMpsAnalogLinkNode: Vivado v2018.2, rdsrv221 (x86_64), Built Tue Oct 16 17:03:41 PDT 2018 by ruckman upTime 4316 seconds OUT[0 (EVR0)] = 0 (EVR0) OUT[1 (FPGA)] = 0 (EVR0) OUT[2 ( BP)] = 0 (EVR0) OUT[3 (EVR1)] = 3 (EVR1) RxRecClkFreq: 119.01 TxRefClkFreq: 119.02 Link : Up RxPolarity : 0 SOFcounts : 360 CRCerrors : 0 DECerrors : 0 DSPerrors : 0 [lcls laci@cpu-bsys-sp01]$
Now, configure the timing crossbar in the application carriers (starting at slot 3 up to the last slot in the crate) to receive the timing signal from the backplane (Here change <SLOTN_IP> by the IP address of the application carrier you are testing):
[lcls laci@<CPU_NODENAME>]$ ./hps_peek -a <SLOTN_IP> -y ../../yamlb/000TopLevel.yaml -t 0 [lcls laci@<CPU_NODENAME>]$ ./hps_peek -a <SLOTN_IP> -y ../../yamlb/000TopLevel.yaml -X 1,2
Check that the application carrier is locked to the timing system. In this output:
The carrier is locked to the timing system if the "Link" status says "Up",
you should check that "OUT[1 (FPGA)]" points to "2 ( BP)".
[lcls laci@<CPU_NODENAME>]$ ./hps_peek -a <SLOTN_IP> -y ../../yamlb/000TopLevel.yaml -s buildStamp AmcCarrierKU040LoopbackTester: Vivado v2016.4, rdsrv222 (x86_64), Built Tue Aug 22 23:28:14 PDT 2017 by ruckman upTime 258230 seconds OUT[0 (EVR0)] = 0 (EVR0) OUT[1 (FPGA)] = 2 ( BP) OUT[2 ( BP)] = 0 (EVR0) OUT[3 (EVR1)] = 0 (EVR0) RxRecClkFreq: 119.01 TxRefClkFreq: 119.01 Link : Up RxPolarity : 0 SOFcounts : 361 CRCerrors : 0 DECerrors : 0 DSPerrors : 0
Super Conducting (SC) Timing
Login to the CPU, go to the timing application tool, and source the environment:
[softegr@lcls-srv01 ~]$ ssh laci@<CPU_NONENAME> [lcls laci@<CPU_NODENAME>]$ # in dev, the path is /afs/slac/g/lcls/package/timing/tpg/<version>/buildroot-2019.08-x86_64/bin [lcls laci@<CPU_NODENAME>]$ cd /usr/local/lcls/package/timing/tpg/R1.4.0/buildroot-2019.08-x86_64/bin [lcls laci@<CPU_NODENAME>]$
Configure the timing crossbar in the MPS Link Node carrier (slot 2) to receive the timing signal from the RTM input, and distribute it via the backplane (Here change <SLOT2_IP> by the IP address of the carrier in slit 2):
[lcls laci@<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -t 1 [lcls laci@<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -X 1,3 [lcls laci@<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -X 2,3
Check that the MPS Link Node carrier (slot 2) is locked to the timing system. In this output:
The carrier is locked to the timing system if the "Link" status says "Up",
you should check that both "OUT[1 (FPGA)]" and "OUT[2 ( BP)]" point to the "EVR1". This is the timing source for this carrier.
[lcls laci@<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -s buildStamp AmcCarrierMpsAnalogLinkNode: Vivado v2018.2, rdsrv221 (x86_64), Built Tue Oct 16 17:03:41 PDT 2018 by ruckman upTime 4316 seconds OUT[0 (EVR0)] = 0 (EVR0) OUT[1 (FPGA)] = 3 (EVR1) OUT[2 ( BP)] = 3 (EVR1) OUT[3 (EVR1)] = 3 (EVR1) RxRecClkFreq: 185.71 TxRefClkFreq: 185.71 Link : Up RxPolarity : 0 SOFcounts : 928575 CRCerrors : 0 DECerrors : 0 DSPerrors : 0 [lcls laci@cpu-bsys-sp01]$
Now, configure the timing crossbar in the application carriers (starting at slot 3 up to the last slot in the crate) to receive the timing signal from the backplane (Here change <SLOTN_IP> by the IP address of the application carrier you are testing):
[lcls laci@<CPU_NODENAME>]$ ./hps_peek -a <SLOTN_IP> -y ../../yamlb/000TopLevel.yaml -t 1 [lcls laci@<CPU_NODENAME>]$ ./hps_peek -a <SLOTN_IP> -y ../../yamlb/000TopLevel.yaml -X 1,2
Check that the application carrier is locked to the timing system. In this output:
The carrier is locked to the timing system if the "Link" status says "Up",
you should check that "OUT[1 (FPGA)]" points to "2 ( BP)".
[lcls laci@<CPU_NODENAME>]$ ./hps_peek -a <SLOTN_IP> -y ../../yamlb/000TopLevel.yaml -s buildStamp AmcCarrierKU040LoopbackTester: Vivado v2016.4, rdsrv222 (x86_64), Built Tue Aug 22 23:28:14 PDT 2017 by ruckman upTime 258230 seconds OUT[0 (EVR0)] = 0 (EVR0) OUT[1 (FPGA)] = 2 ( BP) OUT[2 ( BP)] = 3 (EVR1) OUT[3 (EVR1)] = 3 (EVR1) RxRecClkFreq: 185.71 TxRefClkFreq: 185.71 Link : Up RxPolarity : 0 SOFcounts : 928575 CRCerrors : 0 DECerrors : 0 DSPerrors : 0
Overview
Content Tools