In order to test the timing signal distribution in an HPS ATCA crate, follow the procedure described below depending on the type of timing system you are using (Normal Conducting vs Super Conducting):
Slot 2 FW image
You don't need to load any MPS Link Node firmware into the carrier in slot 2 for this test. The default image "AmcCarrierKU040LoopbackTester" should be enough for this test.
The Timing/MPS RTM has two fiber inputs for timing, labeled EVR0 and EVR1. By convention, SLAC adopted the connection of the Normal Conducting (NC) timing fiber in EVR0 and the Super Conducting (SC) timing fiber in EVR1. The instructions below are assuming that this convention is in use.
The RF Interlock RTM has only one input for the timing fiber. The input corresponds to EVR0 in the instructions below and either the NC timing fiber or the SC timing fiber can be connected to it.
Login to the CPU, go to the timing application tool, and source the environment:
[softegr@lcls-srv01 ~]$ ssh laci@<CPU_NONENAME> [lcls laci@c<CPU_NODENAME>]$ # in dev, the path is /afs/slac/g/lcls/package/timing/tpg/<version>/buildroot-2019.08-x86_64/bin [lcls laci@c<CPU_NODENAME>]$ cd /usr/local/lcls/package/timing/tpg/R1.4.0/buildroot-2019.08-x86_64/bin [lcls laci@c<CPU_NODENAME>]$
Configure the timing crossbar in the MPS Link Node carrier (slot 2) to receive the timing signal from the RTM input, and distribute it via the backplane (Here change <SLOT2_IP> by the IP address of the carrier in slit 2):
[lcls laci@c<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -t 0 [lcls laci@c<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -X 1,0 [lcls laci@c<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -X 2,0
Check that the MPS Link Node carrier (slot 2) is locked to the timing system. In this output:
The carrier is locked to the timing system if the "Link" status says "Up",
you should check that both "OUT[1 (FPGA)]" and "OUT[2 ( BP)]" point to "EVR0". This is the timing source for this carrier.
[lcls laci@c<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -s buildStamp AmcCarrierMpsAnalogLinkNode: Vivado v2018.2, rdsrv221 (x86_64), Built Tue Oct 16 17:03:41 PDT 2018 by ruckman upTime 4316 seconds OUT[0 (EVR0)] = 0 (EVR0) OUT[1 (FPGA)] = 0 (EVR0) OUT[2 ( BP)] = 0 (EVR0) OUT[3 (EVR1)] = 3 (EVR1) RxRecClkFreq: 119.01 TxRefClkFreq: 119.02 Link : Up RxPolarity : 0 SOFcounts : 360 CRCerrors : 0 DECerrors : 0 DSPerrors : 0 [lcls laci@cpu-bsys-sp01]$
Now, configure the timing crossbar in the application carriers (starting at slot 3 up to the last slot in the crate) to receive the timing signal from the backplane (Here change <SLOTN_IP> by the IP address of the application carrier you are testing):
[lcls laci@c<CPU_NODENAME>]$ ./hps_peek -a <SLOTN_IP> -y ../../yamlb/000TopLevel.yaml -t 0 [lcls laci@c<CPU_NODENAME>]$ ./hps_peek -a <SLOTN_IP> -y ../../yamlb/000TopLevel.yaml -X 1,2
Check that the application carrier is locked to the timing system. In this output:
The carrier is locked to the timing system if the "Link" status says "Up",
you should check that "OUT[1 (FPGA)]" points to "2 ( BP)".
[lcls laci@c<CPU_NODENAME>]$ ./hps_peek -a <SLOTN_IP> -y ../../yamlb/000TopLevel.yaml -s buildStamp AmcCarrierKU040LoopbackTester: Vivado v2016.4, rdsrv222 (x86_64), Built Tue Aug 22 23:28:14 PDT 2017 by ruckman upTime 258230 seconds OUT[0 (EVR0)] = 0 (EVR0) OUT[1 (FPGA)] = 2 ( BP) OUT[2 ( BP)] = 0 (EVR0) OUT[3 (EVR1)] = 0 (EVR0) RxRecClkFreq: 119.01 TxRefClkFreq: 119.01 Link : Up RxPolarity : 0 SOFcounts : 361 CRCerrors : 0 DECerrors : 0 DSPerrors : 0
Login to the CPU, go to the timing application tool, and source the environment:
[softegr@lcls-srv01 ~]$ ssh laci@<CPU_NONENAME> [lcls laci@c<CPU_NODENAME>]$ # in dev, the path is /afs/slac/g/lcls/package/timing/tpg/<version>/buildroot-2019.08-x86_64/bin [lcls laci@c<CPU_NODENAME>]$ cd /usr/local/lcls/package/timing/tpg/R1.4.0/buildroot-2019.08-x86_64/bin [lcls laci@c<CPU_NODENAME>]$
Configure the timing crossbar in the MPS Link Node carrier (slot 2) to receive the timing signal from the RTM input, and distribute it via the backplane (Here change <SLOT2_IP> by the IP address of the carrier in slit 2):
[lcls laci@c<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -t 1 [lcls laci@c<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -X 1,3 [lcls laci@c<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -X 2,3
Check that the MPS Link Node carrier (slot 2) is locked to the timing system. In this output:
The carrier is locked to the timing system if the "Link" status says "Up",
you should check that both "OUT[1 (FPGA)]" and "OUT[2 ( BP)]" point to the "EVR1". This is the timing source for this carrier.
[lcls laci@c<CPU_NODENAME>]$ ./hps_peek -a <SLOT2_IP> -y ../../yamlb/000TopLevel.yaml -s buildStamp AmcCarrierMpsAnalogLinkNode: Vivado v2018.2, rdsrv221 (x86_64), Built Tue Oct 16 17:03:41 PDT 2018 by ruckman upTime 4316 seconds OUT[0 (EVR0)] = 0 (EVR0) OUT[1 (FPGA)] = 3 (EVR1) OUT[2 ( BP)] = 3 (EVR1) OUT[3 (EVR1)] = 3 (EVR1) RxRecClkFreq: 185.71 TxRefClkFreq: 185.71 Link : Up RxPolarity : 0 SOFcounts : 928575 CRCerrors : 0 DECerrors : 0 DSPerrors : 0 [lcls laci@cpu-bsys-sp01]$
Now, configure the timing crossbar in the application carriers (starting at slot 3 up to the last slot in the crate) to receive the timing signal from the backplane (Here change <SLOTN_IP> by the IP address of the application carrier you are testing):
[lcls laci@c<CPU_NODENAME>]$ ./hps_peek -a <SLOTN_IP> -y ../../yamlb/000TopLevel.yaml -t 1 [lcls laci@c<CPU_NODENAME>]$ ./hps_peek -a <SLOTN_IP> -y ../../yamlb/000TopLevel.yaml -X 1,2
Check that the application carrier is locked to the timing system. In this output:
The carrier is locked to the timing system if the "Link" status says "Up",
you should check that "OUT[1 (FPGA)]" points to "2 ( BP)".
[lcls laci@c<CPU_NODENAME>]$ ./hps_peek -a <SLOTN_IP> -y ../../yamlb/000TopLevel.yaml -s buildStamp AmcCarrierKU040LoopbackTester: Vivado v2016.4, rdsrv222 (x86_64), Built Tue Aug 22 23:28:14 PDT 2017 by ruckman upTime 258230 seconds OUT[0 (EVR0)] = 0 (EVR0) OUT[1 (FPGA)] = 2 ( BP) OUT[2 ( BP)] = 3 (EVR1) OUT[3 (EVR1)] = 3 (EVR1) RxRecClkFreq: 185.71 TxRefClkFreq: 185.71 Link : Up RxPolarity : 0 SOFcounts : 928575 CRCerrors : 0 DECerrors : 0 DSPerrors : 0