Current Development Machine Name
lcls-pc83236
Preparing git ssh keys
Add your ssh keys to git (This is unfortunately necessary because the .gitmodules file in lcls2-pcie-apps uses the "git" form of the URL instead of the "http" form):
https://help.github.com/articles/adding-a-new-ssh-key-to-your-github-account/
Also note that the firmware requires a new version of git that supports "links to large files" (lfs). Add /afs/slac/g/reseng/git/git/bin to PATH.
Conda Commands to Create Rogue Environment
This is in addition to the other packages that must be built for the rogue library (see next step). These conda commands are derived from:
https://github.com/slaclab/rogue/blob/master/Readme_python3.txt
conda env create -n timetool source activate timetool conda install pyyaml conda install pyzmq conda install -c conda-forge parse conda install click conda install MySQLdb conda install -c bioconda mysqlclient conda install -c conda-forge pyro4 conda install numpy pip install recordclass
Building Rogue
git clone https://github.com/slaclab/rogue.git
Needs a conda env with a bunch of stuff (previous step). Follow build instruction files in the rogue root directory README files (although I suggest setting up a conda python3 env (previous step) instead of using pip install):
https://github.com/slaclab/rogue/blob/master/Readme_build.txt
cd rogue
git submodule init
git submodule update
make
To run, source this script:
https://github.com/slaclab/lcls2-pcie-apps/blob/master/software/TimeTool/setup_env_template.csh
Some applications are not built by default. cd to directory and make.
Building Firmware
Follow instructions in the README.md here (make sure to use the modern AFS version of git described here so you can use git-lfs):
https://github.com/slaclab/lcls2-pcie-apps.git
git submodule init
git submodule update
(old: source /afs/slac/g/reseng/xilinx/vivado_2017.3/Vivado/2017.3/settings64.sh #this isn't working on 1/31/2018. sourcing below instead)
source lcls2-pcie-apps/firmware/setup_env_slac.sh
if you want to store the output of "make" on your local machine: in the "firmware/" directory, "ln -s /u1/sioan/build ."
cd firmware/targets/TimeToolKcu1500
make
Making Vivado communicate with board over USB/JTAG
Larry has some slides on how to program the flash chips (mt25qu512) on the KCU1500 via USB/JTAG. Startup "vivado" after setting up the firmware
Do this to program with flash chips on the KCU1500 for the first time. Before programming lspci will show:
02:00.0 Serial controller: Xilinx Corporation Device 8638 (rev ff)
After programming powercycle the machine. Then lspci should show:
02:00.0 Signal processing controller: SLAC National Accelerator Lab PPA-REG Device 2030
#https://www.xilinx.com/support/answers/59128.html
1) Disconnect all Xilinx USB cables from the host computer.
2) Open a shell or terminal console.
3) Extract the driver script and its support files to a local drive of the machine where the cable will be used by typing:
#must cd to directory. Can't run install_drivers from arbitrary directory.
cd /afs/slac.stanford.edu/g/reseng/xilinx/vivado_2017.4/Vivado/2017.4/data/xicom/cable_drivers/lin64/install_script/install_drivers/
sudo ./install_drivers
#now vivado hardware manager will see the kcu1500 board.
#instructions to program using vivado hardware manager in link below
SLAC Driver
Build/install the datadev.ko driver using the instructions here:
https://github.com/slaclab/lcls2-pcie-apps
This needs to be done on the machine where the KCU1500 lives. You need sudo on the machine to install the driver.
Programming FPGA over PCI
After the first programming (and power-cycling) described above, use this script to reprogram:
https://github.com/slaclab/lcls2-pcie-apps/blob/master/software/TimeTool/scripts/updateProm.py
TimeTool Software Files
lcls2-pcie-apps/firmware/applications/TimeTool/python/TimeTool.py: a description of the "addValue" register
lcls2-pcie-apps/firmware/submodules/surf/python/surf/protocols/clink/*.py: descriptions of the "clink" (cameralink) parameters
lcls2-pcie-apps/software/TimeTool/python/TimeToolDev.py: top level class called by gui.py. opens /dev/datadev_0, glues together various register maps using classes like ClinkTest, TimeToolCore, dataWriter
In TimeToolDev.py:
- self.add adds registers and associated GUI control
Settings Needed To Run Camera
Use channela to talk to the front-end board in the "Variables" tab:
- send escape right away after powering up
- linkmode: medium
- datamode: 8bit
- framemode: our camera only gives a line valid (indication that there is valid data) so we need to set to "line"
- tapcount: our camera sends 4 bytes where it could send 6
- baudrate: 9600
- use sendGCP to test serial link is working (output should appear on terminal)
- swcontrolvalue/swcontrolen: bits for hardware vs software trigger (0/0 internal trigger). These bits control CC1 through CC4.
Now tell the camera how to send data using these three-letter-commands (TLC) in the "Commands" tab:
- CLM 1 (cameralink medium mode)
- SVM 1 (test pattern ramp)
- SSF 1 (software trigger rate 1Hz, although seems to read back as 6Hz? and 2 reads back as 12Hz?)
Now enable triggers:
- in "Variables" tab set DataEn for channel A to true
- Framecount field should increment at 6Hz
- Dropcount field counts "3 channels misaligned" errors, and so should stay at 0
Camera output should appear on terminal after setting ClinkTop->ChannelA->DataEn to True
Output of SendGCP:
Got Response: Got Response: Model P4_CM_02K10D_00_R Got Response: Microcode 03-081-20296-13 Got Response: CCI 03-110-20294-03 Got Response: FPGA 03-056-20470-03 Got Response: Serial # 12102856 Got Response: BiST: Good Got Response: Got Response: DefaultSet 1 Got Response: Ext Trig Off Got Response: Trig Overlap Off Got Response: Line Rate 1 [Hz] Got Response: Meas L.R. 6 [Hz] Got Response: Max L.R. 19607 [Hz] Got Response: Exp. Mode Timed Got Response: Multi Exp. Mode Off Got Response: Exp. Time[0] 50000 [ns] Got Response: Meas E.T.[0] 50000 [ns] Got Response: Max E.T. 3000500 [ns] Got Response: Got Response: Test Pat. 1:Ramp1 Got Response: Direction Internal, Forward Got Response: TDI Stages 2 Got Response: Vert. Bin 1 Got Response: Hor. Bin 1 Got Response: Flat Field Off Got Response: Offset 0 Got Response: System Gain 1.00 Got Response: Mirror Off Got Response: AOI Mode: Off Got Response: Scan Type Line Scan Got Response: CL Speed 85MHz Got Response: CL Config Medium Got Response: Pixel Fmt 8 bits Got Response: CPA ROI 1-2048
Notes From Matt On Evr Firmware
wrapper to transceivers:
https://github.com/slaclab/lcls-timing-core/blob/release-lcls2/LCLS-II/core/rtl/TimingGthWrapper.vhd
decoding the output:
https://github.com/slaclab/lcls-timing-core/blob/release-lcls2/LCLS-II/core/rtl/TimingCore.vhd
this record has a strobe that says when it's valid:
appTimingBus : out TimingBusType;
has the record structure:
https://github.com/slaclab/lcls-timing-core/blob/release-lcls2/LCLS-II/core/rtl/TimingPkg.vhd
strobe : sl; -- which clock cycle it is valid
valid : sl; --
message : TimingMessageType; -- for lcls-II
stream : TimingStreamType; -- for lcls-I (eventcodes in this record)
v1 : LclsV1TimingDataType;
v2 : LclsV2TimingDataType;
modesel : sl; -- LCLS-II selected -- tells us the mode, another register sets it
axi-stream: amba-xilinx-interconnect: no address involved, like a port, push/acknowledge
axi: full memory interface: address/values and can burst multiple values
axi-lite: used for register interfaces: 32-bit value with address
What we learned about timing stream:
eventcode is part of TimingStreamType
timingstreamrx (timing message) and timingrx (timing message) have outputs of type timingstream
timingrx instantiates both LCLS-I and LCLS-II timing streams:
- lcls1 is timingstreamrx
- lcls2 is timingframerx
timing core instantiates timingrx
TimingCore outputs TimingBusType and TimingStreamType is a member of this and has the eventcodes.
timingcore is instantiated by EvrFrontEnd
EvrFrontEnd is instantiated by Hardware. Hardware has TimingBusType as a local variable. This is the highest it gets.
Hardware is instantiated by the TimeToolKcu1500
type AxiStreamMasterType is record
tValid : sl; data is ready to be clocked in or out (equivalent of write enable)
tData : slv(127 downto 0); the actual data that the FIFO transmits
tStrb : slv(15 downto 0);
tKeep : slv(15 downto 0); which bytes from the tData to keep
tLast : sl; identifies the last tData of the frame.
Everything in the following lines are axi extensions called (slac streaming interface) ssi
tDest : slv(7 downto 0); identifies the destination for when using multiple axis on a single bus
tId : slv(7 downto 0); transaction id for handshaking (validation the signal was )
tUser : slv(127 downto 0); user bits. can be used for anything. have been used for start of frame (SOF) and end of frame errors (EOFE),
end record AxiStreamMasterType;
To do:
see where the image comes from
Overview
Content Tools