A video demo from Matt Weaver on Oct. 9, 2020: https://pswww.slac.stanford.edu/swdoc/tutorials/lcls2_daq_demo.mp4
Launching
hutch | machine | account | directory |
|---|---|---|---|
tmo | tmo-daq | tmoopr | /cds/group/pcds/dist/pds/tmo/scripts |
rix | rix-daq | rixopr | /cds/group/pcds/dist/pds/rix/scripts |
asc | det-daq | detopr | /cds/home/opr/detopr/scripts |
ued | ued-daq | uedopr | /cds/home/opr/uedopr/daq |
From the hutch console (or remotely after "ssh tmo-daq -l tmoopr"):
cd /cds/group/pcds/dist/pds/tmo/scripts
source setup_env.sh
./restart_daq.sh (for TMO DAQ)
./restart_fee.sh (for FEE DAQ)
Launching the DAQ opens the GUI windows:
ProcStat
DAQ Control
Trigger Monitoring
AMI Client
Closing
./stop_daq.sh (for TMO DAQ)
./stop_fee.sh (for FEE DAQ)
ProcStat: Status of DAQ Processes
This window summarizes the status of the running processes used by the DAQ. Interaction with this GUI is only for diagnostic purposes.
DAQ Control
This window is used to manage detector readout configuration, select detectors which are included in the DAQ, and control the running and recording of the detector readouts.
The "Configuration" section has an "Edit" button for opening the detector configurations, as seen on the right. The operator may select from any of the detectors supported. The resulting GUI is a tree of configuration parameters with typically three sections: "help", "user", and "expert". The help section is a read-only textual description of the user fields, those used most often. Changes may be entered and saved with the "Apply" button.
The "Partition" section is for choosing the detectors that will take part in the DAQ. The "Select" button opens the selection GUI. It's important that all the required detectors are selected. Also, a readout group "grp" may be assigned. Detectors in the same readout group are triggered by the same eventcode.
Once the set of detectors is selected, the running and recording is managed by the "Control" section of the GUI. The partition automatically enters the "Allocated" state. The user may advance to the RUNNING state to begin acquiring data. Detector readouts will be configured when passing through the CONFIGURED state. The option to record data is selected by clicking the cylindrical hard disk icon; no recording is indicated by a bold X through the icon. A new recorded run may be started by backing out to the CONFIGURED state before entering the RUNNING state again. The RESET state is used to recover from errors, after which the partition must be selected again (above paragraph). Note that BLD may not configure if it is not being generated by the sources, for example, if beam is not being delivered to the soft x-ray line.
A full diagram of the state machine (and the transitions, like BeginRun, that move between states) is shown here.
Detector Triggering: Event Codes
One of the important things to control is when detectors get triggered. This is set in the timing-system configuration, typically named "timing" in the DAQ configuration editor (see lower right). Each detector can be assigned to "readout group" in the "Select Partition" window (shown upper right, in the "grp" column). All detectors assigned to the same readout group are triggered by the same timing-system event code. At the time this is being written (June, 2021) TMO uses group 0 while RIX uses group 2 (for beam-rate detectors) and 5 (for slow 10Hz cameras). NOTE: it is a requirement (from the online event-builder software) that the highest-rate readout group be the same as the platform number in the .cnf file (e.g. 0 for TMO, 2 for RIX).
A diagram showing which event codes are relevant in different parts of LCLS is here. When there is beam going to the soft X-ray line the beam-rate readout groups should be set to either event-code 120 or 136 (the difference between these two is that 120 also triggers when the beam has been intentionally "dropped" by a BYKIKS signal). These are typically the event codes that are used to trigger devices outside the DAQ (e.g. BLD or non-daq "controls" cameras). When there is no beam in the soft X-ray line these event codes will not fire. For debugging the DAQ system when they are not firing, one should use the fixed-rate event codes 10,11,12,13,14,15 (120Hz, 60Hz, 30Hz, 10Hz, 5Hz, 1Hz). These event-codes should also be used for slower non-beam-rate detectors. Note that the event codes 40,41,42,... should NOT be used: those are for use only by the hard X-ray line.
DAQ Monitoring
To monitor the status of the DAQ, open a browser and go to one of the following "grafana" web pages:
It is important to watch for large dead time and large damage. These can be caused by crashing DAQ processes or incorrect DAQ configuration (e.g. event codes used to trigger the DAQ).
There are drop-down menus on the top left for the instrument (tmo, rix, ued, ...) partition number (also called "platform number" in, for example, tmo.cnf) and readout groups (described above). There are four plots the should be monitored: the top-right shows the trigger rates for various detectors, middle-right shows the integrated number of damaged events for each detector (reset at each run boundary) the bottom-right shows the instantaneous damage-rate for each detector, and the left plot shows overall trigger rates and deadtime. On the upper right one can also select the time range to display and update interval.
For experts a TEB Grafana page.
AMI: Graphical Realtime Data Analysis
This window is used to operate the online monitoring. Instructions for the use of AMI may be found here (ami).
Overview
Content Tools