Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The EventCodes tab is where you can select to record the arrival of various eventcodes and choose the eventcode from which the trigger pulses will be generated.  The Sequencer Codes section is now depreacted and cannot be selected anymore.  An eventcode can be designated as Readout, Command, Control(Transient), and Control(Latch).  An eventcode designated as Readout will generate the trigger pulses on each occurrence.  There should be only one eventcode designated as such.  An eventcode designated as Command will generate a software event which can be used for software command generated readout (like for a Princeton camera).  That software generated event will collected along with the other detectors on the next occurrence of a Readout event.  An eventcode designated as Control(Transient) will be recorded in the datastream for each occurrence with the specified delay and duration (in units of readout occurrences).  This allows a record to track the occurrence of some event triggered from that eventcode elsewhere, like the pump laser for instance. It is also possible to ass add a list of event codes to record in the <hutch>.cnf file, removing the need to set commonly used event codes in the EVR config if they are needed for recording only. An eventcode designated as Control(Latch) will be recorded in the datastream on every readout event following its occurrence until the complementary Control(Latch) code is received.  For example, a pulse picker state can be recorded by the occurrence of the commands responsible for its "Open" and "Close" operation.  For many experiments, only the Readout eventcode needs to be designated.

...

In the DAQ, you have the option to select "Sync Sequence" where you have to enter the sequence you want to sync to. The DAQ will now start the sequencer on the "enable" transition and stop it on "disable". This is useful for scans in which a sequence will be repeated for each step. Please check that the timing of the sequence running the the DAQ works, in particular if you have slow to configure detectors in the DAQ.

 


Anchor
Recovering from errors
Recovering from errors

Recovering from errors:

...

LCLS-1 DAQ Tier-1 Troubleshooting

Scans etc (presentation from Oct 2021)

Running scans, PVs, elog, and smalldata tools overview

To (re)start the DAQ system open the icon labeled "Restart DAQ" on the DAQ console:

Image Removed

In XPP or XCS use the "restartdaq <-w>" command. If you only want to stop the DAQ, call "stopdaq".

 

...

Troubleshooting the DAQ

Ami does not work: the DAQ is fine after a restart, but I see no updates in ami

Make sure that all dss nodes are selected. If you needed to take a node out due to problems, you need to edit the <hutch>.cnf file.

The DAQ shows an error message, requesting a restart,  indicating a given IP as culprit

XPP/XCS: use "serverStat <ip>" to check if both interfaces of the node in question are up. This script will also tell you which machine has the issue.

One/both of the pings fails: use "serverStat <ip/node name> cycle" to power cycle the machine. After the script returns, continue to run "serverStat <ip/node name>" until both pings work. If you can ssh into the node, you can restart the DAQ.

Otherwise: decide if you'd rather restart the DAQ and hope for the best. Power cycling a machine takes a few minutes.

The IP is a dss-node: here you have an additional option: you can edit the <hutch>.cnf file to take out the node: look for "dss_nodes = [....]" and take out the problematic node. 

(XPP/XCS specific):Is this the first node, notify the PCDS-POC as this node runs a special process that will NOT stop when the DAQ stops. Depending on the data rate, you can run with 2 or 3 nodes (cspad + other detectors: 3 nodes, two EPIX: 2 nodes). As we run all the data into a single ami session and the best mapping allows max one ami node/dss node, you have less ami power if you have less dss nodes.

One of my DAQ devices has a problem (damage,....):

use "serverStat <DAQ device alias>" to check on the health of the node. Most likely it is prudent to power-cycle this node. Does this not help, you should power cycle the detector/camera/device itself as well. If the problematic detector is a big CsPad, please note that after you turn the detector off, you will need to also power cycle the concentrator as it will not see all the quads when you power up again!

My data does not seem to be moving well:

check the status of the data moving here: Data Mover Monitoring, this page can be seen one the main "pswww" page as well. Take our the nodes with the issue, assuming your problem is limited to a single node. If it's wider spread, it might warrant a call.

Technical note:

"serverStat" at this moment lives are /reg/g/xpp/scripts, but should work in all hutches. A better place for common scripts will created & populated soon.