Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Recovering from errors:  restarting the DAQ

To (re)start the DAQ system open the icon labeled "Restart DAQ" on the DAQ console:

Image Removed

In XPP or XCS use Use the "restartdaq <-w>" command. If you only want to stop the DAQ, call "stopdaq".

 

Or  open the icon labeled "Restart DAQ" on the DAQ console:

Image Added

Using the script in a terminal will give you a printout

 

Anchor
Troubleshooting the DAQ
Troubleshooting the DAQ

...

Recovering from errors:  restarting AMI (the client)

In XPP or XCS use Use the "startami" command. This will start a second client if you are not on the main DAQ machine and it will restart the DAQ AMI client if run on the main DAQ machine (after asking for confirmation, so you will need to use a terminal). Should the server side by unhappy, you will need to restart the DAQ.

...

Make sure that all dss nodes are selected. If you needed to take a node out due to problems, you need to edit the <hutch>.cnf file. You can use "serverStat <ip>" to get the name for the node that causes a problem and then edit the list of dss_nodes to exclude this node. If the problematic node is the last one, you might see that you have to reselect the Bld upon a restart. This means that the DAQ will make you allocate twice (the first time it'll fail with a complaint about a Bld change).

The DAQ shows

...

"NO CONNECT" in procStat: 

use "serverStat <DAQ device alias> <command>". "cycle" will power cycle the node with some time between off/on in the script. It will tie up the terminal, so if you have to deal with several nodes, you can also call "serverStat <DAQ device alias> off" and "serverStat <DAQ device alias> on" explicitly. Remember to wait a few second between off/on. After the script returns from turning the node(s) on, continue to run "serverStat <ip/node name>" until both pings work. If you can ssh into the node(s), you can restart the DAQ.

The DAQ shows an error message, requesting a restart,  indicating a given IP as culprit

XPP/XCS: use Use "serverStat <ip>" to check if both interfaces of the node in question are up. This script will also tell you which machine has the issue.

...

The IP is a dss-node: here you have an additional option: you can edit the <hutch>.cnf file to take out the node: look for "dss_nodes = [....]" and take out the problematic node. 

(XPP/XCS specific):Is this the first node, notify the PCDS-POC as this node runs a special process that will NOT stop when the DAQ stops. Depending Depending on the data rate, you can run with 2 or 3 nodes (cspad + other detectors: 3 nodes, two EPIX: 2 nodes). As we run all the data into a single ami session and the best mapping allows max one ami node/dss node, you have less ami power if you have less dss nodes.

...

Go to the EVR configuration (with the exception of pgp triggered devices as the EPIX), if you are using aliases in your DAQ, it should be straightforward to find the right trigger channel. Look at all available EVR cards. For the run trigger setting for e.g. CsPad/Cs140k detectors, it might be best to contact your POC. 

My data does not seem to be moving well:

...