Psana Data Processing

This release introduces a major change in how psana orders events. This change allows it to merge IOC control streams with DAQ streams to create events with data from multiple sources (offline event building). IOC control streams include xtcav data. In particular:

  • Psana now merges a datagram from a IOC control stream that is under DAQ control (these are xtc files whose stream number is 80 or above) with a datagram from a DAQ stream when the fiducials are an exact match, and the clock times are close.
  • Likewise the HDF5 Translation labels data from the IOC control streams (such as xtcav data) with the DAQ timestamps. Users will not need to do their own fiducial matching to group data.
  • IOC control stream datagrams that do not match DAQ datagrams are still presented to the user in their own event.
  • Psana now uses a fuzzy clock and fiducial based comparison algorithm to accomplish this. Psana no longer orders datagrams based on the clock time.

Psana has gone through additional changes due to using several datagrams to form a Psana event rather than just one:

  • Psana no longer puts a Dgram in the event. It puts a DgramList in the event. 
    • The Dgram is a low level piece of information that was only accessible through the C++ interface. This is not expected to affect the general user. Special applications that were using psana to order Dgrams will have to work with a DgramList. An example that demonstrates how to work with a DgramList can be found in the psana_examples package: psana_examples/src/DumpDgram.cpp
  • The EventId is taken from the DAQ datagram if available. If there is no DAQ datagram in the event, the EventId comes from the IOC control datagram.
    • Note: plots of clock times will jump when IOC control datagrams arrive with no matching DAQ datagram - this is not a sign of damage.
    • The clock time of merged IOC control stream datagrams is not readily available. If needed, users can work with the DgramList (discussed above) at the C++ level to obtain this information.
  • Interfaces to internally used input processing classes have changed in order to parametrize the starting IOC control stream (80) and maximum tolerable difference between stream clocks (85 seconds). These changes should not affect users.

Please, check your code to see if it's not broken after you switch to this release.

Other improvements

  • psana now automatically merges DAQ IOC streams (s80, s81, ...) with regular ones
  • improvements in the direct access to events in psana
  • improvements in psmon
  • improvements in calibrations and imaging algorithms

New/upgraded packages

  • added fftw package version 1.8.1

  • added Python wrapper package pyFFTW version 0.9.2

Bugs fixed

  • bug fixes in HDF5 translator to address problematic data
    • Data missing a proper beginCalibCycle transition
  • address issue with merging DAQ controlled IOC streams in the HDF5 translator
    • multiple data mapping to the same hdf5 group path

Known Issues

 

  • the HDF5 translator will not translate the DAQ src alias list from both the DAQ and the IOC streams. It tries to translate the DAQ list to stay backward compatible, but which one is translated is undefined. 

  • No labels