...

Below we discuss the new translator, which is called psana-translate. The input and output formats have not changed between o2o-translate and psana-translate, however if filtering events with psana-translate, the h5 output file will include new groups discussed below.

Psana-translate User's Guide

...

Would invoke the translator.  It will translate all the xtc files in run 001. This runs with default values for all the translator options. These are the recommended option values to use for translation.  The options include gzip compression at level 1 and no filtering on events or data.

If you are going to use many of the translator options, it will be easier write a command line using psana-translate - a command line wrapper to the Translator.H5Output module. psana-translate is run as

The easiest way to try different translator options to write a psana.cfg file.  Copy the file default_psana.cfg that is included below (this is also in the Translator package directory) and modify option values that you wish.  The file default_psana.cfg includes extensive documentation on all the translator options.

One can also use the command line tool psana-translate.  This is similar to running psana from the command line, but will have a shorter command line when setting many of the Translator options. psana-translate is run as

   psana-translate [simple psana arguments] [--modules=...   psana-translate [psana arguments] --output_file=h5outfile [optional H5Output options] [--save_cfg=filename] input files

For For example

   psana-translate -v -v -n 5 /reg/d/psdm/mec/mec01/xtc/e01-r001-s0*-c0*.xtc --output_file=output.h5 --store_epics=no --Epics=exclude Control=exclude src_filter="exclude NoDetector.0.Evr.2" /reg/d/psdm/mec/mec01/xtc/e01-r001-s0*-c0*.xtc

would use psana options to get debugging output and only read the first 5 events from the mec01 run1 files. Then it uses translator options to exclude all epics data, as well as the Control types (Psana types that start with ControlData::ConfigV) and it excludes data coming from the src NoDetector.0.Evr.2).  Note the use of double quotes to specify the multiword value for the src_filter option.

The easiest way to try different translator options to write a psana.cfg file.  Copy the file default_psana.cfg that is included below (this is also in the Translator package directory) and modify option values that you wish.  The file default_psana.cfg includes extensive documentation on all the translator options.

New Features

With psana-translate, you can

New Features

With psana-translate, you can

• filter out whole events from translation
• filter out whole events from translation
• filter out certain data, by data type, or by data source
• write ndarray's that other modules add to the event store
• write std::string's that other modules add to the event store

...

Since psana-translate runs as a psana module, it is possible to filter translated events through psana options and other modules. psana options allow you to start at a certain event, and process a certain number of events.  Moreover a user module that is loaded before the Translator module can tell psana that it should not pass this event on to any other modules, hence the Translator.H5Output module will never see the event and it will not get translated.psana-translate also provides a C++ interface to filtering that will record the event times of the filtered messages, as well as a optional user log message (to record a note as to why the event was filtered).  A C++ module that is loaded before Translator.H5Ouput would do

A downside of having modules loaded before the translator skip events is that updates to epics pv's, or configuration data will not get recorded. One may also wish to record the reason for filtering the event in the hdf5 file, as well as the event id's for the filtered events.  psana-translate provides an interface for doing these things. To use this mechanism, a module must put an object in the eventStore with a key that starts with

do_not_translate

For example, if a C++ module implements the event method to do the following:

  #include "Translator/doNotTranslate.h"

// define user Module,

virtual void event(Event& &evt, Env& &env) {
 Translator::doNotTranslateEvent(evt, std::string("the beam energy is to low"));
}

For more information on event filtering with logging, see the function doNotTranslate and the example class TestDoNotTranslate in the Translator package.  Using this function will cause a group 'filtered' to be created in each CalibCycle where events are filtered.  The filtered group will include the datasets 'time' (with the even id's) of the filtered events and a 'data' dataset with the log messages.

...

	boost::shared_ptr<int> message = boost::make_shared<int>();        
    evt.put(message,"do_not_translate");
  }

Then none of the event data will get translated in any of the calib cycles.  The Translator will do the following

• For each calib cycle, it will make a filtered group
  • For instance, if the file has the group /Configure:0000/Run:0000/CalibCycle:0000, then it will also have:
    the group: /Configure:0000/Run:0000/Filtered:0000
• within each Filtered group, a time dataset that holds event id's of the filtered events.  

Suppose a user modue has made some measurements that indicate this event should be filtered (for instance the beam energy is wrong). These measurements can be recorded in the hdf5 file by adding data to the event store that the Translator knows how to write.  As discussed below, the Translator can write ndarrays and strings as well as simple new types that user modules register. If a user module implements event to do the following:

// define user Module,

  virtual void event(Event& evt, Env& env) {
    boost::shared_ptr<std::string> message = boost::make_shared<std::string>("The beam energy is bad");        
    evt.put(message,"do_not_translate:message");
    const unsigned shape[1] = {4};
    boost::shared_ptr< ndarray<float,1> measurements = boost::make_shared< ndarray<float,1> >(shape);
    float *data = measurements->data();
    data[0] = 0.4;  data[1] = 1.3; data[2] = 2.2; data[3] = 3.1;
    evt.put(measurements,"do_not_translate:measurements");
  }

Note the use of the key "do_not_translate:xxx" the :xxx is not necessary, but it helps to uniquely quality the event data, and it will affect the group name where the do_not_translate event data is written to.  Since both std::string and a ndarray<float,1> are types that the Translator knows how to write, it will create the following groups and datasets in the hdf5 file:

• /Configure:0000/Run:0000/Filtered:0000/time  - this is as discussed above
• /Configure:0000/Run:0000/Filtered:0000/std::string/message/data  - this will be a dataset of variable length strings, each entry will be the string "The beam energy is bad"
• /Configure:0000/Run:0000/Filtered:0000/NDArray/measurements/data - this will be a dataset where each entry is a 1D array of 4 floats, with the values 0.4, 1.3, 2.2, 3.1
  
  

Filtering Types

The psana.cfg file accepts a number of parameters that will filter out sets of psana types.  For example setting

...