...

Below we discuss the new translator, which is called psana-translate. The input and output formats have not changed between o2o-translate and psana-translate, however if filtering events with psana-translate, the h5 output file will include new groups discussed below.

Running the Translator

If the Interface Controller has been configured to run psana-translate, then psana can be run through the interface controller.

Running the Translator

psana-translate is in the final stages of testing and development, and consequently is not yet used for automatic translation by the Interface Controller. nor is it part of the current analysis release. To run it, you will have to create a release directory based on ana-current, and add the development package. The commands are:

newrel ana-current myrelease
cd myrelease
sit_setup
addpkg Translator
scons

At this point, if you are in the release directory you just made (myrelease) you should be able to run psana-translate. It is run Otherwise, run psana-translate as you would any other psana module. Either through psana command line options or by writing a psana configuration file.  The module is Translator.H5Output.

...

• filter out whole events from translation
• filter out certain data, by data type, or by data source
• write ndarray's that other modules add to the event store
• write std::string's that other C++ modules add to the event store
• advanced: have a C++ module register a new type for translation

Filtering Events

Since psana-translate runs as a psana module, it is possible to filter translated events through psana options and other modules. psana options allow you to start at a certain event, and process a certain number of events.  Moreover a user module that is loaded before the Translator module can tell psana that it should not pass this event on to any other modules, hence the Translator.H5Output module will never see the event and it will not get translated.

...

• For each calib cycle, it will make a filtered group
  • For instance, if the file has the group /Configure:0000/Run:0000/CalibCycle:0000, then it will also have:
    the group: /Configure:0000/Run:0000/Filtered:0000
• within each Filtered group, a time dataset that holds event id's of the filtered events.  

Suppose a user modue module has made some measurements that indicate this event should be filtered (for instance the beam energy is wrong). These measurements can be recorded in the hdf5 file by adding data to the event store that the Translator knows how to write.  As discussed below, the Translator can write ndarrays and strings as well as simple new types that user modules register. If a user module implements event to do the following:

...

Filtering from Python Modules

A python Python module can use standard psana features to skip events as discussed above. However to use the Translator filtering features, a Python module will have to add data to the event store that the Translator knows about, as discussed above, and features, a Python module will have to add data that psana knows how to convert into a C++ type for C++ modules to see. Presently the only types that a Python module can add to the event store which will be seen by C++ modules are a number of ndarrays. A python Python module will need to add one of the ndarrays that the translator knows about these ndarray types to filter events, the data of the ndarray will be recorded in the hdf5 file.

Filtering Types

The psana.cfg file accepts a number of parameters that will filter out sets of psana types.  For example setting

...

C++ modules can register new types. Note, this is an advanced feature that requires familiarity with the Hdf5 programming in C.  The present interface supports simple types, but will be difficult to use for more complex typesPresently this feature is only suitable for simple types. An example is found in the file Translator/src/TestModuleNewWriter.cpp. We go through the example here. First a module will define the data type that they want to store. This type is a simple C struct of native types in the C language:

...

The key "MyDataWriter" added when putting the newWriter in the event store is not important.  Giving it a distinct name will help with debuggingcan help debug problems that may arise in the Translator.

The translator, in each calib cycle, will make the following groups (for example in calib cycle 0):

...

Technical Difference's with o2o-translate

Below is a list of technical differences between psana-translate and o2o-translate. These differences should not affect end users.

• File attributes runNumber, runType and experiment not stored, instead expNum, experiment, instrument and jobName are stored (from the psana Env object)
• The attribute :schema:timestamp-format is always "full", there is no option for "short"
• The output file must be explicitly specificed in the psana cfg file. It is not inferred from the input.
• The File attribute origin is now psana-translator as opposed to translator
• The end sec and nanoseconds are not written into the Configure group at the end of the job as there is no EventId in the Event at the end.
• integer size changes - a number of fields have changed size, a few examples are below.  In one quirky case, this caused translation to be different.  The reason was that the data was uninitialized, and the new 32 bit value was different than the old 16 bit value. Data produced from 2014 onward will not include unitialized data in the translation, users will not have to worry about.  Unitialized data is very rare in pre 2014 data and, due to its location, not likely to be used in analysis.
• A few Examples of field size changes:
  • EvrData::ConfigV7/seq_config - sync_source - enum was uint16, now uint32
  • EvrData::ConfigV7/seq_config - beam_source - enum was uint16, now uint32
  • Ipimb::DataV2 - source_id was uint16, now uint8
  • Ipimb::DataV2 - conn_id was uint16 now uint8
  • Ipimb::DataV2 - module was uint16, now uint8

...