...

ndarrays (up to dimension 4 of the standard integral and float types, float types and C++ complex types) and as well as std::string's that are written into the event store will be written to the hdf5 by default.  ndarrays can be passed to the Translator by Python modules as well as C++ modules. These events can be filtered as well.  See the section Psana Configuration File and all Options for more details.

Registering New Types

C++ modules can register new types. An example is found in the file Translator/src/TestModuleNewWriter.cpp. We go through the example here. First a module will define the data type that they want to store - simple structs of basic types are easiest:

struct MyData {
  int32_t eventCounter;
  float energy;
};

Next, the module must define functions that create the hdf5 type for MyData, and fill a buffer to be written to the hdf5 file.  These functions must satisfy a particular signature:

  typedef hid_t (*CreateHDF5Type)(const void *userDataType);
  typedef const void * (*FillHdf5WriteBuffer)(const void *userDataType);

Here is what these functions might look like for MyData:

#include "hdf5/hdf5.h"
#include "MsgLogger/MsgLogger.h"

hid_t createMyDataHdf5Type(const void *) {
  static bool firstCall = true;
  static hid_t h5type = -1;
  if (not firstCall) return h5type;
  firstCall = false;
  h5type = H5Tcreate(H5T_COMPOUND, sizeof(MyData));
  
  herr_t status1 = H5Tinsert(h5type, "eventCounter", 
                             offsetof(MyData,eventCounter), 
                             H5T_NATIVE_UINT32);
  herr_t status2 = H5Tinsert(h5type, "energy", 
                             offsetof(MyData,energy), 
                             H5T_NATIVE_FLOAT);
  if ((h5type < 0) or (status1 < 0) or (status2<0)) {
    MsgLog("mydata",fatal,"unable to create MyData compound type");
  }
  MsgLog("mydata",trace,"Created hdf5 type for MyData  " << h5type);  
  return h5type;
}

const void * fillMyDataWriteBuffer(const void *data) {
  return data;
}

The function createMyDataHdf5Type must return an hdf5 type for MyData.  The void * that it is being passed will point to an actual instance of the MyData struct that was found in the eventStore.  Because MyData is so simple, the function createMyDataHdf5Type does not need to use this argument.  However in general, a complex type may include arrays of different sizes, and so the exact hdf5 type that describes the data cannot be determined without looking at the object.

The function fillMyDataWriteBuffer receives a void pointer to an instance of MyData that was found in the eventStore.  The function must then return a void pointer to a memory buffer that holds the data to be written into the hdf5 file.  Since MyData is so simply, the memory layout of the C++ object coincides with that of the hdf5 type, so we can simply return the original pointer to MyData. For more complex types, this will not be the case and fillMyDataWriteBuffer will have to manage a buffer of memory that persists after the function is called. It would then transfer the data in the complex C++ object into this memory buffer.

To register this new type for writing in the system, the user module must, in the beginJob() function, put a special object in the eventStore.  The Translator module will look for these special objects when it handles the beginJob() function. Then the user module can add MyData into the eventStore during the event() function:

#include "Translator/HdfWriterNew.h"

...
class TestNewHdfWriter : public Module {
public:
  TestNewHdfWriter(std::string moduleName) : Module(moduleName) {}
  virtual void beginJob(Event& evt, Env& env) {
    boost::shared_ptr<Translator::HdfWriterNew> newWriter = 
      boost::make_shared<Translator::HdfWriterNew>(&typeid(MyData), 
                                                   "data", 
                                                   createMyDataHdf5Type, 
                                                   fillMyDataWriteBuffer);
    evt.put(newWriter,"MyDataWriter");
  }
  
  virtual void event(Event& evt, Env& env) {
    boost::shared_ptr<MyData> myData = boost::make_shared<MyData>();
    myData->eventCounter = 11;
    myData->energy = 23.239;
    evt.put(myData,"example");
  }
};

he module Note that for a new type that is a simple struct of native types, it is relatively simple to create the hdf5 type, and we can use the object is

Psana Configuration File and all Options

...