Page History

...

Filtering from Python Modules

A python module can use standard psana features to skip events as discussed above. However to use the Translator filtering features, a Python module will have to add data to the event store that the Translator knows about, as discussed above, and that psana knows how to convert into a C++ type for C++ modules to see. Presently the only types that a Python module can add to the event store which will be seen by C++ modules are ndarrays. A python module will need to add one of the ndarrays that the translator knows about to filter events.

...

C++ modules can register new types. Note, this is an advanced feature that requires familiarity with the Hdf5 programming in C.  The present interface supports simple types, but will be difficult to use for more complex types. An example is found in the file Translator/src/TestModuleNewWriter.cpp. We go through the example here. First a module will define the data type that they want to store - simple structs of basic types are easiest. This type is a simple C struct of native types in the C language:

struct MyData {
  int32_t eventCounter;
  float energy;
};

...

The function createMyDataHdf5Type must return an hdf5 type for MyData.  The void * that it is being passed will point to an actual instance of the MyData struct that was found in the eventStore.  Because MyData is so simple, the function createMyDataHdf5Type does not need to use this argument.  However in general, a more complex type may include arrays of different sizes, and so the exact hdf5 type that describes the data cannot be determined without looking at the object.

...

The special type, HdfWriterNew, that is part of the Translator namespace, has the following arguments:

•  the the C++ std::type_info pointer for the new type being registered (&typeid(MyData) in the example)
• the name of the dataset ("data")
• the function that creates the hdf5 type (which we discussed above)
• the function that returns the memory buffer for writing (which we discussed above)

HdfWriterNew also takes an optional fifth argument that users can use to clean up resources.  Since MyData is so simple, there is no need to use this part of the API.  We will create the hdf5 type once, and not worry about closing it.  More complex types may make numerous subtypes along the way, that depend on the object.  If the hdf5 resources created are extensive, the user module will want to close hdf5 resources when they are no longer needed. The create function and optional fifth argument for closing would have to share a persistent structure that indexes these resources and closes them when they are no longer needed.

The key "MyDataWriter" added when putting the newWriter in the event store is not important.  Giving it a distinct name will help with debugging.

...