Page History

...

Saving Larger Data to HDF5 ("Translation")

The MPIDataSource pattern can be used to "translate" data from xtc to hdf5.  It offers the following advantages over the old translation method:

• users can choose what data they want to store in HDF5 (e.g. raw image data, calibrated)
• users can use python algorithms (e.g. saving only part of a camera Image) to reduce the output data volume which dramatically speeds up translation
• can be run in parallel on many cores
• datasets are guaranteed to be "aligned"

TIP: If you save larger data to HDF5 (not recommended, for performance/space reasons) be sure to set the gather_interval parameter to 1 in order to avoid using up all machine memory.

...