Table of Contents

SWMR

...

Recently (Jan 29 2015) I (davidsch) took a look at the beta work of the SWMR model (single writer multiple readers) that the hdf5 group is working on. As it is planned now, there are a few limitations that would necessitate changes in how the DAQ records data, as well as in the Translator. Below is an email exchange with the hdf5 group that covers this. The summary is that SWMR will only support the datasets, all the groups have to be known and created before SWMR can take place. They need more funding to support the groups, as well as vlen data.

Another limitation that we gathered after meeting with Quincy at SLAC sometime in late 2014 was that you cannot start moving hdf5 files as soon as they are written. That is once the file is closed, the library needs to do some things to the header of the file. This would affect our data management model were we to write natively to hdf5.

...

This would be a big change for the Translator. It seems we would have to wait for the xtc to be written so that we know what datasets to create, and how long each one is (how many events we took, etc). So we wouldn't be able to do 'live' translation, and let users read parts of the translation (like a translated calib cycle) while the whole run is getting translated (see Outdated: The XTC - to - HDF5 Translator - the MPI split scan section). Also no compression is a big limitation for users moving files offsite. However I would expect faster translation.

...