Page History

This script lives in /reg/g/psdm/tutorials/examplePython/userSmallHDF5.py.

We do not assume we know the final size of the dataset. The example demonstrates using the hdf5 chunked storage and resize functions to grow a dataset. The same sort of pattern can be used to write in-memory data at the end of a run, but it is easier because at that time the size of the dataset it known.

Code Block

language	python

import numpy as np
import psana
import h5py

NUM_EVENTS_TO_WRITE=3

ds = psana.DataSource('exp=xpptut15:run=54:smd')

h5out = h5py.File("userSmallData.h5", 'w')
saved = h5out.create_dataset('saved',(0,), dtype='f8', chunks=True, maxshape=(None,))

cspad = psana.Detector('cspad', ds.env())

for idx, evt in enumerate(ds.events()):
    if idx > NUM_EVENTS_TO_WRITE: break
    calib = cspad.calib(evt)
    if (calib is None): continue
    saved.resize((idx+1,))
    saved[idx] = np.sum(calib)

h5out.close()

Good tools to inspect an h5 files are h5ls and h5dump. For example, doing:

 h5ls -d -r userSmallData.h5

shows the dataset and its values:

Code Block
[cpo@psana1511]$ h5ls -d -r userSmallData.h5 / Group /saved Dataset {4/Inf} Data: (0) 23773.12109375, 135712.25, 65513.67578125, 16749.18359375

A more advanced tutorial on saving data to an hdf5 file can be found on the page: More Advanced Tutorial on Saving Output in Hdf5

NOTE: psana now provides a more automated way to do this that most users use as shown here.

Page tree

Versions Compared

Old Version 5

New Version Current

Key