This section details how to set up custom binning of the full detector images and save the output to an hdf5 file.
The main cube script is ./producers/letsCube.py
, but this file should generally not be modified. The configuration is done through a hutch-specific file of the type ./producers/cube_config_<hutch>.py
.
Config file
The configuration consists of the following main sections (see example below too):
- Custom bin definition: Using a run-based logic, one can define how to bin the data here. If no bins are given for a run, then the unique position of the corresponding scan motor will be used. The former is typically used for time or energy fly-scans and the latter is recommended for regular motor scans, such as phi or laser lens scans.
- Filter definition: Default filter name is
filter1
. Any other filter name will lead to the creation of separate cube files ending with_filtername
. This can be useful in case where an external field is applied on certain shots for example. - Laser on/off: If True will create
_on
and_off
cubed file. - Detectors: Area detectors dictionaries to be cubed are defined here. The options are as follow:
- Pixel intensity threshold (float):
thresADU (float)
- Common mode selection (int/None):
common_mode
This is generally set by your data analysis PoC and should rarely be changed. - Whether to return the image or the calib (individual tiles) (bool):
image
- Not-implemented (ignore for now):
full
- Pixel intensity threshold (float):
- All the variable to add to the cubed file are then added to
varList
. The syntax for each non-area detector variable should follow the structure in the smalldata h5 file. Area detectors dictionaries must be added to this list as well. - Histograms: At the end of the cube, a summary is posted to the elog. A list of variables can be customized here. The IPMs and timing tool are generally considered default. If a variable is also in the filter list, the filter boundaries will be shown in the histogram too. The pixel histograms for the area detector defined are also included in the summary. These plots can help you choose relevant filters and thresholds. Adding event codes to this list might be a good idea in cases where event code-based filters are used.
Example config file for XPP:
import numpy as np # custom bins def binBoundaries(run): if isinstance(run,str): run=int(run) if run>0: return np.arange(-5.,50.,0.2) return None # filters to apply to the data # format: list of [det (field), low, high, name] # 'filter1' is the standard name and will not be added to the h5 file name. filters = [ ['lightStatus/xray',0.5,1.5,'filter1'], ['ipm2/sum',3e2,6e4,'filter1'], ['evr/code_41',0.5,1.5,'custom'] ] # Laser on/off. laser = True # List detectors to be cubed. Area detector have additional options such as threshold # For now only full image or calib works. TODO: Add photon maps. And then any detObjectFunc # Detectors should then be added to varList detDict = {'source':'jungfrau1M', 'full':1, 'image':0, 'thresADU':6.5, 'common_mode':0} varList = ['ipm2/sum','ipm3/sum','diodeU/channels', detDict] # histogram configuration. Usually does not need to be changed # field: destination in smd, list: [low,high,n] or None (then default to some percentile) hist_list = { 'ipm2/sum': [0, 5e4, 70], 'ipm3/sum': [0, 5e3, 70], 'tt/FLTPOS_PS': [-0.5, 0.5, 70], 'tt/AMPL': [0, 0.2, 70], 'tt/FLTPOSFWHM': [0, 300, 70] } # save as tiff file (ignore unless MEC) save_tiff = False
Cube file content
The cubed data are saved in a h5 files, whose fields are the following:
binVar_bins
: bin edges as defined in the config or the default (unique) motor positions. In the latter case, the first value is NaN, as a bin is manually added to catch outliers on the left for the general case.nEntries
: number of shots in each binbinVar
: sum of the binned variable. When binning linearly, binVar/nEntries should return the most accurate position. A field with the binned variable name contains the same information.- All the binned variables defined in
varList
: for the regular detectors (non-area detectors) the syntax follow that of the varList, where/
have been replaced by__
(two underscores). Binned area detectors images or calib are under<detname>_data
. A<detname>_nEntries
field should match thenEntries
field. If not, contact your controls and data PoC.
Example from the config file above:ipm2__sum
ipm3__sum
diodeU__channels
jungfrau1M_data
jungfrau1M_nEntries
<detname>_cfg
: a config group for each area detector that contains the detector config (pedestals, gain map, mask, etc.)
An example notebook that load and plots a cube file can found at: /cds/group/psdm/sw/tools/smalldata_tools/example_notebooks/cube.ipynb
Please don't modify this file, copy it to your home to test and explore.
Overview
Content Tools