Page History
...
- All calibration constants for particular detector id (
detid- index of the detector version) will be kept in a single hdf5 file named as<dettype>-<detid>.h5- Schema of this file contains a few levels which accounts for constant types, time stamp ranges, versions of constants.
- Beside main functionality each level contains dictionary of tagged parameters and dictionary of history records.
- These files are grouped in the directory for detector type; ex.:
cspad, cspad2x2, pnccd, etc.- The same directory contains files with aliases
<aliases1>.alswhich map human readable detector name-and-version with appropriate<dettype>-<detid>.h5file.
- The same directory contains files with aliases
- All detector type folders in regular case are collected under
calibdirectory, although it is assumed that calibration files can be used directly. - The
calibdirectory may be nested in three locations;- repository - experiment-independent space with FS back-up, contains most complete calibration files in terms of time stamp and versions,
- experimental work-space - experiment-dependent space, contains squeezed version of calibration files with time stamp and versions essential for particular experiment,
- local work-space - experiment-independent arbitrary space, with squeezed files similar to experimental work-space.
...
| Code Block |
|---|
/ # top-root-level contains info for the unique detector version dettype (str) # detector type name, ex: cspad, pnccd, etc detid (str) # unique detector id, ex: 01234 tscfile (float) # time stamp of creation of this structure predecessor (str) # name of the previous detector if available or None successor (str) # name of the next detector if available or None tagsdictpars (dict) # dictionary of tagsparameters associated with this version of detector history (dict) # dictionary of history records paired as (tstamp:record) <ctype>/ # folder for calibration type, ex.: pedestals, rms, mask, etc ctype (str) # calibration type name tagsdictpars (dict) # dictionary of tagsparameters associated with calibration type history (dict) # dictionary of history records paired as (tstamp:record) <tstamp-range>/ # folder for time stamp validity range <tstamp>[-<tstamp-end>]/ # folder for validity range. If <tstamp-end> is not specified - then valid to the end tsbegin (float) # time stamp for the beginning of the validity range tsend (float) # time stamp for the end of the validity range defaultv # reference to the default calibration, ex: <vers-tstamp2> tagsdictpars (dict) # dictionary of tagsparameters associated with validity range history (dict) # dictionary of history records paired as (tstamp:record) <vers-tstamp1>/ # folder for version created on tstamp1 tsvers (float) # tstamp1 of this version production calib (ndarray) # calibration data history (dict) # dictionary of history records paired as (tstamp:record) tagsdictpars (dict) # dictionary of tagsparameters, # ex: array size, number of dimensions, shape, data type, experiment, run, comments, author <vers-tstamp2>/ # folder for version created on tstamp2 tsvers (float) calib (ndarray) history (dict) tagsdictpars (dict) <tstamp2>[-<tstamp2-end>]/ # folder for the next validity range. pedestals/ # folder for the next calibration type, pedestals rms/ # folder for the next calibration type, rms |
...
- Detector name consists of a common part
<dettype>and unique part<detid>. - Alias to the detector name should be kept in separate dictionary outside the file scheme.
- Each detector may have optional predecessor, successor, and other parameters kept in the dictionary of tagsparameters.
- Calibration type folders contain info about calibrations of particular type, ex: pedestals, rms, status, mask, background, etc.
- Each calibration type contains a set of time stamp ranges defining calibration validity range. If the second time stamp of the range is missing it is considered as infinity (by the end).
- The time stamp range folder contains tstamp_begin, tstamp_end (int) values, dictionary of tags parameters associated with this folder, reference to the default calibration, and folders with calibration versions, distinguished by there production time stamp.
- The number of calibration versions for each time stamp range is unlimited. Default calibration is defined by the reference default version
defaultv.
Metadata in
...
dictpars
Below are the lists of metadata fields which potentially can be used to define detector configuration, calibration parameters etc.
...
| Field name | Description | More details, example |
|---|---|---|
| dettype | detector type | CSPAD, CSPAD2X2, EPIX100A, etc. |
| detname | detector unique name | (if any) ex.: Camera1 |
| detalias | alias name | if it is hard to memorize the entire name, ex.: 'cspad1' |
| detidx | detector index | integer number which codes the hardware version |
| detidxalias | symbolic alias of the index | can be used if it is hard to memorize the index integer number |
| detcompidx:001 | list of component indexes | just in case if we are going to retrieve calibration parameters for separate components |
| detidxprev | detector index for previous version | detector index for previous version (if available) for the purpose of old calibration search |
| detidxnext | detector index for next version | detector index for next version (if available) for the purpose of new calibration search |
| dettsec | time-stamp | time stamp associated with beginning of the validity range for new configuration |
| detcom:001 | comments for this hardware version | as it says {key:comment} |
| dettagdetpar:001 | other tagsparameters | just in case if something is forgotten in this table |
...
| Field name | Description | More details, example |
|---|---|---|
| calibtype | calibration type | ex.: geometry, pixel_status, pixel_gain, pedestals, common_mode, etc. |
| tsec | time stamp | beginning of the validity range |
| exp | original experiment | (if available) where calibration constants were obtained |
| runnum | original run number | (if available) where calibration constants were obtained |
| runbegin | begin run number | (if available) for validity range |
| runend | end run number | (if available) for validity range |
| source | original DAQ data source | data source from DAQ, ex.: 'CxiDs2.0:Cspad.0' |
| srcalias | data source alias | ex.: 'cspad' |
| calibvers | version tagpar | in order to access using symbolic name or some alias |
| calibversalias | version alias | if it is hard to memorize version tagpar |
| com:001 | comments for this version | as it says {key:comment} |
| tagpar:001 | other tagsparameters | just in case if something is forgotten in this table |
...
Is reserved to support common methods of all project classes. For now it stands for manipulations with tags dictpars but not limited to.
| Code Block |
|---|
o = DCBase() # acess methods tagsdictpars = o.tagsdictpars() # returns (dictdictpars) dictionary of tagsdictpars associated with each object tagpar = o.tagpar(k) # returns tagpar value for key k log = o.history(fmt) # returns (str) history records preceded by the time stamp (default fmt='%Y-%m-%dT%H:%M:%S%Z') as a text d = o.histdict() # returns (dict) history dictionary associated with current object # management methods o.set_tagsdictpars(d) # set (dictd) dictionary of tagsparameters for object o.add_tagpar(k,v) # add (k,v) tagpar to the dictionary of tagsparameters for object o.del_tagsdictpars() # delete all tagsparameters from the dictionary o.del_tagpar(k) # delete tagpar with key k o.set_history(d) # set (dict) as a history dictionary of the current object o.add_history(rec, ts) # add (str) record with (float) time stamp to the history dictionary (ts:rec). If ts is None - call current time is used as a key. |
...
Overview
Content Tools