Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Content

Table of Contents

Purpose

  • short-term goal - support in a single place calibration data for detectors moving across LCLS experiments.
  • perspective goal long-term goal - calibration store for LCLS-II.

It is assumed that hardware Hardware configuration of modern detectors will detector can be associated with configuration index coming from DAQunique index hardwired on detector controller chip. This index will be used to recognize detector and extract hardly retrievable calibration data such as parameters for geometry, gain factors, masks etc. in case of transition of the detector from one experiment to another.

Requirements

  • Instrument independent - detector can be moved between instruments and hatches. / Experiment independent - detector can be moved between experiments, hatches, and instruments.
  • Portability - calibration data for particular detector should be portable as a self consistent file or (part of) calibration directory.
  • Combination of FS & DB files (hdf5 for uniformity).
  • API + Interfaces - API, Command Line (CL), GUI for central management.Management ; management can ONLY be done through the API/CL/GUI in order to prevent adding/removing unknown files without history.
  • Time stamp (sec) - is THE ONLY value for validity range check.
  • The <detname> is equivalent to <dettype>-<detid> and is THE ONLY choice for the file name. This name is mapped to the custom names in the files with aliases.

Architecture

Calibration directory and files

Default calibration directory in LCLS  FS

instrument and experiment-independent path

  • /reg/g/psdm/detector/calib

in contrast to the current instrument and experiment-dependent path

  • /reg/d/psdm/<INS>/<experiment>/calib

In local space

define calib path

  • <path>/calib/
    assuming further structure of the calibration directory <dettype>/<dettype>-<detid>.h5

or use detector calibration file directly

...

  • ; the same rules are applied to the time stamp like to the run number in current calibration system.
  • File name - the <detname> in the calibration store is <dettype>-<detid>.
  • Aliases to the calibration file names and specific data can be used to simplify access.
  • Versions - support versions of calibration data.
  • Links to predecessor and successor - support access to predecessor and successor if this info is available.

Architecture

To accommodate requirements Detector Calibration Store (DCS) suppose to be implemented as a combination of file system (FS) and data base (DB) files (hdf5 for uniformity). Functionality of the DCS from bottom to top level can be listed as follows.

  • All calibration constants for particular detector id (detid - index of the detector version) will be kept in a single hdf5 file named as <dettype>-<detid>.h5

      Structure of calibration directory

      Everything in lower case:

      Code Block
      <path>/calib/
                   <dettype>/
                   cspad2x2/
                   pnccd/
                   epix100a/
                   cspad/
                             <files>
                             <aliases1>.als           # file with a map of aliases to detector names
                             ...
                             <aliasesN>.als           # one of these files can be used to map "sources" to detector names
                             <dettype>-<detid>.h5
                             <dettype>-<detid>.h5

      Detector type <dettype>  is used as a folder to organize files under the calib directory. In case of direct access to the hdf5 file the detector type is duplicated in the name of the file. Unique part of the detector name <detid> is used to assign calibration file to the particular detector version. If the detector is changing the new calibration file is created with new <detid>. Information about predecessor and successor (if available) can be saved under the root level of the calibration file. Current sources (ex: 'Camp.0:pnCCD.1' ) should be presented in the dictionary of aliases.

      Calibration file structure

      Calibration data hierarchical scheme

      Access to calibration data is based on time stamp. Time stamp internally is presented in Unix time sec which can be easily converted forth and back to human readable format YYYY-MM-DDTHH:MM:SS+HH:MM

      The scheme of the calibration data, which presumably will be implemented in <dettype>-<detid>.h5

        • Schema of this file contains a few levels which account for constant types, time stamp ranges, versions of constants.
        • Beside main functionality each level contains dictionary of parameters and dictionary of history records.
      • These files are grouped in the directory for detector type; ex.: cspad, cspad2x2, pnccd, etc.
        • The same directory contains files with aliases <aliases1>.als which map human readable detector name-and-version with appropriate <dettype>-<detid>.h5 file.
      • All detector type folders in regular case are collected under calib directory, although it is assumed that calibration files can be used directly.
      • The calib directory may be nested in three locations;
        • repository - experiment-independent space with FS back-up, contains most complete calibration files in terms of time stamp and versions,
        • experimental work-space - experiment-dependent space, contains squeezed version of calibration files with time stamp and versions essential for particular experiment,
        • local work-space - experiment-independent any local directory, with squeezed files similar to experimental work-space.

       The repository and experimental works-pace have predictable location in the file system, local work-space may have an arbitrary path. Direct file access pick up calibration file from any place.

      •  Two-levels of interaction with DCS:
        • interaction with calibration data in particular calibration file,
          • add/get/remove constants for type, time stamp, version, etc.
          • add/remove/edit metadata.
        • data exchange between files between repository, experimental and local work-spaces.
          • file difference
          • transfer constants for type, time stamp, version, etc.
          • use web-service mechanism
      • Access Control Lists (ACL) depends on file location and is assumed to be
        • repository - limited to dedicated persons
        • experimental work-spaces - members of experimental group
        • local work-spaces - all


      Calibration directory and files

      Repository

      instrument and experiment-independent path to calibration directory and file

      • <PATH>/detector/calib/
                                                  <dettype>/<dettype>-<detid>.h5

      Experiment calibration directory

      experiment-dependent path to calibration directory and file

      • <PATH>/<INS>/<experiment>/calib/
                                                                     <dettype>/<dettype>-<detid>.h5

      Local calibration directory

      local calib path to calibration directory and file

      • <path-to-calib>/calib/
                            <dettype>/<dettype>-<detid>.h5

      Direct path to calibration file

      direct path to calibration file

      • <path-to-calib>/<dettype>-<detid>.h5

      Structure of calibration directory

      Everything in lower case:

      Code Block
      <path-to-calib>/calib/                          # top calib directory
                   <dettype>/                         # detector type folders
                   cspad2x2/
                   pnccd/
                   epix100a/
                   cspad/
                             <files>                  # level of files
                             <aliases1>.als           # file with a map of aliases to detector names
                             ...
                             <aliasesN>.als           # one of these files can be used to map "sources" to detector names
                             <dettype>-<detid>.h5     # files with detector-dependent calibration data
                             <dettype>-<detid>.h5


      • The calib directory is a top level DCS directory.
      • Detector type <dettype> folders are used to organize files under the calib directory. In case of direct access to the hdf5 file the detector type is duplicated in the name of the file.
      • Unique part of the detector name <detid> is used to assign calibration file to the particular detector hardware configuration version. If the detector hardware configuration is changing a new calibration file is created with new <detid>.
      • Information about predecessor and successor (if available) can be saved under the root level of the calibration file.
      • Current sources (ex: 'Camp.0:pnCCD.1' ) could be presented in the dictionary of aliases.

      Calibration file structure

      Calibration data hierarchical scheme

      Access to calibration data is based on time stamp. Time stamp internally is presented in Unix time sec which can be easily converted forth and back to human readable format YYYY-MM-DDTHH:MM:SS+HH:MM

      The schema (structure) of the DCS files <dettype>-<detid>.h5

      Code Block
      DCStore  DCType                   DCRange         DCVersion    
      
      /                                                                   # top-root-level contains info for the unique detector version
       dettype (str)                                                      # detector type name, ex: cspad, pnccd, etc
       detid (str)                                                        # unique detector id, ex: 01234
       tscfile (float)                                                    # time stamp of creation of this structure
       predecessor (str)
      Code Block
      /                                                                   # top-root-level contains info for the unique detector version
       dettype name of the previous detector if available or None
       successor (str)                                                    # name #of detectorthe typenext name,detector ex:if cspad,available pnccd,or etcNone
       detiddictpars (strdict)                                                        # unique detector id, ex: 01234
       tscfile (uint)   # dictionary of parameters associated with this version of detector
       history (dict)                                       # time stamp of creation of this structure
       predecessor (str)     # dictionary of history records paired as (tstamp:record)
       <ctype>/                                      # name of the previous detector if available or None
       successor (str)          # folder for calibration type, ex.: pedestals, rms, mask, etc
                ctype (str)                      # name of the next detector if available or None
       tags (dict)              # calibration type name
                dictpars (dict)                            # dictionary of tags associated with this version of detector
       <ctype>/     # dictionary of parameters associated with calibration type
                history (dict)                                    # folder for calibration type
          # dictionary of history records paired ctypeas (strtstamp:record)
                <tstamp-range>/                                     # calibration type name
         # folder for time stamp validity range
         tags  (dict)     <tstamp>[-<tstamp-end>]/                                  # folder for validity range. If <tstamp-end> is #not dictionaryspecified of- tagsthen associatedvalid withto calibrationthe typeend
                <tstamp-range>/                        tsbegin (float)                   # folder for time stamp validityfor range
      the beginning of the validity range
           <tstamp>[-<tstamp-end>]/                             tsend (float)    # folder for validity range. If <tstamp-end> is not specified - then valid to the end
         # time stamp for the end of the validity range
                            tsbegin (uint)           defaultv         # time stamp for the beginning of the validity range
              # reference to the default calibration, ex:  <vers-tstamp2>
                        tsend (uint)               dictpars (dict)      # time stamp for the end of the validity range
          # dictionary of parameters associated with validity range
                             defaultv           history (dict)              # reference to the default calibration, ex:# dictionary <vers-tstamp2>
      of history records paired as (tstamp:record)
                                   tags (dict)    <vers-tstamp1>/                   # dictionaryfolder for ofversion tagscreated associatedon withtstamp1
       validity range
                                        <vers-tstamp1>/             tsvers (float)     # tstamp1 folderof forthis version createdproduction on tstamp1
                                                       tsverscalib (uintndarray)    # calibration #data
       tstamp1 of this version production 
                                                 history (dict)     calib# (ndarray)dictionary of history records #paired calibration dataas (tstamp:record)
                                                       dictpars tags (dict)        # dictionary of tagsparameters, 
                                                                          #   ex: array size, number of dimensions, shape, data type, experiment, run, comments, author
      
                                        <vers <vers-tstamp2>/                   # folder for version created on tstamp2
                                                       tsvers (float)
                               (uint                        calib (ndarray)
                                                       calib history (ndarraydict)
                                                       tagsdictpars (dict)
                 <tstamp2>[-<tstamp2-end>]/                               # folder for the next validity range.
       pedestals/                                                         # folder for the next calibration type, pedestals
       rms/                                                               # folder for the next calibration type, rms
      

      ...

      Schema features

      • Detector name consists of a common part <dettype> part <dettype> and unique part <detid>.
      • Alias to the detector name should be kept in separate dictionary outside the file scheme.
      • Each detector may have optional predecessor, successor, and other tags parameters kept in the dictionary of parameters.
      • Calibration type folders contain info about calibrations of particular type, ex: pedestals, rms, status, mask, background, etc.
      • Each calibration type contains a set of time stamp ranges defining calibration validity range. If the second time stamp of the range is missing it is considered as infinity (by the end).
      • The time stamp range folder contains tstamp_begin, tstamp_end (int) values, dictionary of tags parameters associated with this folder, reference to the default calibration, and folders with calibration versions, distinguished by there production time stamp.
      • The number of calibration versions for each time stamp range is not limitedunlimited. Default calibration is defined by the reference default _ version defaultv.

      ...

      Metadata in

      ...

      dictpars

      Below are the lists of metadata fields which potentially can be used to define detector and access calibrationsconfiguration, calibration parameters etc.

      Detector metadata

      Field nameDescriptionMore details, example
      dettypedetector typeCSPAD, CSPAD2X2, EPIX100A, etc.
      detnamedetector unique name(if any) ex.: Camera1
      detaliasalias nameif it is hard to memorize the entire name, ex.: 'cspad1'
      detidxdetector indexinteger number which codes the hardware version
      detidxaliassymbolic alias of the indexcan be used if it is hard to memorize the index integer number
      detcompidx:001list of component indexesjust in case if we are going to retrieve calibration parameters for separate components
      detidxprevdetector index for previous versiondetector index for previous version (if available) for the purpose of old calibration search
      detidxnextdetector index for next versiondetector index for next version (if available) for the purpose of new calibration search
      dettsectime-stamptime stamp associated with beginning of the validity range for new configuration
      detcom:001comments for this hardware versionas it says {key:comment}
      dettag
      detpar:001other
      tags
      parametersjust in case if something is forgotten in this table

      Calibration parameters' metadata

      Field nameDescriptionMore details, example
      calibtypecalibration typeex.: geometry, pixel_status, pixel_gain, pedestals, common_mode, etc.
      tsectime stampbeginning of the validity range
      exporiginal experiment(if available) where calibration constants were obtained
      runnumoriginal run number(if available) where calibration constants were obtained
      runbeginbegin run number(if available) for validity range
      runendend run number(if available) for validity range
      sourceoriginal DAQ data sourcedata source from DAQ, ex.: 'CxiDs2.0:Cspad.0'
      srcaliasdata source aliasex.: 'cspad'
      calibversversion
      tag
      parin order to access using symbolic name or some alias
      calibversaliasversion aliasif it is hard to memorize version
      tag
      par
      com:001comments for this versionas it says {key:comment}
      tag
      par:001other
      tags
      parametersjust in case if something is forgotten in this table

      In case of numpy array their metadata are stored with an object.
      Text file needs in n-d array metadata

      n-d array metadata

      Field nameDescriptionMore details, example
      dtype
      DTYPE(str) data typeint, float, double, etc.
      ndims
      NDIMS(int) number of dimensions (N)
      as it saysdim
      ex.: 3
      DIM:1(int) size of dim.1ex.
      ..
      : 185
      dim
      DIM:2
      (int) size of dim.2ex.: 388
      .........
      DIM:N(int) size of dim.Nex.
      .dim:Nsize of dim.N...

      API

      Parameters

      dettype (str) - detector type: cspad, cspad2x2, pnccd, fccd, opal, epix100a, etc.

      ctype (str) - calibration type: pedestals, status, rms, mask, gain, bkgd, common_mode, geometry

      detid (str) - detector unique id number, ex: 123456

      detname (str) - detector unique name, combination of <dettype>-<detid>

      detalias (str) - detector alias name assigned to the detname

      cpath (str) - path to the calibration directory (ex: cpath='<path>/calib') or direct hdf5 file name (ex: cpath='<path>/<detname>-<detid>.h5'). Default calibration directory CPATH_DEF='/reg/g/psdm/detector/calib'.

      version (int) - time stamp of the version creation

       

      Initialization

      : 2

      API

      Parameters

      dettype (str) - detector type: cspad, cspad2x2, pnccd, fccd, opal, epix100a, etc.

      ctype (str) - calibration type: pedestals, status, rms, mask, gain, bkgd, common_mode, geometry

      detid (str) - detector unique id number, ex: 123456

      detname (str) - detector unique name, combination of <dettype>-<detid>

      detalias (str) - detector alias name assigned to the detname

      cpath (str) - path to the calibration directory (ex: cpath='<path>/calib') or direct hdf5 file name (ex: cpath='<path>/<detname>-<detid>.h5'). Default calibration directory CPATH_DEF='/reg/g/psdm/detector/calib'.

      version (int) - time stamp of the version creation

      versind (int) - consecutive version index assigned/mapped to the version production time stamp.


      Initialization

      Code Block
      # Import
      from PSCalib.DCStore import DCStore       # inport DCStore (Detector Calibration Store) object 
      
      # Initialization
      REPO = '<PATH>/detector/calib'            # default calib repository
      
      cdir = env.calibDir()                     # '<PATH>/<INS>/<experiment>/calib' - accept current directory
      path = cdir + <dettype>
      
      fname = '[path/]pnccd-12345678'           # standard name includes detector type, dash, and n-digit id number
      fname = '[path/]Camera1'                  # alias
      
      cs = DCStore(fname)    
      Code Block
      # Import
      from PSCalib.DCStore import DCStore       # inport DCStore (Detector Calibration Store) object 
      
      # Initialization
      CPATH_DEF = '/reg/g/psdm/detector/calib'  # creates defaulta calibDCStore directory
      cpath = '<path>/calib'    object.
      """ get calibration store object
          Input parameters:
          fname [str] - file name/alias of the # local calib directory
      cpath = '<path>/<dettype>-<detid>.h5'     # direct to file
      cpath = env.calibDir()                    # '/reg/d/psdm/<INS>/<experiment>/calib' - accept current directory
      
      detname = 'pnccd-12345678'              detector 
      """
      • If path is missing - use repository.
      • If calibration file is not found - throw raise IOError('File %s is not available' % fname)

      UI acess methods

      Code Block
      ctype = pedestals # status, rms, mask, gain, bkgd, common_mode, geometry, etc
      tsp = tstamp parameter to identify constants, which can be retrieved from evt.run() - run number, evt
      vers = None # for default or versind or version time stamp.
      
      # generic access method:
      obj = cs.get(ctype, tsp, vers=None)
      

      Base class DCBase

      Is reserved to support common methods of all project classes. For now it stands for manipulations with dictpars but not limited to.

      Code Block
      o = DCBase()
      
      # acess methods
      dictpars = o.dictpars()            # returns (dictpars) dictionary of dictpars associated with each object
      par  = o.par(k)  # standard name includes detector type, dash, and n-digit id number
      detname = 'Camera1'                       # alias
      
      cs = DCStore(cpath, detname)  # returns par value for key k
      log  = o.history(fmt)   # creates a DCStore object.
      """ get calibration store object
         # Input parameters:
          cpath [str] - path to the hdf5 file or calibration directory. 
         returns (str) history records preceded by the time stamp (default fmt='%Y-%m-%dT%H:%M:%S%Z') as a text 
      d    = o.histdict()               If cpath# isreturns a(dict) pathhistory todictionary hdf5associated filewith - next parameter is ignored.
      current object
      
      # management methods
      o.set_dictpars(d)                  # If cpath=None - default path is used.
      set (d) dictionary of parameters for object
      o.add_par(k,v)                  If cpath is specified# as a pathadd (k,v) par to directorythe (ordictionary default)of thenparameters detname can be an alias.  
      for object
      o.del_dictpars()      detname [str] - name/alias of the detector 
      """

      UI - acess methods

      Code Block
      p = tstamp parameter to identify constants, which# candelete beall retrievedparameters from evt.run() - run number, evt
      
      nda = cs.get(ctype, p) # where ctype stands for pedestals, status, rms, geometry, etc.
      
      # or specific access methods:
      nda = cs.pedestals(p)
      nda = cs.pixel_status(p)
      nda = cs.pixel_rms(p)
      nda = cs.pixel_mask(p)
      nda = cs.pixel_gain(p)
      nda = cs.pixel_bkgd(p)
      nda = cs.common_mode(p)
      
      geo = cs.geometry(p)
      

       

      Base class

      Code Block
      DCBase - base class for this project
      ====================================
      tags = obj.tags() the dictionary
      o.del_par(k)                       # delete par with key k
      o.set_history(d)                   # set (dict) as a history dictionary of the current object
      o.add_history(rec, ts)             # add (str) record #with (dictfloat) dictionarytime ofstamp tagsto associatedthe withhistory each object
      tag  = obj.tag(k)                  # returns tag value for key k
      
      obj.set_tags(d)                    # set (dict) dictionary of tags for object
      obj.add_tag(k,v)dictionary (ts:rec). If ts is None - call current time is used as a key. 

      Class DCStore

      Code Block
      cs = DCStore(cpath, detname)       # (str) path to calib directory or file, (str) detector name
      
      # acess methods
      nda   = cs.get(ctype, tsp, vers)   # (str) ctype - calibration type
                                # add (k,v) tag to the dictionary of tags for object
      obj.del_tags()        # (...) tsp - parameter to get time stamp (evt, runnum, ts_sec) 
                   # delete all tags from the dictionary
      obj.del_tag(k)                # (int) vers - version #of deletecalibration, tagNone with- key k 

      Access methods

      Code Block
      DCStore methods
      ===============
      use default 
      tscfile     = cs.tscfile()               # (int) time stamp of the file creation
      dettype     = cs.dettype()               # (str) detector type
      detid       = cs.detid()                 # (str) detector id
      detname     = cs.detname()               # (str) detector name of self object
      predecessor = cs.predecessor()           # (str) detname of predecessor or None
      successor   = cs.successor()             # (str) detname of successor or None
      ctypes      = cs.ctypes()      None
      successor   = cs.successor()       # (liststr) calibrationdetname typesof insuccessor theor file
      cto   None
      ctypes      = cs.ctypeobjctypes(ctype)          # (DCTypelist) ~ h5py.Group) calibration type object
      _________________________________________
      ndacalibration types in the file
      cto         = cs.getctypeobj(ctype, tsp, vers)   # (str) ctype - calibration type
            )   # (DCType ~ h5py.Group) calibration type object
      
      # management methods
      cs.set_tscfile(ts)                 # set (int) time stamp of the file creation  
      cs.set_dettype(dettype)            # set (str) detector type
      cs.set_detid(detid)                # set (str) # (...) tsp - parameter to get time stamp (evt, runnum, ts_sec) 
      _________________________________________# (int) vers - version of calibration, None - use default 
      
      
      DCType methods
      ==============
      ctypedetector id
      cs.set_detname(detname)            # set (str) detector name of self object
      cs.set_predecessor(pred)           # =set cto.ctype(str) detname of predecessor or None
      cs.set_successor(succ)             # set (str) detname of ctype name
      tsranges    = cto.ranges()successor or None
      cs.add_ctype(ctype)                # add (liststr) ofcalibration type timeto rangesthe forDCStore ctype
      tsro        = cto.rangeobj(tsrangeobject
      cs.del_ctype(ctype)      #   (DCRange ~ h5py.Group) time stamp validity range object
      
      
      DCRange methods
      ===============
      tsbegin     = tsro.begin(# delete ctype (str) from the DCStore object
      cs.save(path)                      # (int) time stamp beginning validity range
      tsend       = tsro.end( save current calibration in the file specified by path, if path is Null - update current file.
      

      Class DCType

      Code Block
      cto = DCType(dettype)                 # (intstr) time stamp ending validity range
      versionsdetector type
      
      # acess methods
      ctype      = tsrocto.versionsctype()            # (liststr) of uint) versions of calibrations
      versodef ctype name
      tsranges   = tsrocto.versdefranges()             # (DCVersion ~ h5py.Grouplist) referenceof totime theranges default version in the time-range object
      versofor ctype
      tsro       = tsrocto.versobjrangeobj(verstsrange)         # (DCVersionDCRange ~ h5py.Group) specifiedtime versionstamp in thevalidity time-range object
      
      
      DCVersion methods
      =================
      tsvers# management methods
      cto.add_range(tsr)      = verso.tsprod()          #   #add (intstr) of time stampranges of the version production
      calibdatafor ctype
      cto.del_range(tsr)     = verso.calib()           # delete  # (np.array) calibration array
      

      ...

      range from the DCType object

      Class DCRange

      Code Block
      DCStoretsro methods
      ===============
      cs.set_tscfile(ts= DCRange(tsrange)            # (str) time stamp validity #range
      tsro set= DCRange(inttsbegin, tsend) time stamp of the file creation  
      cs.set_dettype(dettype)        # (int,int) time stamp validity range
      
      # acess methods
      tsbegin    # set= tsro.begin(str)  detector type
      cs.set_detid(detid)      # (int) time stamp beginning validity range
      tsend    # set (str) detector= id
      cs.set_detname(detnametsro.end()            # set (strint) detectortime namestamp ofending selfvalidity object
      cs.set_predecessor(pred)range
      versions    = tsro.versions()      # set (str) detnamelist of float) predecessorversions orof None
      cs.set_successor(succ)calibrations
      versodef    = tsro.versdef()       # #(DCVersion set (str~ h5py.Group) detnamereference ofto successorthe or None
      cs.add_ctype(ctype)    default version in the time-range object
      verso       = tsro.versobj(vers)   # #(DCVersion add (str~ h5py.Group) calibrationspecified typeversion toin the DCStoretime-range object
      cs.del_ctype(ctype
      
      # management methods
      tsro.set_begin(tsbegin)                # deleteset ctype (strint) fromtime thestamp DCStore object
      
      
      DCType methods
      ==============
      cto.add_range(tsr) beginning validity range
      tsro.set_end(tsend)                # addset (strint) oftime timestamp rangesending forvalidity ctyperange
      ctotsro.deladd_rangeversion(tsrvers)                 # deleteset range(DCVersion from the DCType object
      
      
      DCRange methods
      ===============~ h5py.Group) versions of calibrations
      tsro.set_beginversdef(tsbeginvers)             # set (int)DCVersion time stamp beginning validity range~ h5py.Group) versions of calibrations
      tsro.setdel_endversion(tsendvers)                # set (int) time stamp ending validity range
      tsro.add_versiondelete version
      

      Class DCVersion

      Code Block
      verso = DCVersion(vers)             # set (DCVersionstr) ~ h5py.Group) versions of calibrations
      tsro.set_versdef(vers)version name
      
      # acess methods
      tsvers      = verso.tsprod()       # set (DCVersion ~ h5py.Group) versions of calibrations
      tsro.del_version(vers)    (int) time stamp of the version production
      calibdata   = verso.calib()        # # delete version
      
      
      DCVersion methods
      =================
      (np.array) calibration array
      
      # management methods
      verso.set_tsprod(tsprod)           # set (int) time stamp of the version production
      verso.add_calib(nda)               # set (np.array) calibration array

       

       

      TBD

      Open questions

      • What is included in the detector Id version?  FEE version, controller version, etc?
      • when constants are updated (file open to write) they are not available... Lock to resolve.

      Data flow

      • who produces and supply constants
      • who is allowed to submit constants
      • who is allowed to access constants
      • ACL inside API or using OS
      • ACL for all or particular detector/type/ etc.

      References

       set (np.array) calibration array

      TBD

      Open questions

      • logic for default version
        • always last added constants
        • set specified version
        • what to do with default if new version is added?


      • when constants are updated (file open to write) they are not available... Lock to resolve.

      Data flow

      • who produces and supply constants
      • who is allowed to submit constants
      • who is allowed to access constants
      • ACL inside API or using OS
      • ACL for all or particular detector/type/ etc.

      2016-10-27 Mtg with cpo

      • table of aliases with records: <alias> <source> <begin> <end>
      • repository
      • <PATH>/detector/calib  "g" or "d"
      • epix100a full name  - 6 numbers of hardware versions, firmware version is a 7th number
      • time double to two integers or int64 (32bit-sec, 32-bit-nsec)
      • geometry needs to be saved as a text
      • repository file has geometry and pixel_gain - like constants
      • do not search in <experiment>/calib
      • access through network for AMI (cpo: TCP with hardwired IP addresses, m:web service/reddis)

      References

      ...