Page History

...

More advanced and recommended way is to provide input data as a special dataset string. The dataset string encodes various parameters, some of which are needed to locate data files, while others specify optional behavior such as filtering or live data reading. The general syntax of the dataset string is:

par[=val][:par[=val][...]

• experiment name (which may optionally contain the name of an instrument)
  

   

  Code Block
  language none
  exp=CXI/cxi12313 exp=cxi12313

• run number specification (can be a single run, a range of runs, a series of runs, or a combination of all above):

   

  Code Block
  language none
  run=1 run=10-20 run=1,2,3,4 run=1,20-20,31,41

   

   

• file type (presently the default type is , if not specified then 'xtc' )is the default

   

  Code Block
  language none
  xtc

  h5

• Location of the files, if not specified then files will be searched in a standard location (/reg/d/psdm/...). If this parameter is specified it needs to be full path name of the directory where files are located

  Code Block
  language none
  dir=/reg/d/ffb/cxi/cxi12345/xtc

• Input number stream number for XTC files, if value is omitted then one pseudo-random stream is selected (this is useful to balance the load on FFB storage system for example):

  Code Block
  language none
  one-stream=1 one-stream

  a random stream (if a value is omitted) or a specific stream. Note this option only makes a sense for XTC files:

   

  one-stream
  one-stream=2

• allow reading from live XTC files while they're still being recorded (by the DAQ or by the Data Migration service). Note that this feature is only available when running psana at PCDS, in all other cases the option will be ignored:

  Code Block
  language

   

  none

  live

Putting all together one would see a data set specification which would tell the framework to read data of stream #2 from XTC files of run 41 while these files were sill being recorded (by the DAQ, or the data migration service, or by another process of the same user, etc.):

Few examples of dataset specification:

• To read XTC data from specific run number:

  Code Block
  language none
  exp=xpp12345:run=123

• To read HDF5 from several runs:

  Code Block
  language none
  exp=xpp12345:run=1,5,7-10:h5

• To read live XTC data from a random stream from FFB directory

  Code Block
  language none
  exp=xpp12345:run=1123:live:one-stream:dir=/reg/d/ffb/xpp/xpp12345/xtc

...

The complete description of the data set string syntax and allowed parameters can be found in the specification document.

...