About

This page provides a list of existing modules for psana framework. Only the modules that are included in the standard analysis releases appear on this page.

...

This page provides examples for selected modules in Psana Module Examples.

Source Code Access

You can find the source code at SLAC (e.g. for module CSPadPixCoords::PixCoordsTest) in these two directories:

/reg/g/psdm/sw/releases/ana-current/CSPadPixCoords/src/PixCoordsTest.cpp (.cpp for c++ source code, or .py for python source code)
/reg/g/psdm/sw/releases/ana-current/CSPadPixCoords/include/PixCoordsTest.h (c++ include file)

Package psana

Psana package include several simple modules which do generic tasks that do not need knowledge of the event data types.

...

Event keys:
  EventKey(type=Psana::EvrData::DataV3, src=DetInfo(NoDetector.0:Evr.0))
  EventKey(type=Psana::Camera::FrameV1, src=DetInfo(CxiDg1.0:Tm6740.0))
  EventKey(type=Psana::Ipimb::DataV1, src=DetInfo(CxiDg1.0:Ipimb.0))
...

This module is useful to display the content of the events without dumping all the data. Example command which uses this module is:have two optional parameters to control output:

This module is useful to display the content of the events without dumping all the data. Example command which uses this module is:

% psana -m EventKeys 

% psana -m EventKeys <input-files>

Note that one can specify EventKeys instead of psana.EventKeys as psana package name is optional.

...

See also Example for Module CSPadPixCoords::CSPad2x2NDArrProducer.

Package ImgPixSpectra

For complete reference see the Doxygen documentation.

Package ImgPixSpectra is intended to accumulate the spectra for all pixels of the image array.
Different modules of this package work with different data types for detectors like CSPad, CSPad2x2, Opal, Princeton camera, etc. All modules have the same interface and the same functionality.
In the loop over events from beginJob to endJob the image pixel amplitudes are accumulated in the 2-d array,
of the shape (<number-of-pixels>, <number-of-spectral-bins>). The first parameter is defined by the image size. The second is passed as an external parameter of the psana configuration file (psana.cfg) along with minimal and maximal amplitudes. At the endJob the spectral array is saved in file with specified name. Auxiliary file with the name extension *.sha is created in order to save the shape parameters. For example, the "cspad2x2-pix-spectra.txt.sha" outpuit file contains

NPIXELS  143560
NBINS    100
AMIN     500
AMAX     1000
NEVENTS  2549
ARRFNAME cspad2x2-pix-spectra.txt

This information can be used in analysis or presentation of this array.

Module ImgPixSpectra::CSPadPixSpectra

List of parameters:

See also: Example for Module ImgPixSpectra::CSPadPixSpectra.

Module ImgPixSpectra::CSPad2x2PixSpectra

The only difference in interface of this module from previous is in the default name for the source parameter and the output file name.

See also: Example for Module ImgPixSpectra::CSPad2x2PixSpectra.

Module ImgPixSpectra::CameraPixSpectra

The only difference in interface of this module from previous is in the default name for the source parameter and the output file name.

or:

See also Example for Package ImgPixSpectra.

Package ImgAlgos

This packages contains a few psana modules for analysis and image processing

Module ImgAlgos::Tahometer

This module measures the time interval for entire job and for each dn events and prints the rate info as requested by the print_bits parameter.

See also Example for Module ImgAlgos::Tahometer.

Module ImgAlgos::TimeStampFilter

This module passes only the events if their time stamp is in the requested range.
The range of allowed time stamps is defined by the configuration parameters.

The time-stamp string is accepted in various formats:

• YYYYMMDD HHMMSS.FFF
• YYYYMMDDTHHMMSS.F
• YYYY-MM-DD HH:MM:SS.FFF
  but the date field has to be presented mandatory.
  If the tsinterval is defined and is different from the default, it will be used in filter and override the tstamp_min and tstamp_max.

See also Example for TimeStampFilter and XtcOutputModule.

Module ImgAlgos::EventNumberFilter

This filter selects events by their number counting from the beginning of job, starting from 0. The event number is not a parameter which is associated with event. Use this filter cautiously on your own risk for debugging purpose only.

...

Module CSPadPixCoords::CSPad2x2NDArrReshape

CSPad2x2NDArrReshape is a module for psana framework. This module reshapes ndarrays from cspad2x2 data format (185,388,2) to cspad-"natural" format (2,185,388).  Input arrays for re-shaping are specified by the source and  keys_in  parameters. Output arrays - by the source and  keys_out  parameters. Currently four data type are supported for input arrays; int16, int, float, and double of const and non-const types. Output arrays have the same data type as input.

Conventions for key lists description

1. keys_in should not be empty to convert something,
2. keys should be space separated, ex.: keys_in = k1 key2 key3 k4,
3. if keys_out is non-empty then the number of keys in keys_out should be equal to the number of keys in keys_in and their names will be used without any modifications,
4. If keys_out is empty, then all keys are defined by the list of key_in
   • if the key in key_in has a suffix specified by the colon ":", this suffix will be dropped for the output key, ex.: key12:xyz -> key12
   • if the key suffix is missing in the input key, then it will be added to the output key as :2x185x388, ex.: key12 -> key12:2x185x388

Configuration parameters:

See also Example for Module CSPadPixCoords::CSPad2x2NDArrReshape.

JIRA Issue:

Package ImgPixSpectra

For complete reference see the Doxygen documentation.

Package ImgPixSpectra is intended to accumulate the spectra for all pixels of the image array.
Different modules of this package work with different data types for detectors like CSPad, CSPad2x2, Opal, Princeton camera, etc. All modules have the same interface and the same functionality.
In the loop over events from beginJob to endJob the image pixel amplitudes are accumulated in the 2-d array,
of the shape (<number-of-pixels>, <number-of-spectral-bins>). The first parameter is defined by the image size. The second is passed as an external parameter of the psana configuration file (psana.cfg) along with minimal and maximal amplitudes. At the endJob the spectral array is saved in file with specified name. Auxiliary file with the name extension *.sha is created in order to save the shape parameters. For example, the "cspad2x2-pix-spectra.txt.sha" outpuit file contains

NPIXELS  143560
NBINS    100
AMIN     500
AMAX     1000
NEVENTS  2549
ARRFNAME cspad2x2-pix-spectra.txt

This information can be used in analysis or presentation of this array.

Module ImgPixSpectra::CSPadPixSpectra

List of parameters:

See also: Example for Module ImgPixSpectra::CSPadPixSpectra.

Module ImgPixSpectra::CSPad2x2PixSpectra

The only difference in interface of this module from previous is in the default name for the source parameter and the output file name.

See also: Example for Module ImgPixSpectra::CSPad2x2PixSpectra.

Module ImgPixSpectra::CameraPixSpectra

The only difference in interface of this module from previous is in the default name for the source parameter and the output file name.

or:

See also Example for Package ImgPixSpectra.

Package ImgAlgos

This packages contains a few psana modules for analysis and image processing.

Module ImgAlgos::Tahometer

This module measures the time interval for entire job and for each dn events and prints the rate info as requested by the print_bits parameter.

There are two modes of operation of this filter.

1. If the first and/or last event numbers are defined, then the filter will select events in this range only.

2. If the evtstring is defined, only listed events of event ranges will be selected. For example, the evtstringparameter can be defined as

   Code Block
   bgColor #F7F7ED
   2,5,11-15,20-25,29,30

   that means the list of events:

   Code Block
   bgColor #F7F7ED
   2 5 11 12 13 14 15 20 21 22 23 24 25 29 30

   In the evtstring parameter the comma "," and sign minus "-" as a dash are the only allowed separators. Blank spaces are also allowed. Other characters may abort the program. The evtstring mode has higher priority than the 1st mode.
   The filterIsOn allows easy turn on/off this filter in *.cfg file.

Module ImgAlgos::EventCounterFilter

ImgAlgos::EventCounterFilter (ImgAlgos > V00-03-46) module is created by request of Thomas Kroll for experiment with mobile rack in SACLA .

Functionality:

Filter select events which numbers are listed in the input file ifname. This module uses local counter started from 1 for the 1-st event of the job and incremented in the event(...) method.  The file ifname contains the list of  integer event numbers in ascending order separated by space or '\n'.

See also Example for Module ImgAlgos::Tahometer.

Module ImgAlgos::TimeStampFilter

This module passes only the events if their time stamp is in the requested range.
The range of allowed time stamps is defined by the configuration parameters.

Input file (ifname)consists of integer numbers in ascending order separated by space or '\n', for example:

2 3 6 7 9 11 15
17 28 32 ...

Module ImgAlgos::AndorImageProducer

Functionality:

• gets Andor data from the event for specified source and key_in parameters,
• puts the ndarray<const double,2> object with camera image in the event using specified key_out parameter.

Module ImgAlgos::PnccdNDArrProducer

Functionality:

• gets pnCCD data from Psana::PNCCD::FramesV1 object from the event for specified source and key_in parameters,
• puts the ndarray<const TOUT,3> object of shape 4x512x512 specified key_out parameter.

The time-stamp string is accepted in various formats:

• YYYYMMDD HHMMSS.FFF
• YYYYMMDDTHHMMSS.F
• YYYY-MM-DD HH:MM:SS.FFF
  but the date field has to be presented mandatory.
  If the tsinterval is defined and is different from the default, it will be used in filter and override the tstamp_min and tstamp_max.

See also Example for TimeStampFilter and XtcOutputModule.

Module ImgAlgos::EventNumberFilter

This filter selects events by their number counting from the beginning of job, starting from 0. The event number is not a parameter which is associated with event. Use this filter cautiously on your own risk for debugging purpose only.

Module ImgAlgos::PnccdImageProducer

Functionality:

• gets from the event store the object with pnCCD data of type
  • Psana::PNCCD::FullFrameV1 containing four [512][512] frames with T=uint16_t, or
  • ndarray<const T,3> , where shape=[4][512][512], T=unsigned short, float, double, int, or int16,
  for specified source and inkey parameters

• puts the ndarray<const T,2> object with pnccd [1024+gap_rows][1024+gap_cols] image in the event using specified outimgkey parameter.

There are two modes of operation of this filter.

1. If the first and/or last event numbers are defined, then the filter will select events in this range only.

2. If the evtstring is defined, only listed events of event ranges will be selected. For example, the evtstringparameter can be defined as

   Code Block
   bgColor #F7F7ED
   2,5,11-15,20-25,29,30

   that means the list of events:

   Code Block
   bgColor #F7F7ED
   2 5 11 12 13 14 15 20 21 22 23 24 25 29 30

   In the evtstring parameter the comma "," and sign minus "-" as a dash are the only allowed separators. Blank spaces are also allowed. Other characters may abort the program. The evtstring mode has higher priority than the 1st mode.
   The filterIsOn allows easy turn on/off this filter in *.cfg file.

Module ImgAlgos::EventCounterFilter

ImgAlgos::EventCounterFilter (ImgAlgos > V00-03-46) module is created by request of Thomas Kroll for experiment with mobile rack in SACLA .

Functionality:

Filter select events which numbers are listed in the input file ifname. This module uses local counter started from 1 for the 1-st event of the job and incremented in the event(...) method.  The file ifname contains the list of  integer event numbers in ascending order separated by space or '\n'.

...

Input file (ifname)consists of integer numbers in ascending order separated by space or '\n', for example:

2 3 6 7 9 11 15
17 28 32 ...

...

Module ImgAlgos::

...

EventCodeFilter

...

ImgAlgos::

...

This module takes care about any generic camera image.
Functionality:

...

EventCodeFilter (ImgAlgos > V00-03-80) module is created by request of Silke Nelson for dark image selection based on EvrData eventCode.

Functionality: Filter selects events which have requested evtcode in Psana::EvrData::DataV4, 3, ... data objects in the list of Psana::EvrData::FIFOEvent objects. Events which do not have the event code are skipped

...

.

See also Example for Module ImgAlgos::CameraImageProducer.

Module ImgAlgos::PrincetonImageProducer

Functionality:

Example of configuration parameters for this module:

modules = ... ImgAlgos.EventCodeFilter ...

[ImgAlgos.EventCodeFilter]
evtcode    = 41
mode       = 1
print_bits = 5

Module ImgAlgos::AndorImageProducer

Functionality:

• gets Andor data gets the Princeton or Pimax camera image data Princeton::FrameV1/2 or Pimax::FrameV1 from the event store for specified source and key_in parameters,
• puts the ndarray<const T,2> object with camera image in the event store using specified type outtype and key_out parameters parameter, where the output type T is defined by the outtype parameter.

...

...

Module ImgAlgos::

...

EpixNDArrProducer

Functionality:

• gets Epix data from the event store for specified

...

Module ImgAlgos::AcqirisArrProducer

• Gets acqiris configuration and data from Acqiris::ConfigV1 and Acqiris::DataDescV1,  objects using parameters source and key_in;
• produces ndarray<const double,2> of shape[] = {nbrChannels, nbrSamples} for waveforms and time stamps;
• saves configuration data in the file defined by fname_prefix;
• parameters,
• puts the ndarray<const T,2> object with image in the event store using specified key_out parameter, where the output type T is defined by the outtype parametersaves waveforms and time stamps in the event store with keys key_wform and key_wtime.

...

See also Example for Module ImgAlgos::

...

EpixNDArrProducer.

...

...

Module ImgAlgos::

...

PnccdNDArrProducer

Functionality:

• gets pnCCD data from Psana::PNCCD::FramesV1 object from the event for specified Gets Acqiris waveforms from event store as ndarray<const double,2> object using parameters source and key_in;performs waveform selection controlled by parameters: thresholds, is_postive_signal, do_inverse_selection, in the range depending on local event numbers skip_events and proc_events;

• after number of events proc_events or at the end of job (whatever happens first), saves array of averaged waveforms in the text file with name constructed from fname_ave_prefix and in the event store using parameters source and key_average.
• ,
• puts the ndarray<const TOUT,3> object of shape 4x512x512 specified key_out parameter.

Module ImgAlgos::PnccdImageProducer

Functionality:

• gets from the event store the object with pnCCD data of type
  • Psana::PNCCD::FullFrameV1 containing four [512][512] frames with T=uint16_t, or
  • ndarray<const T,3> , where shape=[4][512][512], T=unsigned short, float, double, int, or int16,
  for specified source and inkey parameters

• puts the ndarray<const T,2> object with pnccd [1024+gap_rows][1024+gap_cols] image in the event using specified outimgkey parameter.

See also  Example for Module ImgAlgos::AcqirisAverage

...

PnccdImageProducer.

Module ImgAlgos::CameraImageProducer

This module works with any generic camera image stored in data type Camera::FrameV#:

• gets any camera image data Camera::FrameV1 from the event store for specified source and key_in parameters,

• puts the ndarray<const T,2> object with camera image in the event store using specified type outtype and key_out parameters.

   

  Note
  Special treatment for fccd960: if  outtype is not "asdata" the gain factor depending on gain bits is applied, See FCCD-Detector

...

Module ImgAlgos::AcqirisCalib

• Gets Acqiris waveforms from event store as ndarray<const double,2> object using parameters source and key_in;
• processes events in the range depending on local event numbers skip_events and proc_events;
• at the 1st processed event loads the fname_base_line file with baseline  ndarray<const double,2>;
• subtract baseline from waveforms;
• save corrected waveforms in the event store as ndarray<const double,2> object using parameters source and key_out

  .

See also  Example for Module ImgAlgos::AcqirisCalib AnchorAcqirisCFDAcqirisCFDCameraImageProducer.

Module ImgAlgos::PrincetonImageProducer

Functionality:

...

• gets the Princeton or Pimax camera image data Princeton::FrameV1/2 or Pimax::FrameV1 from the event store for specified source and key_in parameters,
• puts the ndarray<const T,2> object with camera image in the event using specified type outtype and key_out parameters.
• Gets Acqiris waveforms from event store as ndarray<const double,2> object using parameters source, key_wform, key_wtime;
• runs constant-fraction discriminator algorithm on all acqiris channels using user-specified per-channel parameters;

• saves edges into the event as a set of ndarray<double,1>;

   

...

...

...

...

"acqiris_wform"

...

Key for input ndarray with waveforms (either raw, or subtracted using AcqirisCalib module) from evt store.

...

key_wtime

...

"acqiris_wtime"

...

Key for input ndarray with waveform times from evt store.

...

"acqiris_edges_"

...

Key for output ndarray<double,1> with calibrated waveforms. This key will have the acqiris channel number (1 thru 20) appended to the end of it, and the data for that channel will be added to the event only if edges were found.

...

""

...

A list of baseline values (one per channel) to subtract from the waveform in volts.

...

""

...

A list of fractions (one per channel) between 0 and 1. The edge-time reported will be at the time when the pulse is at this fraction of the peak value.

...

""

...

A list of threshold values (one per channel) in volts that indicate a new edge should be found. If this value is less than the baseline, then the algorithm will look for negative pulses, otherwise it will look for positive pulses.

...

""

...

A list of deadtimes (one per channel) in seconds. After each edge the algorithm will ignore any new hits in this time interval.

...

""

...

A list of 0/1 values (one per channel) indicating whether edge-times are desired for leading edges (1) or falling edges (0).

See also Example for Module ImgAlgos::PrincetonImageProducer.

Module ImgAlgos::AcqirisArrProducer

• Gets acqiris configuration and data from Acqiris::ConfigV1 and Acqiris::DataDescV1,  objects using parameters source and key_in;
• produces ndarray<const double,2> of shape[] = {nbrChannels, nbrSamples} for waveforms and time stamps;
• saves configuration data in the file defined by fname_prefix;
• saves waveforms and time stamps in the event store with keys key_wform and key_wtime.

 Example for Module ImgAlgos::AcqirisArrProducer

Module ImgAlgos::AcqirisAverage

• Gets Acqiris waveforms from event store as ndarray<const double,2> object using parameters source and key_in;
• performs waveform selection controlled by parameters: thresholds, is_postive_signal, do_inverse_selection, in the range depending on local event numbers skip_events and proc_events;

• after number of events proc_events or at the end of job (whatever happens first), saves array of averaged waveforms in the text file with name constructed from fname_ave_prefix and in the event store using parameters source and key_average.

 Example for Module ImgAlgos::AcqirisCFD

Module ImgAlgos::NDArrAverage

This module averages over events the per-element data of the image array (ndarray<const T,NDim>, where T is implemented for almost all types: int, int16, uint, float, double etc., NDim≤5) and saves files for sum, averaged, rms values, mask, and, the hot pixel map. Input ndarray can be specified by the source and key parameters. Averaging may have up to three stages, depending on configuration parameters:

• 0-stage: the pixel amplitudes are averaged without any constrains for events from 0 to evts_stage1, the preliminary averaged and rms values are defined for each pixel at the end of this stage.
• 1-stage: starting from event evts_stage1 the pixel data are collected only for abs(amplitude-average0) < gate_width1. At the end of this stage the preliminary averaged and rms values are defined for each pixel.
• 2-stage: starting from the event evts_stage1 + evts_stage2 the pixel data are collected only for abs(amplitude-average1) < gate_width2. At the end of this stage the preliminary averaged and rms values are defined for each pixel and saved in the files specified by the avefile and rmsfile parameters, respectively.
  This 3-stage averaging algorithm eliminates large statistical fluctuations in the pixel amplitude spectrum.

If the parameter thr_rms_ADU is set ≤0 then threshold value is defined automatically using constrained averaging of pixel rms values in 3 iterations.  The threshold value is defined as mean+5*rms.

If all file names are empty (by default), the files with pre-defined names "arr-ave-<exp>-r<run>.dat" and "arr-rms-<exp>-r<run>.dat" will be saved for averaged and rms arrays, respectively. Otherwise, the files with specified names will be saved.
Default parameters are set for regular single-stage averaging without any constrains.
See also Example for Module ImgAlgos::NDArrAverage.

Module ImgAlgos::NDArrCalib

Functionality

• NDArrCalib uses the source and key_in parameters to get any ndarray<const T,NDIM> object from the event store, where T stands forint16_t, uint16_t, int, float, uint8_t, or double, 1≤NDim≤5,
• automaticly gets parameters from calibration store for types pedestals, common_mode, pixel_status, pixel_gain, and pixel_rms,
• gets parameters from user-defined files fname_bkgd and  fname_mask, if their names are specified,

• the specified by the do_... parameter corrections are applied to raw data  ndarray<const T,NDIM> as follows:

1. 1. 1. 1. subtracts pedestals,
         2. subtracts common mode,
         3. subtracts normalized background,
         4. apply gain factors,
         5. apply hot/bad pixel_status mask,
         6. apply mask  (1/0 = good/bad pixels),
         7. apply threshold as a common low level,
         8. apply per-pixel threshold as N*RMS,

• and saves the corrected ndarray<const double,NDim> in the event with key key_out
• In ImgAlgos V00-02-01 implemented detectors: CsPad, CsPad2x2, Pnccd, Princeton, Andor, Opal1000, Opal4000

Control on corrections

A_cor = A_raw
        (1) - pedestal             | if do_peds==true and pedestals are available in calib store
        (2) - common mode          | if do_cmod==true and common_mode parameters are available in calib store
        (3) - N*background         | if do_bkgd==true and the file name is specified in the parameter fname_bkgd and bkgd_ind_* are set
        (4) * gain                 | if do_gain==true and pixel_gain are available in calib store
        (5) apply bad pixel status | if do_stat==true and pixel_status are available in calib store
        (6) apply mask             | if do_mask==true and the file name is specified in the parameter fname_mask. Parameter masked_value is used to substitute masked values.
        (7) apply N*RMS threshold  | if do_nrms==true and pixel_rms are available in calib store, parameters threshold_nrms and below_thre_value are set
        (8) apply common threshold | if do_thre==true. Parameter below_thre_value is used to substitute below threshold values.

Configuration parameters

 Example for Module ImgAlgos::AcqirisAverage

Module ImgAlgos::AcqirisCalib

• Gets Acqiris waveforms from event store as ndarray<const double,2> object using parameters source and key_in;
• processes events in the range depending on local event numbers skip_events and proc_events;
• at the 1st processed event loads the fname_base_line file with baseline  ndarray<const double,2>;
• subtract baseline from waveforms;
• save corrected waveforms in the event store as ndarray<const double,2> object using parameters source and key_out.

 Example for Module ImgAlgos::AcqirisCalib

Module ImgAlgos::AcqirisCFD

• Gets Acqiris waveforms from event store as ndarray<const double,2> object using parameters source, key_wform, key_wtime;
• runs constant-fraction discriminator algorithm on all acqiris channels using user-specified per-channel parameters;

• saves edges into the event as a set of ndarray<double,1>;

   

  parameter	default value	description
  source	"DetInfo(:Acqiris)"	Source of data.
  key_wform	"acqiris_wform"	Key for input ndarray with waveforms (either raw, or subtracted using AcqirisCalib module) from evt store.
  key_wtime	"acqiris_wtime"	Key for input ndarray with waveform times from evt store.
  key_edges	"acqiris_edges_"	Key for output ndarray<double,1> with calibrated waveforms. This key will have the acqiris channel number (1 thru 20) appended to the end of it, and the data for that channel will be added to the event only if edges were found.
  baselines	""	A list of baseline values (one per channel) to subtract from the waveform in volts.
  fractions	""	A list of fractions (one per channel) between 0 and 1. The edge-time reported will be at the time when the pulse is at this fraction of the peak value.
  thresholds	""	A list of threshold values (one per channel) in volts that indicate a new edge should be found. If this value is less than the baseline, then the algorithm will look for negative pulses, otherwise it will look for positive pulses.
  deadtimes	""	A list of deadtimes (one per channel) in seconds. After each edge the algorithm will ignore any new hits in this time interval.
  leading_edges	""	A list of 0/1 values (one per channel) indicating whether edge-times are desired for leading edges (1) or falling edges (0).

 Example for Module ImgAlgos::AcqirisCFD

Module ImgAlgos::NDArrImageProducer

This module converts any (detector-associated) ndarray in to image.

For any available in the event store ndarray<const T,NDim>, defined by the source and key_in parameters, and which size is equal to the number of pixels in the detector,  this module creates 2-d image ndarray<const TOUT,2> and saves it in the event store with key_out. TOUT is defined by the type_out. Pixel geometry array is retreived from the calibration file of  "geometry" type from default standard or specified in calibdir directory.

Configuration parameters:

To add implementation for any other new detector "Det":

        1) add file pdscalibdata/DetBaseV1.h

        2) in PSCalib/src/GenericCalibPars.cpp add
             #include "pdscalibdata/DetBaseV1.h"
             ...
             template class PSCalib::GenericCalibPars<pdscalibdata::Opal4000BaseV1>;

        3) in PSCalib/include/CalibParsStore.h add
             #include "pdscalibdata/DetBaseV1.h"
             ...
             return new PSCalib::GenericCalibPars<pdscalibdata::DetBaseV1>(calibdir, type_group, src, runnum, prbits);

Doxygen documentation for interface: CalibParsStore,  GenericCalibPars

 See also Example for Module ImgAlgos::NDArrCalib,

Test of the NDArrCalib module for pnCCD.

Additional parameters which can be used for special applications like alignment:

See also Example for Module ImgAlgos::NDArrImageProducer.

Module ImgAlgos::NDArrAverage

This module averages over events the per-element data of the image array (ndarray<const T,NDim>, where T is implemented for almost all types: int, int16, uint, float, double etc., NDim≤5) and saves files for sum, averaged, rms values, mask, and, the hot pixel map. Input ndarray can be specified by the source and key parameters. Averaging may have up to three stages, depending on configuration parameters:

• 0-stage: the pixel amplitudes are averaged without any constrains for events from 0 to evts_stage1, the preliminary averaged and rms values are defined for each pixel at the end of this stage.
• 1-stage: starting from event evts_stage1 the pixel data are collected only for abs(amplitude-average0) < gate_width1. At the end of this stage the preliminary averaged and rms values are defined for each pixel.
• 2-stage: starting from the event evts_stage1 + evts_stage2 the pixel data are collected only for abs(amplitude-average1) < gate_width2. At the end of this stage the preliminary averaged and rms values are defined for each pixel and saved in the files specified by the avefile and rmsfile parameters, respectively.
  This 3-stage averaging algorithm eliminates large statistical fluctuations in the pixel amplitude spectrum.

If the parameter thr_rms_ADU is set ≤0 then threshold value is defined automatically using constrained averaging of pixel rms values in 3 iterations.  The threshold value is defined as mean+5*rms.

If all file names are empty (by default), the files with pre-defined names "arr-ave-<exp>-r<run>.dat" and "arr-rms-<exp>-r<run>.dat" will be saved for averaged and rms arrays, respectively. Otherwise, the files with specified names will be saved.
Default parameters are set for regular single-stage averaging without any constrains.
See also Example for Module ImgAlgos::NDArrAverage.

Module ImgAlgos::NDArrCalib

Functionality

• NDArrCalib uses the source and key_in parameters to get any ndarray<const T,NDIM> object from the event store, where T stands forint16_t, uint16_t, int, float, uint8_t, or double, 1≤NDim≤5,
• automaticly gets parameters from calibration store for types pedestals, common_mode, pixel_status, pixel_gain, and pixel_rms,
• gets parameters from user-defined files fname_bkgd and  fname_mask, if their names are specified,

• the specified by the do_... parameter corrections are applied to raw data  ndarray<const T,NDIM> as follows:

1. 1. 1. 1. subtracts pedestals,
         2. subtracts common mode,
         3. subtracts normalized background,
         4. apply gain factors,
         5. apply hot/bad pixel_status mask,
         6. apply mask  (1/0 = good/bad pixels),
         7. apply threshold as a common low level,
         8. apply per-pixel threshold as N*RMS,

• and saves the corrected ndarray<const TOUT,NDim> in the event with key key_out, where TOUT is controlled by the parameter outtype, which can be set to double(default), float, int, and int16.

• In ImgAlgos V00-02-01 implemented detectors: CsPad, CsPad2x2, Pnccd, Princeton, Andor, Opal1000, Opal4000

Control on corrections

A_cor = A_raw
        (1) - pedestal             | if do_peds==true and pedestals are available in calib store
        (2) - common mode          | if do_cmod==true and common_mode parameters are available in calib store
        (3) - N*background         | if do_bkgd==true and the file name is specified in the parameter fname_bkgd and bkgd_ind_* are set
        (4) * gain                 | if do_gain==true and pixel_gain are available in calib store
        (5) apply bad pixel status | if do_stat==true and pixel_status are available in calib store
        (6) apply mask             | if do_mask==true and the file name is specified in the parameter fname_mask. Parameter masked_value is used to substitute masked values.
        (7) apply N*RMS threshold  | if do_nrms==true and pixel_rms are available in calib store, parameters threshold_nrms and below_thre_value are set
        (8) apply common threshold | if do_thre==true. Parameter below_thre_value is used to substitute below threshold values.

Configuration parameters

To add implementation for any other new detector "Det":

        1) add file pdscalibdata/DetBaseV1.h

        2) in PSCalib/src/GenericCalibPars.cpp add
             #include "pdscalibdata/DetBaseV1.h"
             ...
             template class PSCalib::GenericCalibPars<pdscalibdata::Opal4000BaseV1>;

        3) in PSCalib/include/CalibParsStore.h add
             #include "pdscalibdata/DetBaseV1.h"
             ...
             return new PSCalib::GenericCalibPars<pdscalibdata::DetBaseV1>(calibdir, type_group, src, runnum, prbits);

Doxygen documentation for interface: CalibParsStore,  GenericCalibPars

 See also Example for Module ImgAlgos::NDArrCalib,

Test of the NDArrCalib module for pnCCD.

Module ImgAlgos::NDArrDropletFinder

Finds "droplets" (wide peaks) in data ndarray and saves their list in output ndarray. This is a re-implementation of algorithm ImgPeakFinder for ndarray of 2-d segments with functionality as follows.
1. Selects pixels in the windows defined by the list of parameters:  segment, rowmin,  rowmax, colmin,  colmax with amplitudes above the threshold_low.
2. Optionally smears image for selected pixels using 2-d matrix of weights in the range -smear_radius to +smear_radius in both dimensions around smeared pixel. The matrix of weights is defined by the 2-d Gaussian function of width sigma. If sigma=0 smearing is not applied.
3. Finds peaks as pixels with absolute-maximal amplitude above the threshold_high in the center of the matrix covering the range -peak_radius to +peak_radius  in both dimensions.
4. Puts the ndarray of found peaks in the event store with key key_droplets. Each row of this ndarray has parameters from struct Droplet, containing segment number,  row, column of the droplet center, peak pixel amplitude, the total amplitude inside peak_radius the region, and the number of pixels in this region above threshold_low:

  struct Droplet{
    unsigned seg;
    double   row;
    double   col; 
    double   ampmax;  // amplitude in the peak maximum
    double   amptot;  // total amplitude in the range of peak_radius
    unsigned npix;    // number of pixels in the range of peak_radius
  };

Remarks:

• if print_bits is not zero - warning messages will be printed.
• Saves table of droplets/peaks as  ndarray<float,2> with shape=[ndroplets,6]  (see struct Droplet) if the keyword  peaks_droplets is non-empty.
• A set of threshold parameters should be different in cases when smearing is applied (sigma is not zero) or not.

See also Example for Module ImgAlgos::NDArrDropletFinder

Module ImgAlgos::PixCoordsProducer

Functionality

For data source  in each run loads/updates calibration geometry file from the calibration DB, evaluates pixel coordinate, area, mask arrays using class PSCalib::GeometryAccess and saves them as ndarray<const TYPE,1> in the env.calibStore() for keys (TYPE=float) key_out_x, key_out_y, key_out_z, key_out_area(unsigned), key_out_area(int), key_out_ix, key_out_iy, key_gfname(string) and (TYPE=uint8_t) key_fname.

The main idea of this module is that calibration geometry file will be found and loaded (if available) automatically and pixel coordinate, index, and other arrays will be produced and saved in the env.calibStore(), if appropriate keys are not empty.


Configuration parameters

Current version of this module works with CSPAD and CSPAD2x2. It can be extended for other detectors, whenever necessary.

See Example for Module ImgAlgos::PixCoordsProducer

Module ImgAlgos::ImgAverage

...

This algorithm was motivated by users of amo42112:
1. Select the pixels in the window xmin, xmax, ymin, ymax
with amplitudes above the threshold_low.
2. Smear Optionally smears image for selected pixels, using 2-d matrix of weights over pixels
from -smear_radius to +smear_radius around each smeared pixel amplitude.
The matrix of weights is defined by the 2-d Gaussian function of width sigma. If sigma=0 smearing is not applied.
3. Find peaks as pixels with absolute-maximal amplitude above the threshold_high in the center of the matrix -peak_radius to +peak_radius.
4. Put the vector or ndarray of found peaks in the event with key peaksKey or peaks_nda respectively. Each entry of this vector has an object of the struct Peak, containing x, y positions, peak pixel amplitude, the total amplitude in the matrix, defined by the peak_radius, and the number of pixels in the matrix above threshold_low:

...

...

• This algorithm consumes ~15 ms/event on psana0101 for full Opal1000 (1024x1024) camera image.
• Smearing algorithm use a "safety margin" which is currently set to 10 pixels (offset from each boarder of the full image size).
• Since V00-03-58 saves table of peaks as  ndarray<float,2> with shape=[npeaks,5] if key peaks_nda is non-empty.

See also Example for Module ImgAlgos::ImgPeakFinder.

...

This module is a part of complex algorithm, described in Command Line Interface For Time Correlation Analysis.

This module is designed for parallel image processing for correlation analysis.
Functionality:

...

...

This package contains python modules which work with both frameworks pyana and psana. Functionality of these modules resembles modules from C++ package ImgAlgos. The difference between two frameworks at code level is explained in Migration from pyana to psana.

Module pyimgalgos.cspad_arr_producer

...

The translator package include the H5Output module which translates xtc to hdf5. For more  information see the page Outdated: The XTC - to - HDF5 Translator

Package psana_test

Anchor
psana_test psana_test

The psana_test package includes the psana module dump, some Python library code for testing, and a command line tool for providing a line oriented dump of xtc files. The psana_test package is primarily for psana developers to do software testing, however the dump module and xtclinedump can be generally useful to users.

module dump

The dump module . This will take a standard psana datasource and dump all the event, config, and epics data found. The entire contents of large arrays are not printed. However a checksum over all the array data is, as well as the min, 25th percentile, median, 75th percentile, and max over the data. The dump module does not serve as a good example of how to retrieve and work with objects from the event store – see the psana_examples package for this. The psana_test package is primarily for psana developers to do software testing.

...

Running psana_test dump

An example of running the module is

...

epics = False       do not print epics
aliases = False     do not print the EPICS alias list
dump_aliases=True   follow EPICS aliases to print the EPICS pv's they point to to print the EPICS pv's they point to
dump_sml=True       dump the small data type (if found, psana should automatically replace these proxies)
regress_dump=True   do not print the DAQ assigned pvId when printing EPICS
dump_beginjob_evt=False   do not dump begin job data
output_file = filename   write output to filename
config = False      do not print the contents of the configStore, only regular event data
counter = False     do not print the counter string that labels event numbers and calib cycle numbers
header = False
indent = 4          change the indent from the default of 2 to 4

...

from psana_test import obj2str, epicsPvToStr

def getPsanaState(event, configStore, epicsStore):
    evtDict = {}
    cfgDict = {}
    epicsDict = {}
    for key in event.keys():
        if key.type() is None: continue
        obj = event.get(key.type(), key.src(), key.key())
        if (obj is None): continue
        if not hasattr(obj,'TypeId'):  continue
        evtDict[str(key)]=obj2str(obj)
    for key in configStore.keys():
        if key.type() is None: continue
        obj = configStore.get(key.type(), key.src())
        if (obj is None): continue
        if not hasattr(obj,'TypeId'):  continue
        cfgDict[str(key)]=obj2str(obj)
    for pvName in epicsStore.pvNames():
        pv = epicsStore.getPV(pvNameobj = configStore.get(key.type(), key.src())
        if not pv(obj is None): continue
        epicsDict[pvName] = epicsPvToStr(pv)
    return evtDict, cfgDict, epicsDict

Package TimeTool

Modules for analyzing recorded data from a timetool camera setup. The timetool camera measures the time difference between laser and FEL in one of two methods:

1. spatial encoding, where the X-rays change the reflectivity of a material and the laser probesthat change by the incident angle of its wavefront; or
2. spectral encoding, where the X-rays change the transmission of a material and the chirped laser probes it by a change in the spectral components of the transmitted laser.

Below the package modules are described. The package includes sample configuration files that describe all the options. From a psana release directory, users are encouraged to add the TimeTool package to obtain the latest source. For instance:

newrel ana-current myrel
cd myrel
kinit             # get ticket to check out package from svn repository
addpkg TimeTool
sit_setup
scons
# now examine the files in TimeTool/data:  sxr_timetool.cfg  sxr_timetool_setup.cfg  timetool_setup.py  xpp_timetool.cfg  xpptut.cfg

 timetool_setup.py is a python script to calculate the digital filter weights.

Module Analyze

a module that analyzes the camera image by projecting a region of interest onto an axis and dividing by a reference projection acquired without the FEL.  The resulting projection is processed by a digital filter which yields a peak at the location of the change in reflectivity/transmission.  The resulting parameters are written into the psana event. The default behavior is to write these values in as a set of double's. Unfortunately double's are presently not visible to Python scripts or modules in the psana framework. In the next analysis release (ana-0.13.1) these values can also written as a set of ndarray's, each with one double. These will be visible on the Python side. This can be done by adding the option

put_ndarrays=True

to the config file. To use the features prior to that release, when adding the TimeTool package to your test release, do

addpkg TimeTool V00-00-03

Module Check

a module that retrieves results from the event for either the above module or from data recorded online.

Module Setup

if not hasattr(obj,'TypeId'):  continue
        cfgDict[str(key)]=obj2str(obj)
    for pvName in epicsStore.pvNames():
        pv = epicsStore.getPV(pvName)
        if not pv: continue
        epicsDict[pvName] = epicsPvToStr(pv)
    return evtDict, cfgDict, epicsDict

xtclinedump

xtclinedump is a command line tool to dump xtc and datagram header information in a line oriented style. By keeping the output for each header to one line, it makes it easy to use grep to filter the output. The command can be run by

xtclinedump dg xtcfile.xtc
or
xtclinedump xtc xtcfile.xtc

The first just dumps datagram headers, the latter dumps xtc headers. There are some additional options, how much of the xtc payloads to print, and if you want parsed output for epics. A help string is available by typing xtclinedump with no arguments. Except for the non-default epics argument, xtclinedump does no parsing of the xtc payloads, it simply prints the first few bytes in hex. For reading through payloads, the intel architechture uses little endian, so 0x00040000 = 1024,a module that calculates the reference autocorrelation function from events without FEL for use in the digital filter construction.

References

• Psana User Manual - Old
• Psana Reference Manual - Old
• psana - Module Examples
• Migration from pyana to psana
• psana - Migration from pyana

dump_sml            dump small data type