You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 131 Next »

Unable to render {include} The included page could not be found.
Unable to render {include} The included page could not be found.

About

This page provides a list of existing modules for psana framework. Only the modules that are included in the standard analysis releases appear on this page.

Note that most modules have configuration parameters which are documented in tables. For example:

parameter

default value

description

separator

"="

separator string

repeat

80

number of repeats of separator string on
a separator line

 

This page provides examples for selected modules in Psana Module Examples.

Package psana

Psana package include several simple modules which do generic tasks that do not need knowledge of the event data types.

Module psana.EventKeys

This module dumps the list of the event keys in the event or configuration store. Event key is a triplet of data type, data source address, and string key.

Example of the output produced by this module:

Event keys:
  EventKey(type=Psana::EvrData::DataV3, src=DetInfo(NoDetector.0:Evr.0))
  EventKey(type=Psana::Camera::FrameV1, src=DetInfo(CxiDg1.0:Tm6740.0))
  EventKey(type=Psana::Ipimb::DataV1, src=DetInfo(CxiDg1.0:Ipimb.0))
...

This module is useful to display the content of the events without dumping all the data. Example command which uses this module is:

% psana -m EventKeys <input-files>

Note that one can specify EventKeys instead of psana.EventKeys as psana package name is optional.

Module psana.PrintEventId

This module prints the content of the Event ID object on every event.

Example of the output produced by this module:

[info:psana.PrintEventId] event ID: XtcEventId(run=100, time=2010-12-12 11:09:36.300506429-08)
[info:psana.PrintEventId] event ID: XtcEventId(run=100, time=2010-12-12 11:09:36.317163082-08)

Module psana.PrintSeparator

This module prints separator line on every event. Can be used to indicate event boundaries in job's log.

parameter

default value

description

separator

"="

separator string

repeat

80

number of repeats of separator string on
a separator line

Example of the output produced by this module with the default parameters (including output from PrintEventId module):

[info:/root/] ================================================================================
[info:psana.PrintEventId] event ID: XtcEventId(run=100, time=2010-12-12 11:09:36.300506429-08)
[info:/root/] ================================================================================
[info:psana.PrintEventId] event ID: XtcEventId(run=100, time=2010-12-12 11:09:36.317163082-08)

Package psana_examples

This package contains modules that are meant to be used as examples of accessing different data types or framework services. Can be used by developers as templates for new modules. To find the code:

Module psana_examples.DumpAcqiris

Extracts and dumps the content of Acqiris configuration (Psana::Acqiris::ConfigV1) and event data (Psana::Acqiris::DataDescV1) objects.

parameter

default value

description

source

"DetInfo(:Acqiris)"

data source address

Module psana_examples.DumpAcqTdc

Extracts and dumps the content of Acqiris TDC configuration (Psana::Acqiris::TdcConfigV1) and event data (Psana::Acqiris::TdcDataV1) objects.

parameter

default value

description

source

"DetInfo(:AcqTDC.0)"

data source address

Module psana_examples.DumpBld

Extracts and dumps the content of beamline data objects (Psana::Bld::BldDataEBeamV0, Psana::Bld::BldDataEBeam, Psana::Bld::BldDataPhaseCavity, Psana::Bld::BldDataFEEGasDetEnergy, and Psana::Bld::BldDataIpimb).

parameter

default value

description

eBeamSource

"BldInfo(EBeam)"

data source address for Psana::Bld::BldDataEBeam

phaseCavSource

"BldInfo(PhaseCavity)"

data source address for Psana::Bld::BldDataPhaseCavity

feeSource

"BldInfo(FEEGasDetEnergy)"

data source address for Psana::Bld::BldDataFEEGasDetEnergy

ipimbSource

"BldInfo(NH2-SB1-IPM-01)"

data source address for Psana::Bld::BldDataIpimb

Module psana_examples.DumpCamera

Extracts and dumps the content of camera configuration (Psana::Camera::FrameFexConfigV1) and event data (Psana::Camera::FrameV1 and Psana::Camera::TwoDGaussianV1) objects.

parameter

default value

description

source

"DetInfo(:Opal1000)"

data source address

Module psana_examples.DumpControl

Extracts and dumps the content of scan control configuration (Psana::ControlData::ConfigV1) object.

parameter

default value

description

source

"ProcInfo()"

data source address

Module psana_examples.DumpCsPad

Extracts and dumps the content of CsPad configuration (Psana::CsPad::ConfigV*) and event data (Psana::CsPad::DataV*) objects.

parameter

default value

description

source

"DetInfo(:Cspad)"

data source address

Module psana_examples.DumpEncoder

Extracts and dumps the content of encoder configuration (Psana::Encoder::ConfigV1) and event data (Psana::Encoder::DataV*) objects.

parameter

default value

description

source

"DetInfo(:Encoder)"

data source address

Module psana_examples.DumpEpics

Extracts and dumps the list of EPICS PVs and their values and status information on every event.

Module psana_examples.DumpEvr

Extracts and dumps the content of Evr configuration (Psana::EvrData::ConfigV* and Psana::EvrData::IOConfigV1) and event data (Psana::EvrData::DataV*) objects.

parameter

default value

description

source

"DetInfo(:Evr)"

data source address

Module psana_examples.DumpFccd

Extracts and dumps the content of FCCD configuration (Psana::FCCD::FccdConfigV*) objects. To dump event data objects use psana_examples.DumpCamera module.

parameter

default value

description

source

"DetInfo(:Fccd)"

data source address

Module psana_examples.DumpIpimb

Extracts and dumps the content of IPIMB configuration (Psana::Ipimb::ConfigV1) and event data (Psana::Ipimb::DataV1) objects.

parameter

default value

description

source

"DetInfo(:Ipimb)"

data source address

Module psana_examples.DumpLusi

Extracts and dumps the content of LUSI configuration (Psana::Lusi::DiodeFexConfigV1, Psana::Lusi::IpmFexConfigV1, and Psana::Lusi::PimImageConfigV1) and event data (Psana::Lusi::DiodeFexV1 and Psana::Lusi::IpmFexV1) objects.

parameter

default value

description

tmSource

"DetInfo(:Tm6740)"

data source address for Psana::Lusi::PimImageConfigV1

ipimbSource

"DetInfo(:Ipimb)"

data source address for other data

Module psana_examples.DumpOpal1k

Extracts and dumps the content of Opal1000 configuration (Psana::Opal1k::ConfigV1) object. To dump event data objects use psana_examples.DumpCamera module.

parameter

default value

description

source

"DetInfo(:Opal1000)"

data source address

Module psana_examples.DumpPnccd

Extracts and dumps the content of pnCCD configuration (Psana::PNCCD::ConfigV*) and event data (Psana::PNCCD::FrameV1) objects.

parameter

default value

description

source

"DetInfo(:pnCCD)"

data source address

Module psana_examples.DumpPrinceton

Extracts and dumps the content of Princeton configuration (Psana::Princeton::ConfigV1) and event data (Psana::Princeton::FrameV1) objects.

parameter

default value

description

source

"DetInfo(:Princeton)"

data source address

Module psana_examples.DumpPulnix

Extracts and dumps the content of Pulnix configuration (Psana::Pulnix::TM6740ConfigV*) objects.

parameter

default value

description

source

"DetInfo(:Tm6740)"

data source address

Module psana_examples.EBeamHist

This module is an example of histogramming service usage. It extracts beam line data and fills couple of histograms with the beam parameters.

parameter

default value

description

eBeamSource

"BldInfo(EBeam)"

data source address

Package cspad_mod

Package cspad_mod contains common modules useful for analysis of data from CsPad detector.

Module cspad_mod.CsPadPedestals

This module is supposed to run on dark cspad2x2 frame data. It calculates average and standard deviation values for each pixel and writes these values to output files in text format.

parameter

default value

description

source

"DetInfo(:Cspad)"

name of the data source; default value is adequate if there is only one CsPad device in setup, if there is more than one device then more specific value should be provided

output

"cspad-pedestals.dat"

name of the output file for average values, if empty file will not be written

noise

"cspad-noise.dat"

name of the output file for standard deviation values, if empty file will not be written

To use this module with default parameters one could use this command:

psana -m cspad_mod.CsPadPedestals input-files.xtc

which will produce files cspad-pedestals.dat and cspad-noise.dat in the current directory (if input file contains CsPad data).

To change module parameters one could create file with name psana.cfg in the current directory with a contents like this:

[psana]
modules = cspad_mod.CsPadPedestals

[cspad_mod.CsPadPedestals]
source = DetInfo(CxiDs1.0:Cspad.0)
output = pedestals.dat
noise =

and then run psana:

psana input-files.xtc

This should produce file pedestals.dat in the current directory. As the source address is more specific this configuration file can be used even if input data contain more than one CsPad device.

Module cspad_mod.CsPad2x2Pedestals

This module is supposed to run on cspad2x2 dark frame data. It calculates average and standard deviation values for each pixel and writes these values to output files in text format.

parameter

default value

description

source

"DetInfo(:Cspad2x2)"

name of the data source; default value is adequate if there is only one CsPad2x2 device in setup, if there is more than one device then more specific value should be provided

output

"cspad2x2-pedestals.dat"

name of the output file for average values, if empty file will not be written

noise

"cspad2x2-noise.dat"

name of the output file for standard deviation values, if empty file will not be written

To use this module with default parameters one could use this command:

psana -m cspad_mod.CsPad2x2Pedestals input-files.xtc

which will produce files cspad2x2-pedestals.dat and cspad2x2-noise.dat in the current directory (if input file contains CsPad2x2 data).

To change module parameters one could create file with name psana.cfg in the current directory with a contents like this:

[psana]
modules = cspad_mod.CsPad2x2Pedestals

[cspad_mod.CsPad2x2Pedestals]
source = DetInfo(CxiSc1.0:Cspad2x2.0)
output = pedestals.dat
noise =

and then run psana:

psana input-files.xtc

This should produce file pedestals.dat in the current directory. As the source address is more specific this configuration file can be used even if input data contain more than one CsPad2x2 device.

Module cspad_mod.CsPadCalib

This module performs standard CsPad calibration procedures: per-pixel pedestal subtraction and common mode correction. The algorithms used in calibrations are identical to translator algorithms, the result of calibration should be the same as the data stored in HDF5 files.

The module uses three input calibration files which should be located in the standard calibration directory of the corresponding experiment:

  • pedestals – per-pixel pedestal data (produced by the CsPadPedestals module or standalone applications)
  • pixel_status – hot/dead status for every pixel, used by common mode algorithm
  • common_mode – file with parameters controlling common mode calculation
  • pixel_gain – per-pixel gain data, inverse pixel gain (1/gain) as floating number

If any of the files is missing then corresponding algorithm is not executed, if pixel_status file is missing then all pixels are assumed to have working status. Algorithms inside module are executed following this order:

  • common mode is calculated for every segment; common mode uses pedestal and pixel status values
  • if common mode is calculated successfully it is subtracted from every pixel value in a segment
  • if pedestals are defined pedestal values are subtracted from every pixel value
  • if pixel gains are defined then each pixel value is multiplied by corresponding gain factor
  • resulting pixel value is rounded to nearest integer and stored in the output image

The calibration is performed on every CsPad image (full or 2x2) found in the event and works even if there is more than one CsPad data object. The original objects are preserved in event and result of the calibration is stored in event with different string key ("calibrated" by default, can be changed with module parameter).

parameter

default value

description

inputKey

""

string key used to locate uncalibrated data objects in event

outputKey

"calibrated"

string key used to store calibrated data objects in event

doPedestals

"yes"

can be set to "no" to explicitly disable pedestal subtraction algorithm

doPixelStatus

"yes"

can be set to "no" to explicitly disable reading of pixel status data, all pixels will be used for common mode

doCommonMode

"yes"

can be set to "no" to explicitly disable common mode algorithm

doPixelGain

"yes"

can be set to "no" to explicitly disable pixel gain correction algorithm

Module cspad_mod.CsPadFilter

This is a filter module which implements skipping for the events which have too low signal in CsPad. filtering algorithm can be controlled trough the module parameter and/or parameters in calibration file. The name of the calibration file is "filter" and it will be searched in the standard calibration directories of the experiment.

parameter

default value

description

inputKey

''

string key used to locate uncalibrated data objects in event

source

"DetInfo(:Cspad)"

data source address

skipIfNoData

"yes"

when set to "yes" if the module does not find CsPad data in an event then it skips event

mode

-1

filtering mode, see below

parameters

all 0

list of parameters for filtering algorithm, up to 16 floating numbers

Parameter mode defines where the filter get its parameters and how it works. If set to -1 (default) then it reads parameters and actual mode value from a calibration text file which should contain one integer number for mode and up to 16 floating point parameters. If parameter is set to 0 or if the value read from file is 0 then no filtering is done, all events are passed through. If mode is set to 1 (from parameter or file) then filter calculates the number of pixels above certain threshold. Filter expects two values in parameter array: threshold for pixel value and minimum number of pixels above threshold. If second parameter is negative then it's assumed to be a percentage of the full number of pixels.

Filter works on individual image arrays (quadrants). Module checks for CsPad::DataV1, CsPad::DataV2, and CsPad2x2::ElementV1 objects in that order. If any object is found then module runs filter on images in that object. If any one image passes the filter then module returns and event is passed to downstream modules, otherwise framework skips this event.

With the default parameters module works with non-calibrated data, to switch to calibrated data add the module cspad_mod.CsPadCalib before filter and change inputKey parameter to calibrated.

Package CSPadPixCoords

Package CSPadPixCoords calculates the 2x1 section, quad, and CSPad pixel coordinates and produces the image.
For complete reference select Doxygen documentation.

Module CSPadPixCoords::PixCoordsTest

This module demonstrates of how to use the PixCoords2x1, PixCoordsQuad, and PixCoordsCSPad classes
in order to pre-calculate pixel coordinates, taking into account the calibration parameters.
Relevant images are produced in combination of the pixel coordinates with event data and saved in text files.

parameter

default value

description

calibDir

"/reg/d/psdm/CXI/cxi35711/calib"

directory with calibration file

typeGroupName

"CsPad::CalibV1"

data type and group names

source

"CxiDs1.0:Cspad.0"

source of data

runNumber

32

run number for calibration file

events

32

number of events before stop a job

filter

false

on/off for potential selection filter

Module CSPadPixCoords::CSPadImageProducer

CSPadImageProducer works in psana framework. It does a few operation as follows:

  • gets the pixel coordinates from PixCoords2x1, PixCoordsQuad, and PixCoordsCSPad classes,
  • gets data from the event (from Psana::CsPad::DataV#, Psana::CsPad::ElementV# (where # stands for 1 or 2) data objects or  from ndarray<const T,3> of shape [N][185][388])
  • produces the ndarray<const double,2> or Image2D<double> object with CSPad image for each event,
  • adds the image object in the event for processing in other modules.
    Time consumed to fill the CSPad image array (currently 1750x1750) is measured to be about 40 msec/event on psana0105.
  • produces image size pixel map array where real/fake pixels are marked as 1/0, respectively. This array is saved in the env.calibStore() as ndarray<const int16_t ,2> for source. If the file name fname_pixmap is not empty the map will be saved in this file in text format.
  • produces image size map of pixel indexes in the flatten [4][8][185][388] array where fake pixels are marked as -1. This array is saved in the env.calibStore() as ndarray<const int32_t ,2> for source. If the file name fname_pixnum is not empty the map will be saved in this file in text format.

parameter

default value

description

calibDir

 

directory with calibration files, by default it is set to .../<experiment>/calib

typeGroupName

"CsPad::CalibV1"

calibration type and group names

source

"CxiDs1.0:Cspad.0"

source of data

key

""

key for data processing stage

imgkey

"image"

output key for image saved in event

fname_pixmap""

file name with pixel map of the image; real/fake pixels are marked as 1/0, respectively

fname_pixnum""file name with pixel indexes in the flatten [4][8][185][388] array; fake pixels are marked as -1

tiltIsApplied

true

on/off for tilt angle of 2x1-sections and quads.

print_bits

0

verbosity:

  • =0 - print nothing,
  • +1 - input parameters,
  • +2 - calibration parameters,
  • +4 - consumed time per event,
  • +8 - information about image saved in event,
  • +16 - type sizes and id names,
  • +32 - information about pixel maps saved in files,
  • +64 - print everything from PSCalib::CSPadCalibPars

Remarks:

  • By default the empty key corresponds to raw data.

Module CSPadPixCoords::CSPadInterpolImageProducer

CSPadInterpolImageProducer works in psana framework. It does a few operation as follows:

  • gets the pixel coordinates from PixCoords2x1, PixCoordsQuad, and PixCoordsCSPad classes,
  • makes the arrays of neighbour addresses (quad, section, row, column) and weights as a function of the bin indexes (ix, iy) of the CSPad image,
  • gets data from the event,
  • produces the Image2D object with interpolated CSPad image for each event,
  • adds the ndarray<const double,2> or CSPadPixCoords::Image2D<double> object in the event for processing in other modules.
    In this module we use 4-node bi-linear interpolation algorithm.
    Time consumed to fill the CSPad image array (currently 1750x1750) is measured to be about 200 msec/event on psana0106. We consider options for acceleration using GPU or multi-core processing.

Module configuration parameters are the same as for the CSPadPixCoords::CSPadImageProducer.

Module CSPadPixCoords::CSPad2x2ImageProducer

CSPad2x2ImageProducer works in psana framework. It does a few operation as follows:

  • gets geometry alignment parameters center and tilt from /reg/d/psdm/<inst>/<experiment>/calib/... directory
  • gets the pixel coordinates from PixCoords2x1 and PixCoordsCSPad2x2 classes,
  • gets data from the event,
  • produces the Image2D<double> or ndarray<const double,2> object with CSPad image for each event,
  • adds the image object in the event for processing in other modules.
    The CSPad2x2 image array is currently shaped as (400,400).

parameter

default value

description

calibDir

 

path to the calib directory, if different from default

typeGroupName

"CsPad2x2::CalibV1"

calibration software version

source

"DetInfo(:Cspad2x2.1)"

source of data

inkey

 

key for data processing stage

outimgkey

"image"

output key for image saved in event

outtype

int16

type of output data from the list of supported (float, double, int, int16)

tiltIsApplied

true

on/off for tilt angle of 2x1-sections and quads - currently is not used

print_bits

0

verbosity:

  • =0 - print nothing
  • +1 - input parameters
  • +2 - calibration parameters
  • +4 - event ID
  • + 16 - type sizes and id names

Remarks:

  • By default the empty inkey corresponds to raw data.
  • If the outimgkey is defined as "Image2D", the image is saved in the event as a CSPadPixCoords::Image2D<double> object, otherwise (for other names) as a ndarray<const double,2> object.

See also Example for Module CSPadPixCoords::CSPad2x2ImageProducer.

Module CSPadPixCoords::CSPadNDArrProducer

CSPadNDArrProducer is a module for psana framework. It uses specified source and inkey parameters,

  • gets CSPAD configuration from the environment frame Psana::CsPad::ConfigV#,
  • gets CSPAD data from the event frames Psana::CsPad::DataV#, Psana::CsPad::ElementV#,
  • creates the data array as ndarray<const T,3>, where T stands for outtype, which may be double, float, int, or uint16,
  • puts the data array in the event with outkey tag.

The data array ndarray<const T,3> has a shape (N,185,388), where N≤32.
This array is combined from data arrays for quads, ndarray<const int32_t,3> with shape (M,185,388), where M≤8, taking into account the CSPAD configuration.

In real data some 2x1 may be turned off, that can be seen in configuration object.

parameter

default value

description

source

":Cspad.0"

source of data

inkey

 

key for data processing stage

outkey

"cspad_ndarr"

output key for image saved in event

outtype

float

type of output data from the list of supported (float, double, int, int16)

is_fullsizefalsesize of output array; true means [32,185,388], false means "as data" [N,185,388]
is_2darrayfalseif is_fullsize=true && is_2darray=true then shape of output 2-d array is [32*185,388] 

print_bits

0

verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - configuration parameters
  • +4 - consumed time per event
  • +16 - print type size and one character id name

See also Example for Module CSPadPixCoords::CSPadNDArrProducer.

Module CSPadPixCoords::CSPad2x2NDArrProducer

CSPad2x2NDArrProducer is a module for psana framework. It uses specified source and inkey parameters,

  • gets CSPAD2x2 configuration from the environment frame Psana::CsPad2x2::ConfigV#,
  • gets CSPAD2x2 data from the event frames Psana::CsPad2x2::ElementV#,
  • creates the data array as ndarray<const T,3>, where T stands for outtype, which may be double, float, int, or uint16,
  • puts the data array in the event with outkey tag.

The data array ndarray<const T,3> has a shape (185,388,2).

In real data some 2x1 may be turned off, that can be seen in configuration object.

parameter

default value

description

source

":Cspad2x2.0"

source of data

inkey

 

key for data processing stage

outkey

"cspad2x2_ndarr"

output key for image saved in event

outtype

float

type of output data from the list of supported (float, double, int, int16)

is_2darrayfalseif is_2darray=true then shape of output 2-d array is [185,388*2] 

print_bits

0

verbosity:

  • =0 - print nothing
  • +1 - input parameters
  • +2 - configuration parameters
  • +4 - consumed time per event
  • +16 - print type size and one character id name

See also Example for Module CSPadPixCoords::CSPad2x2NDArrProducer.

Package ImgPixSpectra

For complete reference see the Doxygen documentation.

Package ImgPixSpectra is available since release ana-0.4.1. It uses classes from the package CSPadPixCoords V00-02-01 or higher. To add this package in release use command:
addpkg CSPadPixCoords HEAD

Package ImgPixSpectra is intended to accumulate the spectra for all pixels of the image array.
Different modules of this package work with different data types for detectors like CSPad, CSPad2x2, Opal, Princeton camera, etc. All modules have the same interface and the same functionality.
In the loop over events from beginJob to endJob the image pixel amplitudes are accumulated in the 2-d array,
of the shape (<number-of-pixels>, <number-of-spectral-bins>). The first parameter is defined by the image size. The second is passed as an external parameter of the psana configuration file (psana.cfg) along with minimal and maximal amplitudes. At the endJob the spectral array is saved in file with specified name. Auxiliary file with the name extension *.sha is created in order to save the shape parameters. For example, the "cspad2x2-pix-spectra.txt.sha" outpuit file contains

NPIXELS  143560
NBINS    100
AMIN     500
AMAX     1000
NEVENTS  2549
ARRFNAME cspad2x2-pix-spectra.txt

This information can be used in analysis or presentation of this array.

Module ImgPixSpectra::CSPadPixSpectra

List of parameters:

parameter

default value

description

source

"CxiDs1.0:Cspad.0"

source of data for CSPad

events

1<<31U

number of events before stop a job

inputKey

 

input key for data processing stage

amin

0.

minimal spectral amplitude

amax

1000.

maximal spectral amplitude

nbins

100

number of bins in spectra

arr_fname

"..._spectral_array.txt"

output file name

See also: Example for Module ImgPixSpectra::CSPadPixSpectra.

Module ImgPixSpectra::CSPad2x2PixSpectra

The only difference in interface of this module from previous is in the default name for the source parameter and the output file name.

parameter

default value

description

source

"DetInfo(:Cspad2x2)"

source of data for CSPad2x2

See also: Example for Module ImgPixSpectra::CSPad2x2PixSpectra.

Module ImgPixSpectra::CameraPixSpectra

The only difference in interface of this module from previous is in the default name for the source parameter and the output file name.

parameter

default value

description

source

"DetInfo(SxrBeamline.0:Opal1000.1)"

source of data for Opal camera

or:

parameter

default value

description

source

"DetInfo(:Princeton)"

source of data for Princeton camera

See also Example for Package ImgPixSpectra.

Package ImgAlgos

This packages contains a few psana modules for analysis and image processing

Module ImgAlgos::Tahometer

This module measures the time interval for entire job and for each dn events and prints the rate info as requested by the print_bits parameter.

parameter

default value

description

dn

100

number of events between printout

print_bits

2

filter verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - start notice and summary at stop
  • +4 - instant rate performance after each dn events
  • +8 - print summary parameters for parser in log file

See also Example for Module ImgAlgos::Tahometer.

Module ImgAlgos::TimeStampFilter

This module passes only the events if their time stamp is in the requested range.
The range of allowed time stamps is defined by the configuration parameters.

parameter

default value

description

tsinterval

"1970-01-01 00:00:00.000000000 / 2100-01-01 00:00:00.000000000"

time-stamp interval string

tstamp_min

"1970-01-01 00:00:00.000000000"

minimal time-stamp string

tstamp_max

"2100-01-01 00:00:00.000000000"

maximal time-stamp string

filterIsOn

true

On/Off the filter

print_bits

0

filter verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - summary
  • +4 - event ID
  • +8 - time stamp variables for selected event

The time-stamp string is accepted in various formats:

  • YYYYMMDD HHMMSS.FFF
  • YYYYMMDDTHHMMSS.F
  • YYYY-MM-DD HH:MM:SS.FFF
    but the date field has to be presented mandatory.
    If the tsinterval is defined and is different from the default, it will be used in filter and override the tstamp_min and tstamp_max.

See also Example for TimeStampFilter and XtcOutputModule.

Module ImgAlgos::EventNumberFilter

This filter selects events by their number counting from the beginning of job, starting from 0. The event number is not a parameter which is associated with event. Use this filter cautiously on your own risk for debugging purpose only.

The unique parameter associated with the event is a time-stamp. The event number is not defined in the xtc file and is not a recommended to use parameter. The events are counted locally inside this filter from the beginning of job starting from 0. Be cautious of using this filter in consecutive jobs for different runs! For example, this filter returns different subsets of events for the files with raw and selected events.

parameter

default value

description

filterIsOn

true

On/Off filter

first

0

the first event from the beginning of job, starting from 0

last

1<<31

the last event from the beginning of job

evtstring

 

the string of events of intervals from the beginning of job

print_bits

0

filter verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - summary
  • +4 - selected event number and ID

There are two modes of operation of this filter.

  1. If the first and/or last event numbers are defined, then the filter will select events in this range only.
  2. If the evtstring is defined, only listed events of event ranges will be selected. For example, the evtstringparameter can be defined as

     2,5,11-15,20-25,29,30
    

    that means the list of events:

      2  5  11  12  13  14  15  20  21  22  23  24  25  29  30
    

    In the evtstring parameter the comma "," and sign minus "-" as a dash are the only allowed separators. Blank spaces are also allowed. Other characters may abort the program. The evtstring mode has higher priority than the 1st mode.
    The filterIsOn allows easy turn on/off this filter in *.cfg file.

Module ImgAlgos::EventCounterFilter

ImgAlgos::EventCounterFilter (ImgAlgos > V00-03-46) module is created by request of Thomas Kroll for experiment with mobile rack in SACLA .

Functionality:

Filter select events which numbers are listed in the input file ifname. This module uses local counter started from 1 for the 1-st event of the job and incremented in the event(...) method.  The file ifname contains the list of  integer event numbers in ascending order separated by space or '\n'.

 

parameter

default value

description

mode

1

filter mode: 0-off, 1-on, -1-on in inverted decision mode

ifname

 

input file name, is empty by default

print_bits

0

verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - input file content
  • +4 - filter summary
  • +16 - current event number
  • +32 - selection status ("is selected")

Input file (ifname)consists of integer numbers in ascending order separated by space or '\n', for example:

2 3 6 7 9 11 15
17 28 32 ...

 

 

Module ImgAlgos::AndorImageProducer

Functionality:

  • gets Andor data from the event for specified source and key_in parameters,
  • puts the ndarray<const double,2> object with camera image in the event using specified key_out parameter.

parameter

default value

description

source

"DetInfo(:Andor)"

source of data

key_in

 

key for input data

key_out

"andorimg"

output key for image saved in event

outtype

"asdata"

out data type: implemented values: asdata (default, uint16_t), float, double, int and int16.

print_bits

0

verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - data from event()
  • +4 - configuration parameters
  • +8 - table with sizes of types

Module ImgAlgos::PnccdNDArrProducer

Functionality:

  • gets pnCCD data from Psana::PNCCD::FramesV1 object from the event for specified source and key_in parameters,
  • puts the ndarray<const TOUT,3> object of shape 4x512x512 specified key_out parameter.

parameter

default value

description

source

"DetInfo(:pnCCD)"

source of data

key_in

 

key for input data

key_out

"andorimg"

output key for image saved in event

outtype

"asdata"

out data type: implemented values: asdata (default, unsigned short), float, double, int and int16.

print_bits

0

verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - data from event()
  • +4 - configuration parameters
  • +8 - table with sizes of types
  • +16 - warning about missing data

Module ImgAlgos::PnccdImageProducer

Functionality:

  • gets from the event store the object with pnCCD data of type
    • Psana::PNCCD::FullFrameV1 containing four [512][512] frames with T=uint16_t, or
    • ndarray<const T,3> , where shape=[4][512][512], T=unsigned short, float, double, int, or int16,
    for specified source and inkey parameters
  • puts the ndarray<const T,2> object with pnccd [1024+gap_rows][1024+gap_cols] image in the event using specified outimgkey parameter.

parameter

default value

description

source

"DetInfo(:pnCCD)"

source of data

inkey

 

key for input data

outimgkey

"pnccdimg"

output key for image saved in event

gap_rows0

gap between top and bottom segments in number of pixels

gap_cols0

gap between left and right segments in number of pixels

gap_value0Image effective pixel intensity value in the gap

print_bits

0

verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - initial portion of pnccd array
  • +4 - configuration pars
  • +16 - warning about missing data

See also Example for Module ImgAlgos::PnccdImageProducer.

Module ImgAlgos::CameraImageProducer

This module takes care about any generic camera image.
Functionality:

  • gets any camera image data Camera::FrameV1 from the event store for specified source and key_in parameters,
  • puts the ndarray<const T,2> object with camera image in the event using specified type outtype and key_out parameters.

parameter

default value

description

source

"DetInfo(:Camera)"

source of data

key_in

 

key for input data

key_out

"pnccdimg"

output key for image saved in event

outtype

"asdata"

out data type: implemented values: asdata (default, uint16_t), float, double, int and int16.

subtract_offset

true

on/off the amplitude offset using configuration data (not applied for outtype = asdata)

print_bits

0

verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - time stamp for each event
  • +4 - summary at the endJob
  • +8 - details about data format conversion and applied offset
  • +16 - configuration data for each beginCalibCycle

See also Example for Module ImgAlgos::CameraImageProducer.

Module ImgAlgos::PrincetonImageProducer

Functionality:

  • gets the Princeton or Pimax camera image data Princeton::FrameV1/2 or Pimax::FrameV1 from the event store for specified source and key_in parameters,
  • puts the ndarray<const T,2> object with camera image in the event using specified type outtype and key_out parameters.

parameter

default value

description

source

"DetInfo(:Princeton)"

source of data

key_in

 

key for input data (raw - by default)

key_out

"image"

output key for image saved in event

outtype

"asdata"

out data type: implemented values: asdata (default, uint16_t), float, double, int and int16.

print_bits

0

verbosity:

  • =0 - print nothing
  • +1 - input parameters
  • +2 - record for each event
  • +4 - summary at the endJob
  • +8 - first 10 elements of the data array
  • +16 - configuration info for each beginCalibCycle(...)

See also Example for Module ImgAlgos::PrincetonImageProducer.

 

Module ImgAlgos::AcqirisArrProducer

  • Gets acqiris configuration and data from Acqiris::ConfigV1 and Acqiris::DataDescV1,  objects using parameters source and key_in;
  • produces ndarray<const double,2> of shape[] = {nbrChannels, nbrSamples} for waveforms and time stamps;
  • saves configuration data in the file defined by fname_prefix;
  • saves waveforms and time stamps in the event store with keys key_wform and key_wtime.

parameter

default value

description

source

"DetInfo(:Acqiris)"

source of data

key_in

""

key for input data (by default  (empty)  – raw data)

key_wform

"acq_wform"

output key for waveforms saved in event

key_wtime

"acq_wtime"

output key for waveform times saved in event

fname_prefix

""

file name prefix for configuration parameters (by default  (empty)  – do not save file)

correct_ttrueon/off switch for time correction; if =false - array indexes are the same as in raw data waveform

print_bits

0

verbosity:

  • =0 - print nothing
  • +1 - input parameters
  • +2 - configuration parameters
  • +4 - record about saving file with configuration parameters
  • +8 - print part of waveform for all channels and segments in each event
  • +16 - print info about waveform indexes

 

Module ImgAlgos::AcqirisAverage

  • Gets Acqiris waveforms from event store as ndarray<const double,2> object using parameters source and key_in;
  • performs waveform selection controlled by parameters: thresholds, is_postive_signal, do_inverse_selection, in the range depending on local event numbers skip_events and proc_events;
  • after number of events proc_events or at the end of job (whatever happens first), saves array of averaged waveforms in the text file with name constructed from fname_ave_prefix and in the event store using parameters source and key_average.

parameter

default value

description

source

"DetInfo(:Acqiris)"

Source of data.

key_in

"acq_wform"

Key for input data (raw - by default).

key_average"acq-ave"Keyword for averaged waveform array saved in the evt store. If empty – array is not saved.
fname_ave_prefix"acq-ave"Text file name prefix for averaged array, full name will be extended by the experiment name, run number and suffix "-ave-wfs.txt", for example: "acq-amo01509-r0125-ave-wfs.txt".
thresholds""List of threshold values for all Acqiris channels separated by space. If empty – threshold selection is not applied, all waveforms are averaged.
is_positive_signal""Space-separated list of 1/0 values indicating signal polarity. For example, "1 1 1 0 1" (without the quotes!) would indicate Acqiris channels 1,2,3,5 contained positive signal polarity, while channel 4 contained a negative signal polarity.
do_inverse_selection

""

Space-separated ist of 1/0 values indicating which waveforms to include in average. For example, "0 0 0 1 0" (without the quotes) would tell the code to average only waveforms that do not cross the threshold for channels 1,2,3,5 while channel 4 would average only waveforms that do cross the threshold.

skip_events

0

Number of events (from the beginning of job) to skip before begin averaging.

proc_events

10000000

Number of events for averaging.

print_bits

0

Verbosity:

  • =0 - print nothing
  • +1 - input parameters
  • +2 - ndarray dimensions
  • +4 - begin/end accumulate statistics record
  • +8 - record about saving file with averaged array
  • +16 - statistics of averaged waveforms (number of accumulated)
  • +32 - part of the input waveform

 Example for Module ImgAlgos::AcqirisAverage

 

Module ImgAlgos::AcqirisCalib

  • Gets Acqiris waveforms from event store as ndarray<const double,2> object using parameters source and key_in;
  • processes events in the range depending on local event numbers skip_events and proc_events;
  • at the 1st processed event loads the fname_base_line file with baseline  ndarray<const double,2>;
  • subtract baseline from waveforms;
  • save corrected waveforms in the event store as ndarray<const double,2> object using parameters source and key_out.

parameter

default value

description

source

"DetInfo(:Acqiris)"

Source of data.

key_in

"acq_wform"

Key for input ndarray with raw waveforms from evt store. For now it is assumed that this array is produced by the

AcqirisArrProducer or AcqirisAverage modules.

key_out

"wf-calibrated"

Key for output ndarray with calibrated waveforms.

fname_base_line

"acq-ave"

Name of the input text file with array of the baselines for active  Acqiris channels. By default this name coincides with the name of the file produced by the module AcqirisAverage. This allows to use these to modules in chain with auto-generated names of files. If the file with auto-generated extended name (which looks like "acq-ave-amo01509-r0125-ave-wfs.txt") is not found on disk, the file name without extension will be tested. If it is not found as well, baseline subtraction will not be performed and output array will be identical to input one.

skip_events

0

Number of events (from the beginning of job) to skip before begin subtraction.

proc_events

10000000

Number of events for subtraction.

print_bits

0

Verbosity:

  • =0 - print nothing
  • +1 - input parameters
  • +2 - ndarray dimensions
  • +4 - begin/end of subtraction
  • +8 - Information about loading of the input file with baseline array
  • +16 - part of the input array
  • +32 - part of baseline array

 Example for Module ImgAlgos::AcqirisCalib

Module ImgAlgos::AcqirisCFD

  • Gets Acqiris waveforms from event store as ndarray<const double,2> object using parameters source, key_wform, key_wtime;
  • runs constant-fraction discriminator algorithm on all acqiris channels using user-specified per-channel parameters;
  • saves edges into the event as a set of ndarray<double,1>;

     

    parameter

    default value

    description

    source

    "DetInfo(:Acqiris)"

    Source of data.

    key_wform

    "acqiris_wform"

    Key for input ndarray with waveforms (either raw, or subtracted using AcqirisCalib module) from evt store.

    key_wtime

    "acqiris_wtime"

    Key for input ndarray with waveform times from evt store.

    key_edges

    "acqiris_edges_"

    Key for output ndarray<double,1> with calibrated waveforms. This key will have the acqiris channel number (1 thru 20) appended to the end of it, and the data for that channel will be added to the event only if edges were found.

    baselines

    ""

    A list of baseline values (one per channel) to subtract from the waveform in volts.

    fractions

    ""

    A list of fractions (one per channel) between 0 and 1. The edge-time reported will be at the time when the pulse is at this fraction of the peak value.

    thresholds

    ""

    A list of threshold values (one per channel) in volts that indicate a new edge should be found. If this value is less than the baseline, then the algorithm will look for negative pulses, otherwise it will look for positive pulses.

    deadtimes

    ""

    A list of deadtimes (one per channel) in seconds. After each edge the algorithm will ignore any new hits in this time interval.

    leading_edges

    ""

    A list of 0/1 values (one per channel) indicating whether edge-times are desired for leading edges (1) or falling edges (0).

 Example for Module ImgAlgos::AcqirisCFD

Module ImgAlgos::NDArrAverage

This module averages over events the per-element data of the image array (ndarray<const T,NDim>, where T is implemented for almost all types: int, int16, uint, float, double etc., NDim≤5) and saves files for sum, averaged, rms values, mask, and, the hot pixel map. Input ndarray can be specified by the source and key parameters. Averaging may have up to three stages, depending on configuration parameters:

  • 0-stage: the pixel amplitudes are averaged without any constrains for events from 0 to evts_stage1, the preliminary averaged and rms values are defined for each pixel at the end of this stage.
  • 1-stage: starting from event evts_stage1 the pixel data are collected only for abs(amplitude-average0) < gate_width1. At the end of this stage the preliminary averaged and rms values are defined for each pixel.
  • 2-stage: starting from the event evts_stage1 + evts_stage2 the pixel data are collected only for abs(amplitude-average1) < gate_width2. At the end of this stage the preliminary averaged and rms values are defined for each pixel and saved in the files specified by the avefile and rmsfile parameters, respectively.
    This 3-stage averaging algorithm eliminates large statistical fluctuations in the pixel amplitude spectrum.

If the parameter thr_rms_ADU is set ≤0 then threshold value is defined automatically using constrained averaging of pixel rms values in 3 iterations.  The threshold value is defined as mean+5*rms.

parameter

default value

description

source

DetInfo(:Opal1000)

input source of data

key

 

key for input data, for example, it might be "calibrated"

sumfile

""

out file with sum of amplitudes, saved if the name is not empty

avefile

""

out file with averaged amplitudes, saved if the name is not empty

rmsfile

""

out file with rms, saved if the name is not empty

maskfile""out file with pixel mask with 0/1-for bad/good pixels, saved if the name is not empty

hotpixfile

""

out file with pixel bit-words: 0/1/2/4 for good/hot/saturated/cold, saved if the name is not empty

maxfile""out file with maximal value per pixel over events
ftypetxtout file type: txt (default), metatxt, bin

thr_rms_ADU

10000.

threshold on rms (in ADU); =0 - use auto-evaluated threshold. If rms exceeds this threshold - pixel is hot.

thr_min_ADU

-100000.

threshold on minimal intensity (in ADU); if ave exceeds this threshold - pixel is good

thr_max_ADU

100000.

threshold on maximal intensity (in ADU); if ave exceeds this threshold - pixel is bad

evts_stage1

1000000

number of events before stage 1

evts_stage2

0

additional number of events before stage 2

gate_width1

0

gate_width for stage 1

gate_width2

0

gate_width for stage 2

print_bits

0

module verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - event record
  • +4 - beginning of 3 stages
  • +8 - processed statistics at the end of each stage
  • +16 - records for saved files
  • +32 - summary with keywords for parser
  • +64 - statistics of bad pixels
  • +128 - mean and rms in 3 iterations of the threshold auto-evaluation algorithm and evaluated threshold
  • +256 - warning if ndarray shape is not defined yet

If all file names are empty (by default), the files with pre-defined names "arr-ave-<exp>-r<run>.dat" and "arr-rms-<exp>-r<run>.dat" will be saved for averaged and rms arrays, respectively. Otherwise, the files with specified names will be saved.
Default parameters are set for regular single-stage averaging without any constrains.
See also Example for Module ImgAlgos::NDArrAverage.

Module ImgAlgos::NDArrCalib

Functionality

  • NDArrCalib uses the source and key_in parameters to get any ndarray<const T,NDIM> object from the event store, where T stands forint16_t, uint16_t, int, float, uint8_t, or double, 1≤NDim≤5,
  • automaticly gets parameters from calibration store for types pedestals, common_mode, pixel_status, pixel_gain, and pixel_rms,
  • gets parameters from user-defined files fname_bkgd and  fname_mask, if their names are specified,
  • the specified by the do_... parameter corrections are applied to raw data  ndarray<const T,NDIM> as follows:

        1. subtracts pedestals,
        2. subtracts common mode,
        3. subtracts normalized background,
        4. apply gain factors,
        5. apply hot/bad pixel_status mask,
        6. apply mask  (1/0 = good/bad pixels),
        7. apply threshold as a common low level,
        8. apply per-pixel threshold as N*RMS,
  • and saves the corrected ndarray<const double,NDim> in the event with key key_out
  • In ImgAlgos V00-02-01 implemented detectors: CsPad, CsPad2x2, Pnccd, Princeton, Andor, Opal1000, Opal4000

Control on corrections

A_cor = A_raw
        (1) - pedestal             | if do_peds==true and pedestals are available in calib store
        (2) - common mode          | if do_cmod==true and common_mode parameters are available in calib store
        (3) - N*background         | if do_bkgd==true and the file name is specified in the parameter fname_bkgd and bkgd_ind_* are set
        (4) * gain                 | if do_gain==true and pixel_gain are available in calib store
        (5) apply bad pixel status | if do_stat==true and pixel_status are available in calib store
        (6) apply mask             | if do_mask==true and the file name is specified in the parameter fname_mask. Parameter masked_value is used to substitute masked values.
        (7) apply N*RMS threshold  | if do_nrms==true and pixel_rms are available in calib store, parameters threshold_nrms and below_thre_value are set
        (8) apply common threshold | if do_thre==true. Parameter below_thre_value is used to substitute below threshold values.


Configuration parameters

parameter

default value

description

source

DetInfo(:Camera)

source of data

key_in

 

key for input ndarray<const T,NDIM>

key_out

calibrated

output key for calibrated image saved in event

do_peds

false

true: pedestals subtracted if available in calib store

do_cmod

false

true: common mode correction is evaluated and applied [Ref.]

do_stat

false

true: bad/hot pixels in pixel_status are masked

do_mask

false

true: mask is applied if the file fname_mask is available (1/0 = good/bad pixels)

do_bkgd

false

true: normalized background is subtracted if the file fname_bkgd is available

do_gain

false

true: pixel_gain correction is applied if available in calib store

do_nrms

false

true: per-pixel threshold is applied if pixel_rms  is available in calib store

do_thre

false

true: low level threshold in ADU is applied

fname_bkgd

 

input file name for background, applied if the file name is specified

fname_mask

 

input file name for mask, applied if the file name is specified

masked_value

0.

intensity value (in ADU) substituted for masked pixels

threshold_nrms

3.

threshold as a number of sigmas to pixel_rms parameters

threshold

0.

common low level threshold in ADU

below_thre_value

0.

intensity substituted for pixels below threshold

bkgd_ind_min

0

minimal index in flatten ndarray, which is used for background normalization

bkgd_ind_max

100

maximal index in flatten ndarray, which is used for background normalization

bkgd_ind_inc

2

index increment in flatten ndarray, which is used for background normalization

print_bits

0

verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - calibration parameters
  • +4 - common mode algorithm parameters
  • +8 - ndarray parameters; type, ndim, shape, etc.
  • +16- time stamp for each event
  • +32 - first 10 elements of the raw image data
  • +64 - first 10 elements of the calibrated image data

 

To add implementation for any other new detector "Det":

        1) add file pdscalibdata/DetBaseV1.h

        2) in PSCalib/src/GenericCalibPars.cpp add
             #include "pdscalibdata/DetBaseV1.h"
             ...
             template class PSCalib::GenericCalibPars<pdscalibdata::Opal4000BaseV1>;

        3) in PSCalib/include/CalibParsStore.h add
             #include "pdscalibdata/DetBaseV1.h"
             ...
             return new PSCalib::GenericCalibPars<pdscalibdata::DetBaseV1>(calibdir, type_group, src, runnum, prbits);

Doxygen documentation for interface: CalibParsStore,  GenericCalibPars

 See also Example for Module ImgAlgos::NDArrCalib,

Test of the NDArrCalib module for pnCCD.

 

Module ImgAlgos::PixCoordsProducer

Functionality

For data source  in each run loads/updates calibration geometry file from the calibration DB, evaluates pixel coordinate arrays using class PSCalib::GeometryAccess and saves them as ndarray<const float,1> in the event store for keywords x-pix-coords, y-pix-coords, and z-pix-coords.

The main idea of this module is that calibration geometry file will be found and loaded (if available) automatically.

Configuration parameters

parameter

default value

description

source

DetInfo(:Cspad)

source of data

group

 

group of calibration type, by default will be set from source

key_out_x

x-pix-coords

output key pixel x-coordinate[um] array

key_out_y

y-pix-coords

output key pixel y-coordinate[um] array

key_out_z

z-pix-coords

output key pixel z-coordinate[um] array

print_bits

0

verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - calibration file path
  • +4 - list of geometry objects
  • +8 - list of geometry objects with children
  • +16- comments from calibration file
  • +32 - pixel coordinates

Current version of this module works with CSPAD and CSPAD2x2. It can be extended for other detectors, whenever necessary.

See Example for Module ImgAlgos::PixCoordsProducer,

 

  • No labels