This page provides a list of existing modules for psana framework. Only the modules that are included in the standard analysis releases appear on this page.
Note that most modules have configuration parameters which are documented in tables. For example:
parameter | default value | description |
---|---|---|
| "=" | separator string |
| 80 | number of repeats of separator string on |
This page provides examples for selected modules in Psana Module Examples.
You can find the source code at SLAC (e.g. for module CSPadPixCoords::PixCoordsTest) in these two directories:
/reg/g/psdm/sw/releases/ana-current/CSPadPixCoords/src/PixCoordsTest.cpp (.cpp for c++ source code, or .py for python source code) /reg/g/psdm/sw/releases/ana-current/CSPadPixCoords/include/PixCoordsTest.h (c++ include file) |
Psana package include several simple modules which do generic tasks that do not need knowledge of the event data types.
This module dumps the list of the event keys in the event or configuration store. Event key is a triplet of data type, data source address, and string key.
Example of the output produced by this module:
Event keys: EventKey(type=Psana::EvrData::DataV3, src=DetInfo(NoDetector.0:Evr.0)) EventKey(type=Psana::Camera::FrameV1, src=DetInfo(CxiDg1.0:Tm6740.0)) EventKey(type=Psana::Ipimb::DataV1, src=DetInfo(CxiDg1.0:Ipimb.0)) ... |
This module have two optional parameters to control output:
parameter | default value | description |
---|---|---|
| false | Flag: true - print in event() info about env.configStore().keys() |
| false | Flag: true - print in event() info about env.calibStore().keys() |
This module is useful to display the content of the events without dumping all the data. Example command which uses this module is:
% psana -m EventKeys <input-files> |
Note that one can specify EventKeys
instead of psana.EventKeys
as psana
package name is optional.
This module prints the content of the Event ID object on every event.
Example of the output produced by this module:
[info:psana.PrintEventId] event ID: XtcEventId(run=100, time=2010-12-12 11:09:36.300506429-08) [info:psana.PrintEventId] event ID: XtcEventId(run=100, time=2010-12-12 11:09:36.317163082-08) |
This module prints separator line on every event. Can be used to indicate event boundaries in job's log.
parameter | default value | description |
---|---|---|
| "=" | separator string |
| 80 | number of repeats of separator string on |
Example of the output produced by this module with the default parameters (including output from PrintEventId module):
[info:/root/] ================================================================================ [info:psana.PrintEventId] event ID: XtcEventId(run=100, time=2010-12-12 11:09:36.300506429-08) [info:/root/] ================================================================================ [info:psana.PrintEventId] event ID: XtcEventId(run=100, time=2010-12-12 11:09:36.317163082-08) |
This package contains modules that are meant to be used as examples of accessing different data types or framework services. Can be used by developers as templates for new modules. To find the code:
Extracts and dumps the content of Acqiris configuration (Psana::Acqiris::ConfigV1
) and event data (Psana::Acqiris::DataDescV1
) objects.
parameter | default value | description |
---|---|---|
| "DetInfo(:Acqiris)" | data source address |
Extracts and dumps the content of Acqiris TDC configuration (Psana::Acqiris::TdcConfigV1
) and event data (Psana::Acqiris::TdcDataV1
) objects.
parameter | default value | description |
---|---|---|
| "DetInfo(:AcqTDC.0)" | data source address |
Extracts and dumps the content of beamline data objects (Psana::Bld::BldDataEBeamV0
, Psana::Bld::BldDataEBeam
, Psana::Bld::BldDataPhaseCavity
, Psana::Bld::BldDataFEEGasDetEnergy
, and Psana::Bld::BldDataIpimb
).
parameter | default value | description |
---|---|---|
| "BldInfo(EBeam)" | data source address for |
| "BldInfo(PhaseCavity)" | data source address for |
| "BldInfo(FEEGasDetEnergy)" | data source address for |
| "BldInfo(NH2-SB1-IPM-01)" | data source address for |
Extracts and dumps the content of camera configuration (Psana::Camera::FrameFexConfigV1
) and event data (Psana::Camera::FrameV1
and Psana::Camera::TwoDGaussianV1
) objects.
parameter | default value | description |
---|---|---|
| "DetInfo(:Opal1000)" | data source address |
Extracts and dumps the content of scan control configuration (Psana::ControlData::ConfigV1
) object.
parameter | default value | description |
---|---|---|
| "ProcInfo()" | data source address |
Extracts and dumps the content of CsPad configuration (Psana::CsPad::ConfigV*
) and event data (Psana::CsPad::DataV*
) objects.
parameter | default value | description |
---|---|---|
| "DetInfo(:Cspad)" | data source address |
Extracts and dumps the content of encoder configuration (Psana::Encoder::ConfigV1
) and event data (Psana::Encoder::DataV*
) objects.
parameter | default value | description |
---|---|---|
| "DetInfo(:Encoder)" | data source address |
Extracts and dumps the list of EPICS PVs and their values and status information on every event.
Extracts and dumps the content of Evr configuration (Psana::EvrData::ConfigV*
and Psana::EvrData::IOConfigV1
) and event data (Psana::EvrData::DataV*
) objects.
parameter | default value | description |
---|---|---|
| "DetInfo(:Evr)" | data source address |
Extracts and dumps the content of FCCD configuration (Psana::FCCD::FccdConfigV*
) objects. To dump event data objects use psana_examples.DumpCamera module.
parameter | default value | description |
---|---|---|
| "DetInfo(:Fccd)" | data source address |
Extracts and dumps the content of IPIMB configuration (Psana::Ipimb::ConfigV1
) and event data (Psana::Ipimb::DataV1
) objects.
parameter | default value | description |
---|---|---|
| "DetInfo(:Ipimb)" | data source address |
Extracts and dumps the content of LUSI configuration (Psana::Lusi::DiodeFexConfigV1
, Psana::Lusi::IpmFexConfigV1
, and Psana::Lusi::PimImageConfigV1
) and event data (Psana::Lusi::DiodeFexV1
and Psana::Lusi::IpmFexV1
) objects.
parameter | default value | description |
---|---|---|
| "DetInfo(:Tm6740)" | data source address for |
| "DetInfo(:Ipimb)" | data source address for other data |
Extracts and dumps the content of Opal1000 configuration (Psana::Opal1k::ConfigV1
) object. To dump event data objects use psana_examples.DumpCamera module.
parameter | default value | description |
---|---|---|
| "DetInfo(:Opal1000)" | data source address |
Extracts and dumps the content of pnCCD configuration (Psana::PNCCD::ConfigV*
) and event data (Psana::PNCCD::FrameV1
) objects.
parameter | default value | description |
---|---|---|
| "DetInfo(:pnCCD)" | data source address |
Extracts and dumps the content of Princeton configuration (Psana::Princeton::ConfigV1
) and event data (Psana::Princeton::FrameV1
) objects.
parameter | default value | description |
---|---|---|
| "DetInfo(:Princeton)" | data source address |
Extracts and dumps the content of Pulnix configuration (Psana::Pulnix::TM6740ConfigV*
) objects.
parameter | default value | description |
---|---|---|
| "DetInfo(:Tm6740)" | data source address |
This module is an example of histogramming service usage. It extracts beam line data and fills couple of histograms with the beam parameters.
parameter | default value | description |
---|---|---|
| "BldInfo(EBeam)" | data source address |
Package cspad_mod contains common modules useful for analysis of data from CsPad detector.
This module is supposed to run on dark cspad2x2 frame data. It calculates average and standard deviation values for each pixel and writes these values to output files in text format.
parameter | default value | description |
---|---|---|
| "DetInfo(:Cspad)" | name of the data source; default value is adequate if there is only one CsPad device in setup, if there is more than one device then more specific value should be provided |
| "cspad-pedestals.dat" | name of the output file for average values, if empty file will not be written |
| "cspad-noise.dat" | name of the output file for standard deviation values, if empty file will not be written |
To use this module with default parameters one could use this command:
psana -m cspad_mod.CsPadPedestals input-files.xtc |
which will produce files cspad-pedestals.dat
and cspad-noise.dat
in the current directory (if input file contains CsPad data).
To change module parameters one could create file with name psana.cfg
in the current directory with a contents like this:
[psana] modules = cspad_mod.CsPadPedestals [cspad_mod.CsPadPedestals] source = DetInfo(CxiDs1.0:Cspad.0) output = pedestals.dat noise = |
and then run psana:
psana input-files.xtc |
This should produce file pedestals.dat
in the current directory. As the source address is more specific this configuration file can be used even if input data contain more than one CsPad device.
This module is supposed to run on cspad2x2 dark frame data. It calculates average and standard deviation values for each pixel and writes these values to output files in text format.
parameter | default value | description |
---|---|---|
| "DetInfo(:Cspad2x2)" | name of the data source; default value is adequate if there is only one CsPad2x2 device in setup, if there is more than one device then more specific value should be provided |
| "cspad2x2-pedestals.dat" | name of the output file for average values, if empty file will not be written |
| "cspad2x2-noise.dat" | name of the output file for standard deviation values, if empty file will not be written |
To use this module with default parameters one could use this command:
psana -m cspad_mod.CsPad2x2Pedestals input-files.xtc |
which will produce files cspad2x2-pedestals.dat
and cspad2x2-noise.dat
in the current directory (if input file contains CsPad2x2 data).
To change module parameters one could create file with name psana.cfg
in the current directory with a contents like this:
[psana] modules = cspad_mod.CsPad2x2Pedestals [cspad_mod.CsPad2x2Pedestals] source = DetInfo(CxiSc1.0:Cspad2x2.0) output = pedestals.dat noise = |
and then run psana:
psana input-files.xtc |
This should produce file pedestals.dat
in the current directory. As the source address is more specific this configuration file can be used even if input data contain more than one CsPad2x2 device.
This module performs standard CsPad calibration procedures: per-pixel pedestal subtraction and common mode correction. The algorithms used in calibrations are identical to translator algorithms, the result of calibration should be the same as the data stored in HDF5 files.
The module uses three input calibration files which should be located in the standard calibration directory of the corresponding experiment:
pedestals
– per-pixel pedestal data (produced by the CsPadPedestals module or standalone applications)pixel_status
– hot/dead status for every pixel, used by common mode algorithmcommon_mode
– file with parameters controlling common mode calculationpixel_gain
– per-pixel gain data, inverse pixel gain (1/gain) as floating numberIf any of the files is missing then corresponding algorithm is not executed, if pixel_status
file is missing then all pixels are assumed to have working status. Algorithms inside module are executed following this order:
The calibration is performed on every CsPad image (full or 2x2) found in the event and works even if there is more than one CsPad data object. The original objects are preserved in event and result of the calibration is stored in event with different string key ("calibrated" by default, can be changed with module parameter).
parameter | default value | description |
---|---|---|
| "" | string key used to locate uncalibrated data objects in event |
| "calibrated" | string key used to store calibrated data objects in event |
| "yes" | can be set to "no" to explicitly disable pedestal subtraction algorithm |
| "yes" | can be set to "no" to explicitly disable reading of pixel status data, all pixels will be used for common mode |
| "yes" | can be set to "no" to explicitly disable common mode algorithm |
| "yes" | can be set to "no" to explicitly disable pixel gain correction algorithm |
This is a filter module which implements skipping for the events which have too low signal in CsPad. filtering algorithm can be controlled trough the module parameter and/or parameters in calibration file. The name of the calibration file is "filter" and it will be searched in the standard calibration directories of the experiment.
parameter | default value | description |
---|---|---|
| '' | string key used to locate uncalibrated data objects in event |
| "DetInfo(:Cspad)" | data source address |
| "yes" | when set to "yes" if the module does not find CsPad data in an event then it skips event |
| -1 | filtering mode, see below |
| all 0 | list of parameters for filtering algorithm, up to 16 floating numbers |
Parameter mode
defines where the filter get its parameters and how it works. If set to -1 (default) then it reads parameters and actual mode value from a calibration text file which should contain one integer number for mode and up to 16 floating point parameters. If parameter is set to 0 or if the value read from file is 0 then no filtering is done, all events are passed through. If mode is set to 1 (from parameter or file) then filter calculates the number of pixels above certain threshold. Filter expects two values in parameter array: threshold for pixel value and minimum number of pixels above threshold. If second parameter is negative then it's assumed to be a percentage of the full number of pixels.
Filter works on individual image arrays (quadrants). Module checks for CsPad::DataV1, CsPad::DataV2, and CsPad2x2::ElementV1 objects in that order. If any object is found then module runs filter on images in that object. If any one image passes the filter then module returns and event is passed to downstream modules, otherwise framework skips this event.
With the default parameters module works with non-calibrated data, to switch to calibrated data add the module cspad_mod.CsPadCalib before filter and change inputKey
parameter to calibrated
.
Package CSPadPixCoords calculates the 2x1 section, quad, and CSPad pixel coordinates and produces the image.
For complete reference select Doxygen documentation.
This module demonstrates of how to use the PixCoords2x1, PixCoordsQuad, and PixCoordsCSPad classes
in order to pre-calculate pixel coordinates, taking into account the calibration parameters.
Relevant images are produced in combination of the pixel coordinates with event data and saved in text files.
parameter | default value | description |
---|---|---|
| "/reg/d/psdm/CXI/cxi35711/calib" | directory with calibration file |
| "CsPad::CalibV1" | data type and group names |
| "CxiDs1.0:Cspad.0" | source of data |
| 32 | run number for calibration file |
| 32 | number of events before stop a job |
| false | on/off for potential selection filter |
CSPadImageProducer works in psana framework. It does a few operation as follows:
const
T,3> of shape [N][185][388])ndarray<
const
double,2>
or Image2D<double>
object with CSPad image for each event,env.calibStore()
as ndarray<
const
int16_t ,2>
for source
. If the file name fname_pixmap
is not empty the map will be saved in this file in text format.env.calibStore()
as ndarray<
const
int32_t ,2>
for source
. If the file name fname_pixnum
is not empty the map will be saved in this file in text format.parameter | default value | description |
---|---|---|
|
| directory with calibration files, by default it is set to .../<experiment>/calib |
| "CsPad::CalibV1" | calibration type and group names |
| "CxiDs1.0:Cspad.0" | source of data |
| "" | key for data processing stage |
| "image" | output key for image saved in event |
fname_pixmap | "" | file name with pixel map of the image; real/fake pixels are marked as 1/0, respectively |
fname_pixnum | "" | file name with pixel indexes in the flatten [4][8][185][388] array; fake pixels are marked as -1 |
| true | on/off for tilt angle of 2x1-sections and quads. |
| 0 | verbosity:
|
Remarks:
key
corresponds to raw data.CSPadInterpolImageProducer works in psana framework. It does a few operation as follows:
ndarray<
const
double,2>
or CSPadPixCoords::Image2D<double>
object in the event for processing in other modules.Module configuration parameters are the same as for the CSPadPixCoords::CSPadImageProducer.
CSPad2x2ImageProducer works in psana framework. It does a few operation as follows:
center
and tilt
from /reg/d/psdm/<inst>/<experiment>/calib/...
directoryImage2D<double>
or ndarray<
const
double,2>
object with CSPad image for each event,parameter | default value | description |
---|---|---|
|
| path to the |
| "CsPad2x2::CalibV1" | calibration software version |
| "DetInfo(:Cspad2x2.1)" | source of data |
|
| key for data processing stage |
| "image" | output key for image saved in event |
| int16 | type of output data from the list of supported (float, double, int, int16) |
| true | on/off for tilt angle of 2x1-sections and quads - currently is not used |
| 0 | verbosity:
|
Remarks:
inkey
corresponds to raw data.outimgkey
is defined as "Image2D", the image is saved in the event as a CSPadPixCoords::Image2D<double>
object, otherwise (for other names) as a ndarray<
const
double,2>
object.See also Example for Module CSPadPixCoords::CSPad2x2ImageProducer.
CSPadNDArrProducer is a module for psana framework. It uses specified source
and inkey
parameters,
Psana::CsPad::ConfigV#
,Psana::CsPad::DataV#
, Psana::CsPad::ElementV#
,ndarray<
const
T,3>
, where T stands for outtype
, which may be double, float, int, or uint16,outkey
tag.The data array ndarray<
has a shape (N,185,388), where N≤32.const
T,3>
This array is combined from data arrays for quads, ndarray<
with shape (M,185,388), where M≤8, taking into account the CSPAD configuration.const
int32_t,3>
In real data some 2x1 may be turned off, that can be seen in configuration object. |
parameter | default value | description |
---|---|---|
| ":Cspad.0" | source of data |
|
| key for data processing stage |
| "cspad_ndarr" | output key for image saved in event |
| float | type of output data from the list of supported (float, double, int, int16) |
is_fullsize | false | size of output array; true means [32,185,388], false means "as data" [N,185,388] |
is_2darray | false | if is_fullsize=true &&
then shape of output 2-d array is [32*185,388] |
| 0 | verbosity:
|
See also Example for Module CSPadPixCoords::CSPadNDArrProducer.
CSPad2x2NDArrProducer is a module for psana framework. It uses specified source
and inkey
parameters,
Psana::CsPad2x2::ConfigV#
,Psana::CsPad2x2::ElementV#
,ndarray<
const
T,3>
, where T stands for outtype
, which may be double, float, int, or uint16,outkey
tag.The data array ndarray<
has a shape (185,388,2).const
T,3>
In real data some 2x1 may be turned off, that can be seen in configuration object. |
parameter | default value | description |
---|---|---|
| ":Cspad2x2.0" | source of data |
|
| key for data processing stage |
| "cspad2x2_ndarr" | output key for image saved in event |
| float | type of output data from the list of supported (float, double, int, int16) |
is_2darray | false | if
then shape of output 2-d array is [185,388*2] |
| 0 | verbosity:
|
See also Example for Module CSPadPixCoords::CSPad2x2NDArrProducer.
CSPad2x2NDArrReshape is a module for psana framework. This module reshapes ndarrays from cspad2x2 data format (185,388,2) to cspad-"natural" format (2,185,388). Input arrays for re-shaping are specified by the source
and keys_in
parameters. Output arrays - by the source
and keys_out
parameters. Currently four data type are supported for input arrays; int16, int, float, and double of const and non-const types. Output arrays have the same data type as input.
Conventions for key lists description
Configuration parameters:
parameter | default value | description |
---|---|---|
| ":Cspad2x2" | source of data |
|
| list of input keys separated by space. At least one should be specified. May have suffixes as <key>:<suffix>, which will be dropped for default |
|
| list of output keys separated by space. Is empty by default, list will be generated from the list defined in |
| 0 | verbosity:
|
See also Example for Module CSPadPixCoords::CSPad2x2NDArrReshape.
JIRA Issue:
For complete reference see the Doxygen documentation.
Package ImgPixSpectra is available since release |
Package ImgPixSpectra is intended to accumulate the spectra for all pixels of the image array.
Different modules of this package work with different data types for detectors like CSPad, CSPad2x2, Opal, Princeton camera, etc. All modules have the same interface and the same functionality.
In the loop over events from beginJob
to endJob
the image pixel amplitudes are accumulated in the 2-d array,
of the shape (<number-of-pixels>, <number-of-spectral-bins>). The first parameter is defined by the image size. The second is passed as an external parameter of the psana configuration file (psana.cfg
) along with minimal and maximal amplitudes. At the endJob
the spectral array is saved in file with specified name. Auxiliary file with the name extension *.sha
is created in order to save the shape parameters. For example, the "cspad2x2-pix-spectra.txt.sha" outpuit file contains
NPIXELS 143560 NBINS 100 AMIN 500 AMAX 1000 NEVENTS 2549 ARRFNAME cspad2x2-pix-spectra.txt |
This information can be used in analysis or presentation of this array.
List of parameters:
parameter | default value | description |
---|---|---|
| "CxiDs1.0:Cspad.0" | source of data for CSPad |
| 1<<31U | number of events before stop a job |
|
| input key for data processing stage |
| 0. | minimal spectral amplitude |
| 1000. | maximal spectral amplitude |
| 100 | number of bins in spectra |
| "..._spectral_array.txt" | output file name |
See also: Example for Module ImgPixSpectra::CSPadPixSpectra.
The only difference in interface of this module from previous is in the default name for the source parameter and the output file name.
parameter | default value | description |
---|---|---|
| "DetInfo(:Cspad2x2)" | source of data for CSPad2x2 |
See also: Example for Module ImgPixSpectra::CSPad2x2PixSpectra.
The only difference in interface of this module from previous is in the default name for the source parameter and the output file name.
parameter | default value | description |
---|---|---|
| "DetInfo(SxrBeamline.0:Opal1000.1)" | source of data for Opal camera |
or:
parameter | default value | description |
---|---|---|
| "DetInfo(:Princeton)" | source of data for Princeton camera |
See also Example for Package ImgPixSpectra.
This packages contains a few psana
modules for analysis and image processing.
This module measures the time interval for entire job and for each dn
events and prints the rate info as requested by the print_bits
parameter.
parameter | default value | description |
---|---|---|
| 100 | number of events between printout |
| 2 | filter verbosity:
|
See also Example for Module ImgAlgos::Tahometer.
This module passes only the events if their time stamp is in the requested range.
The range of allowed time stamps is defined by the configuration parameters.
parameter | default value | description |
---|---|---|
| "1970-01-01 00:00:00.000000000 / 2100-01-01 00:00:00.000000000" | time-stamp interval string |
| "1970-01-01 00:00:00.000000000" | minimal time-stamp string |
| "2100-01-01 00:00:00.000000000" | maximal time-stamp string |
| true | On/Off the filter |
| 0 | filter verbosity:
|
The time-stamp string is accepted in various formats:
YYYYMMDD HHMMSS.FFF
YYYYMMDDTHHMMSS.F
YYYY-MM-DD HH:MM:SS.FFF
tsinterval
is defined and is different from the default, it will be used in filter and override the tstamp_min
and tstamp_max
.See also Example for TimeStampFilter and XtcOutputModule.
This filter selects events by their number counting from the beginning of job, starting from 0. The event number is not a parameter which is associated with event. Use this filter cautiously on your own risk for debugging purpose only.
The unique parameter associated with the event is a time-stamp. The event number is not defined in the xtc file and is not a recommended to use parameter. The events are counted locally inside this filter from the beginning of job starting from 0. Be cautious of using this filter in consecutive jobs for different runs! For example, this filter returns different subsets of events for the files with raw and selected events. |
parameter | default value | description |
---|---|---|
| true | On/Off filter |
| 0 | the first event from the beginning of job, starting from 0 |
| 1<<31 | the last event from the beginning of job |
|
| the string of events of intervals from the beginning of job |
| 0 | filter verbosity:
|
There are two modes of operation of this filter.
first
and/or last
event numbers are defined, then the filter will select events in this range only.If the evtstring
is defined, only listed events of event ranges will be selected. For example, the evtstring
parameter can be defined as
2,5,11-15,20-25,29,30 |
that means the list of events:
2 5 11 12 13 14 15 20 21 22 23 24 25 29 30 |
In the evtstring
parameter the comma "," and sign minus "-" as a dash are the only allowed separators. Blank spaces are also allowed. Other characters may abort the program. The evtstring
mode has higher priority than the 1st mode.
The filterIsOn
allows easy turn on/off this filter in *.cfg
file.
ImgAlgos::EventCounterFilter (ImgAlgos > V00-03-46) module is created by request of Thomas Kroll for experiment with mobile rack in SACLA .
Functionality:
Filter select events which numbers are listed in the input file ifname
. This module uses local counter started from 1 for the 1-st event of the job and incremented in the event(...) method. The file ifname
contains the list of integer event numbers in ascending order separated by space or '\n'.
parameter | default value | description |
---|---|---|
| 1 | filter mode: 0-off, 1-on, -1-on in inverted decision mode |
| input file name, is empty by default | |
| 0 | verbosity:
|
Input file (ifname
)consists of integer numbers in ascending order separated by space or '\n', for example:
2 3 6 7 9 11 15 17 28 32 ... |
ImgAlgos::EventCodeFilter (ImgAlgos > V00-03-80) module is created by request of Silke Nelson for dark image selection based on EvrData eventCode.
Functionality: Filter selects events which have requested evtcode
in Psana::EvrData::DataV4, 3, ...
data objects in the list of Psana::EvrData::FIFOEvent
objects. Events which do not have the event code are skipped.
parameter | default value | description |
---|---|---|
source | DetInfo(:Evr) | data source |
evtcode | 0 | event code, unsigned number |
| 0 | filter mode: 0-off, 1-on, -1-on in inverted decision mode |
| 0 | verbosity:
|
Example of configuration parameters for this module:
modules = ... ImgAlgos.EventCodeFilter ... [ImgAlgos.EventCodeFilter] evtcode = 41 mode = 1 print_bits = 5 |
Functionality:
source
and key_in
parameters,ndarray<
const
T,2>
object with camera image in the event store using specified key_out
parameter, where the output type T is defined by the outtype
parameter.parameter | default value | description |
---|---|---|
| "DetInfo(:Andor)" | source of data |
|
| key for input data |
| "andorimg" | output key for image saved in the event store |
| "asdata" | out data type: implemented values: asdata (default, uint16_t), float, double, int and int16. |
| 0 | verbosity:
|
Functionality:
source
and key_in
parameters,ndarray<
const
T,2>
object with image in the event store using specified key_out
parameter, where the output type T is defined by the outtype
parameter.parameter | default value | description |
---|---|---|
| "DetInfo(:Epix)" | source of data, it needs to be replaced for real detector to :Epix100a |
|
| key for input data |
| "epix-ndarr" | output key for image saved in the event store |
| "asdata" | out data type: implemented values: asdata (default, uint16_t), float, double, int and int16. |
| 0 | verbosity:
|
See also Example for Module ImgAlgos::EpixNDArrProducer.
Functionality:
source
and key_in
parameters,ndarray<
const
TOUT,3>
object of shape 4x512x512 specified key_out
parameter.parameter | default value | description |
---|---|---|
| "DetInfo(:pnCCD)" | source of data |
|
| key for input data |
| "pnccd-ndarr" | output key for image saved in event |
| "asdata" | out data type: implemented values: asdata (default, unsigned short), float, double, int and int16. |
| 0 | verbosity:
|
Functionality:
ndarray<const T,3>
, where shape=[4][512][512], T=unsigned short, float, double, int, or int16,source
and inkey
parametersndarray<
const
T,2>
object with pnccd [1024+gap_rows][1024+gap_cols] image in the event using specified outimgkey
parameter.parameter | default value | description |
---|---|---|
| "DetInfo(:pnCCD)" | source of data |
|
| key for input data |
| "pnccdimg" | output key for image saved in event |
gap_rows | 0 | gap between top and bottom segments in number of pixels |
gap_cols | 0 | gap between left and right segments in number of pixels |
gap_value | 0 | Image effective pixel intensity value in the gap |
| 0 | verbosity:
|
See also Example for Module ImgAlgos::PnccdImageProducer.
This module works with any generic camera image stored in data type Camera::FrameV#:
source
and key_in
parameters,puts the ndarray<
object with camera image in the event store using specified type
T,2>const
parameters.outtype
and key_out
Special treatment for fccd960: if |
parameter | default value | description |
---|---|---|
| "DetInfo(:Camera)" | source of data |
|
| key for input data |
| "pnccdimg" | output key for image saved in event |
| "asdata" | out data type: implemented values: asdata (default, uint16_t), float, double, int and int16. |
| true | on/off the amplitude offset using configuration data (not applied for |
| 0 | verbosity:
|
See also Example for Module ImgAlgos::CameraImageProducer.
Functionality:
source
and key_in
parameters,ndarray<
const
T,2>
object with camera image in the event using specified type outtype
and key_out
parameters.parameter | default value | description |
---|---|---|
| "DetInfo(:Princeton)" | source of data |
|
| key for input data (raw - by default) |
| "image" | output key for image saved in event |
| "asdata" | out data type: implemented values: asdata (default, uint16_t), float, double, int and int16. |
| 0 | verbosity:
|
See also Example for Module ImgAlgos::PrincetonImageProducer.
Acqiris::ConfigV1
and Acqiris::DataDescV1
, objects using parameters source
and key_in
;ndarray<const double,2>
of shape[] = {nbrChannels, nbrSamples}
for waveforms and time stamps;fname_prefix
;key_wform
and key_wtime
.parameter | default value | description |
---|---|---|
|
| source of data |
|
| key for input data (by default (empty) – raw data) |
|
| output key for waveforms saved in event |
|
| output key for waveform times saved in event |
|
| file name prefix for configuration parameters (by default (empty) – do not save file) |
correct_t | true | on/off switch for time correction; if =false - array indexes are the same as in raw data waveform |
|
| verbosity:
|
Example for Module ImgAlgos::AcqirisArrProducer
ndarray<
const
double,2>
object using parameters source
and key_in
;thresholds
, is_postive_signal
, do_inverse_selection
, in the range depending on local event numbers skip_events
and proc_events;
proc_events
or at the end of job (whatever happens first), saves array of averaged waveforms in the text file with name constructed from fname_ave_prefix
and in the event store using parameters source
and key_average
.
parameter | default value | description |
---|---|---|
|
| Source of data. |
|
| Key for input data (raw - by default). |
key_average | "acq-ave" | Keyword for averaged waveform array saved in the evt store. If empty – array is not saved. |
fname_ave_prefix | "acq-ave" | Text file name prefix for averaged array, full name will be extended by the experiment name, run number and suffix "-ave-wfs.txt" , for example: "acq-amo01509-r0125-ave-wfs.txt". |
thresholds | "" | List of threshold values for all Acqiris channels separated by space. If empty – threshold selection is not applied, all waveforms are averaged. |
is_positive_signal |
| Space-separated list of 1/0 values indicating signal polarity. For example, "1 1 1 0 1" (without the quotes!) would indicate Acqiris channels 1,2,3,5 contained positive signal polarity, while channel 4 contained a negative signal polarity. |
do_inverse_selection |
| Space-separated ist of 1/0 values indicating which waveforms to include in average. For example, "0 0 0 1 0" (without the quotes) would tell the code to average only waveforms that do not cross the threshold for channels 1,2,3,5 while channel 4 would average only waveforms that do cross the threshold. |
skip_events |
| Number of events (from the beginning of job) to skip before begin averaging. |
|
| Number of events for averaging. |
|
| Verbosity:
|
Example for Module ImgAlgos::AcqirisAverage
ndarray<
const
double,2>
object using parameters source
and key_in
;skip_events
and proc_events;
ndarray<
const
double,2>
;ndarray<
const
double,2>
object using parameters source
and key_out.
parameter | default value | description |
---|---|---|
| "DetInfo(:Acqiris)" | Source of data. |
| "acq_wform" | Key for input ndarray with raw waveforms from
|
key_out | "wf-calibrated" | Key for output ndarray with calibrated waveforms. |
fname_base_line | "acq-ave" | Name of the input text file with array of the baselines for active Acqiris channels. By default this name coincides with the name of the file produced by the module |
skip_events |
| Number of events (from the beginning of job) to skip before begin subtraction. |
|
| Number of events for subtraction. |
| 0 | Verbosity:
|
Example for Module ImgAlgos::AcqirisCalib
ndarray<
const
double,2>
object using parameters source,
key_wform, key_wtime
;runs constant-fraction discriminator algorithm on all acqiris channels using user-specified per-channel parameters;
saves edges into the event as a set of ndarray<double,1>;
parameter | default value | description |
---|---|---|
| "DetInfo(:Acqiris)" | Source of data. |
| "acqiris_wform" | Key for input ndarray with waveforms (either raw, or subtracted using AcqirisCalib module) from |
| "acqiris_wtime" | Key for input ndarray with waveform times from |
key_edges | "acqiris_edges_" | Key for output ndarray<double,1> with calibrated waveforms. This key will have the acqiris channel number (1 thru 20) appended to the end of it, and the data for that channel will be added to the event only if edges were found. |
baselines | "" | A list of baseline values (one per channel) to subtract from the waveform in volts. |
fractions | "" | A list of fractions (one per channel) between 0 and 1. The edge-time reported will be at the time when the pulse is at this fraction of the peak value. |
thresholds | "" | A list of threshold values (one per channel) in volts that indicate a new edge should be found. If this value is less than the baseline, then the algorithm will look for negative pulses, otherwise it will look for positive pulses. |
deadtimes | "" | A list of deadtimes (one per channel) in seconds. After each edge the algorithm will ignore any new hits in this time interval. |
leading_edges | "" | A list of 0/1 values (one per channel) indicating whether edge-times are desired for leading edges (1) or falling edges (0). |
Example for Module ImgAlgos::AcqirisCFD
This module converts any (detector-associated) ndarray in to image.
For any available in the event store ndarray<
defined by the const
T,NDim>,source
and key_in
parameters, and which size is equal to the number of pixels in the detector, this module creates 2-d image ndarray<
and saves it in the event store with key_out. TOUT is defined by the type_out. Pixel geometry array is retreived from the calibration file of "geometry" type from default standard or specified in const
TOUT,2>calibdir
directory.
Configuration parameters:
parameter | default value | description |
---|---|---|
| "" | input source of data. This parameter MUST BE specified, for example: CxiDs1.0:Cspad.0 |
| "" | key for input data, for example it might be "calibrated", by default (empty) raw data |
key_out | "image" | key for output image in the event store. |
type_out | "asinp" | type of data in output image array. Possible values - "asinp"-the same type as input array, "int16", "float", "double", "int". |
| 0 | module verbosity:
|
Additional parameters which can be used for special applications like alignment:
parameter | default value | description |
---|---|---|
| "" | non-default calibration directory for "geometry" file |
calibgroup | "" | by-default calibration group is retreived from source name as <detector>::CalibV1 |
oname | "" | top object name in the hierarchial geometry file |
oindex | 0 | top object index in the hierarchial geometry file |
pix_scale_size_um | 0 | set the ize of the image bin. By-default this value is extracted from sensor. |
x0_off_pix | 0 | offset of the top geometry object origin x0 coordinate on the image in number of pixels |
y0_off_pix | 0 | offset of the top geometry object origin y0 coordinate on the image in number of pixels |
mode | 0 | mapping algorithm for overlapping pixels of ndarray on the 2-D image; 0-last pixel substitutes intensity, 1-use pixel with maximal intensity, 2-accumulate(sum) intensity |
do_tilt | true | on/off tilt angle correction for sensors |
See also Example for Module ImgAlgos::NDArrImageProducer.
This module averages over events the per-element data of the image array (ndarray<
, where T is implemented for almost all types: int, int16, uint, float, double etc., NDim≤5) and saves files for sum, averaged, rms values, mask, and, the hot pixel map. Input const
T,NDim>ndarray
can be specified by the source
and key
parameters. Averaging may have up to three stages, depending on configuration parameters:
evts_stage1
, the preliminary averaged and rms values are defined for each pixel at the end of this stage.evts_stage1
the pixel data are collected only for abs(amplitude-average0) < gate_width1
. At the end of this stage the preliminary averaged and rms values are defined for each pixel.evts_stage1 + evts_stage2
the pixel data are collected only for abs(amplitude-average1) < gate_width2
. At the end of this stage the preliminary averaged and rms values are defined for each pixel and saved in the files specified by the avefile
and rmsfile
parameters, respectively.If the parameter thr_rms_ADU
is set ≤0 then threshold value is defined automatically using constrained averaging of pixel rms values in 3 iterations. The threshold value is defined as mean+5*rms
.
parameter | default value | description |
---|---|---|
| DetInfo(:Opal1000) | input source of data |
|
| key for input data, for example, it might be "calibrated" |
| "" | out file with sum of amplitudes, saved if the name is not empty |
| "" | out file with averaged amplitudes, saved if the name is not empty |
| "" | out file with rms, saved if the name is not empty |
maskfile | "" | out file with pixel mask with 0/1-for bad/good pixels, saved if the name is not empty |
| "" | out file with pixel bit-words: 0/1/2/4/8 for good/hot/saturated/cold/cold-rms, saved if the name is not empty |
maxfile | "" | out file with maximal value per pixel over events |
ftype | txt | out file type: txt (default), metatxt, bin |
| 0. | minimal threshold on rms (in ADU); If rms lower than this threshold - pixel is cold-rms. |
| 10000. | maximal threshold on rms (in ADU); =0 - use auto-evaluated threshold. If rms exceeds this threshold - pixel is hot. |
| -100000. | threshold on minimal intensity (in ADU); if ave exceeds this threshold - pixel is good |
| 100000. | threshold on maximal intensity (in ADU); if ave exceeds this threshold - pixel is bad |
| 1000000 | number of events before stage 1 |
| 0 | additional number of events before stage 2 |
| 0 | gate_width for stage 1 |
| 0 | gate_width for stage 2 |
| 0 | module verbosity:
|
If all file names are empty (by default), the files with pre-defined names "arr-ave-<exp>-r<run>.dat" and "arr-rms-<exp>-r<run>.dat" will be saved for averaged and rms arrays, respectively. Otherwise, the files with specified names will be saved.
Default parameters are set for regular single-stage averaging without any constrains.
See also Example for Module ImgAlgos::NDArrAverage.
Functionality
source
and key_in
parameters to get any ndarray<
const
T,NDIM>
object from the event store, where T
stands forint16_t,
uint16_t, int, float, uint8_t,
or double, 1≤NDim≤5
,pedestals, common_mode, pixel_status, pixel_gain,
and pixel_rms,
fname_bkgd
and fname_mask
, if their names are specified,the specified by the do_...
parameter corrections are applied to raw data ndarray<
as follows:const
T,NDIM>
const TOUT,NDim
> in the event with key key_out
, where TOUT
is controlled by the parameter outtype,
which can be set to double
(default), float
, int,
and int16
.
Control on corrections
A_cor = A_raw (1) - pedestal | if do_peds==true and pedestals are available in calib store (2) - common mode | if do_cmod==true and common_mode parameters are available in calib store (3) - N*background | if do_bkgd==true and the file name is specified in the parameter fname_bkgd and bkgd_ind_* are set (4) * gain | if do_gain==true and pixel_gain are available in calib store (5) apply bad pixel status | if do_stat==true and pixel_status are available in calib store (6) apply mask | if do_mask==true and the file name is specified in the parameter fname_mask. Parameter masked_value is used to substitute masked values. (7) apply N*RMS threshold | if do_nrms==true and pixel_rms are available in calib store, parameters threshold_nrms and below_thre_value are set (8) apply common threshold | if do_thre==true. Parameter below_thre_value is used to substitute below threshold values. |
Configuration parameters
parameter | default value | description |
---|---|---|
| DetInfo(:Camera) | source of data |
|
| key for input |
| calibrated | output key for calibrated image saved in event |
outtype | double | output ndarray data type can be set to double (default), float , int, and int16 . |
| false | true: pedestals subtracted if available in calib store |
| false | true: common mode correction is evaluated and applied [Ref.] |
| false | true: bad/hot pixels in pixel_status are masked |
| false | true: mask is applied if the file |
| false | true: normalized background is subtracted if the file |
| false | true: |
| false | true: per-pixel threshold is applied if |
| false | true: low level |
|
| input file name for background, applied if the file name is specified |
|
| input file name for mask, applied if the file name is specified |
| 0. | intensity value (in ADU) substituted for masked pixels |
| 3. | threshold as a number of sigmas to |
| 0. | common low level threshold in ADU |
| 0. | intensity substituted for pixels below threshold |
| 0 | minimal index in flatten ndarray, which is used for background normalization |
| 100 | maximal index in flatten ndarray, which is used for background normalization |
| 2 | index increment in flatten ndarray, which is used for background normalization |
| 0 | verbosity:
|
To add implementation for any other new detector "Det":
1) add file pdscalibdata/DetBaseV1.h 2) in PSCalib/src/GenericCalibPars.cpp add #include "pdscalibdata/DetBaseV1.h" ... template class PSCalib::GenericCalibPars<pdscalibdata::Opal4000BaseV1>; 3) in PSCalib/include/CalibParsStore.h add #include "pdscalibdata/DetBaseV1.h" ... return new PSCalib::GenericCalibPars<pdscalibdata::DetBaseV1>(calibdir, type_group, src, runnum, prbits); |
Doxygen documentation for interface: CalibParsStore, GenericCalibPars
See also Example for Module ImgAlgos::NDArrCalib,
Test of the NDArrCalib module for pnCCD.
Finds "droplets" (wide peaks) in data ndarray and saves their list in output ndarray. This is a re-implementation of algorithm ImgPeakFinder for ndarray of 2-d segments with functionality as follows.
1. Selects pixels in the windows defined by the list of parameters: segment, rowmin
, rowmax
, colmin
, colmax
with amplitudes above the threshold_low
.
2. Optionally smears image for selected pixels using 2-d matrix of weights in the range -smear_radius
to +smear_radius
in both dimensions around smeared pixel. The matrix of weights is defined by the 2-d Gaussian function of width sigma
. If sigma=0
smearing is not applied.
3. Finds peaks as pixels with absolute-maximal amplitude above the threshold_high
in the center of the matrix covering the range -peak_radius
to +peak_radius
in both dimensions.
4. Puts the ndarray of found peaks in the event store with key
. Each row of this ndarray has parameters from key_droplets
struct Droplet
, containing segment number, row, column of the droplet center, peak pixel amplitude, the total amplitude inside peak_radius
the region, and the number of pixels in this region above threshold_low
:
struct Droplet{ unsigned seg; double row; double col; double ampmax; // amplitude in the peak maximum double amptot; // total amplitude in the range of peak_radius unsigned npix; // number of pixels in the range of peak_radius }; |
parameter | default value | description |
---|---|---|
| "DetInfo()" | source of data |
|
| key for input data ndarray, default is empty - raw data |
| key for output list of peaks as std::vector<AlgDroplet::Droplet> (default is empty - do not save) | |
key_smeared | ||
| 10 | low threshold on pixel amplitude |
| 100 | high threshold on pixel amplitude |
| 1.5 | width of the Gaussian for smearing; =0-no smearing |
| 3 | radius in pixel for smearing - radial size of matrix of weights |
| 3 | radius in pixel for peak finding - radial size of the region to search for local maximum |
low_value | 0 | value substituted for pixels with intensity below threshold and outside window |
|
| list of windows, each window is defined by 5 parameters; |
mask | path to the file with mask, by default (empty) mask is not used | |
masked_value | 0 | value substituted for masked pixels (0-masked, non-zero - good pixel) |
|
| Common prefix for saved files. If non-empty - save files with image and list of droplets for each event with found droplets. The file name is formed as <prefix>-r####-e######-<suffix>.txt, where hash stands for number (0-9), suffix may be |
| 0 | module verbosity:
|
Remarks:
print_bits
is not zero - warning messages will be printed.peaks_droplets
is non-empty.sigma
is not zero) or not.See also Example for Module ImgAlgos::NDArrDropletFinder
Functionality
For data source
in each run loads/updates calibration geometry
file from the calibration DB, evaluates pixel coordinate, area, mask arrays using class PSCalib::GeometryAccess and saves them as ndarray<const TYPE,1> in the env.calibStore()
for keys (TYPE=float) key_out_x
, key_out_y
, key_out_z
, key_out_area
(unsigned),
key_out_ix, key_out_area
(int), key_out_iy
,
(TYPE=uint8_t)key_gfname
(string) and key_fname
.
The main idea of this module is that calibration geometry
file will be found and loaded (if available) automatically and pixel coordinate, index, and other arrays will be produced and saved in the env.calibStore()
, if appropriate keys are not empty.
Configuration parameters
parameter | default value | description |
---|---|---|
|
| source of data |
|
| group of calibration type, by default will be set from source |
|
| output key pixel x-coordinate[um] array. If set empty - array is not saved in the env.calibStore. |
|
| output key pixel y-coordinate[um] array. If set empty - array is not saved in the env.calibStore. |
|
| output key pixel z-coordinate[um] array. If set empty - array is not saved in the env.calibStore. |
|
| output key pixel area array, default value is empty - array is not saved in the env.calibStore |
|
| output key pixel mask array, default value is empty - array is not saved in the env.calibStore |
|
| output key pixel x-coordinate index array, default value is empty - array is not saved in the cenv.alibStore |
|
| output key pixel y-coordinate index array, default value is empty - array is not saved in the env.calibStore |
key_fname | geometry-calib | path to "geometry" calibration file saved as ndarray<char,1> DEPRICATED! USE key_gfname |
key_gfname | geometry-fname | path to "geometry" calibration file saved as std::string |
x0_off_pix | 0 | offset of detector origin x0 in number of pixels before evaluation of indexes, by default offset moves xmin to 0. |
y0_off_pix | 0 | offset of detector origin y0 in number of pixels before evaluation of indexes, by default offset moves ymin to 0. |
| 255 | mask control bits. For cspad2x1 sensor mask control bits mean:
|
| 0 | verbosity:
|
Current version of this module works with CSPAD and CSPAD2x2. It can be extended for other detectors, whenever necessary.
See Example for Module ImgAlgos::PixCoordsProducer
This module averages over events the per-pixel data of the image array (ndarray<const
double,2>) and saves files for averaged, rms values, and, if requested, the hot pixel mask. Input data can be specified by the source
and key
parameters. Averaging may have up to three stages, depending on configuration parameters:
evts_stage1
, the preliminary averaged and rms values are defined for each pixel at the end of this stage.evts_stage1
the pixel data are collected only for abs(amplitude-average0) < gate_width1
. At the end of this stage the preliminary averaged and rms values are defined for each pixel.evts_stage1 + evts_stage2
the pixel data are collected only for abs(amplitude-average1) < gate_width2
. At the end of this stage the preliminary averaged and rms values are defined for each pixel and saved in the files specified by the avefile
and rmsfile
parameters, respectively.parameter | default value | description |
---|---|---|
| DetInfo(:Opal1000) | input source of data |
|
| key for input data, for example, it might be "calibrated" |
| "" | out file with sum of amplitudes, saved if the name is not empty |
| "" | out file with averaged amplitudes, saved if the name is not empty |
| "" | out file with rms, saved if the name is not empty |
| "" | out file with hot pixel mask, saved if the name is not empty |
| 10000. | threshold on rms (in ADU); if rms exceeds this threshold - pixel is hot |
| 1000000 | number of events before stage 1 |
| 0 | additional number of events before stage 2 |
| 0 | gate_width for stage 1 |
| 0 | gate_width for stage 2 |
| 0 | module verbosity:
|
If all file names are empty (by default), the files with pre-defined names "img-ave-r####.dat" and "img-rms-r####.dat" (where #### stands for run number) will be saved for averaged and rms images, respectively. Otherwise, the files with specified names will be saved.
Default parameters are set for regular single-stage averaging without any constrains.
See also Example for Module ImgAlgos::ImgAverage.
This module gets the image data array (ndarray<const
T,2>), where T stands for double, float, int, uint8_t, or uint16_t, and evaluates two masks:
In the endJob
this module saves files (if their names are provided) of image size for:
The S/N is estimated by averaging over neigbouring pixels.
parameter | default value | description |
---|---|---|
| "DetInfo(:Opal1000") | input source of data |
|
| key for input data, for example, it might be "calibrated" |
| "img-mask-satu.dat" | out file with saturated mask |
| "img-mask-nois.dat" | out file with noise mask |
| "img-mask-comb.dat" | out file with combined mask |
| "img-frac-satu.dat" | out file with fraction of saturated events |
| "img-frac-nois.dat" | out file with fraction of noisy events |
| 1000000 | saturation-threshold (in ADU if the gain correction was not applied to image) |
| 0 | allowed fraction of saturated events |
| 1 | radial size of the area for S/N evaluation |
| 3 | noise-threshold ( in ADU if the gain correction was not applied to image) |
| 0.5 | allowed fraction of noisy events |
| 0 | module verbosity:
|
See also Example for Module ImgAlgos::ImgMaskEvaluation.
ImgCalib gets the raw image from data and process it as follows:
Functionality:
source
and key_in
parameters to get the input raw image (as ndarray<
const
T,2>
object), where T stands for int16_t(from V00-03-46), uint16_t, int, float, uint8_t, or double,fname_peds
, fname_bkgd
, fname_gain
, fname_mask
, and fname_rms
, if their names are specified,the specified by the file name corrections are applied per-pixel to raw data image as follows:
A_cor = A_raw (1) - pedestal | if the file name is specified in the parameter "fname_peds" (2) - N*background | if the file name is specified in the parameter "fname_bkgd" (3) * gain | if the file name is specified in the parameter "fname_gain" (4) apply mask | if the file name is specified in the parameter "fname_mask" (5) apply N*RMS threshold | if the file name is specified in the parameter "fname_nrms" (6) apply threshold | if the "do_threshold" = true |
key_out
as double
type.Details:
bkgd_row_min
, bkgd_row_max
, bkgd_col_min
, and bkgd_col_max
. Normalization factor N
is evaluated for pixel amplitudes in this window as:N = sum(A_raw - pedestal) / sum(A_bkgd)
.fname_mask
file should be marked by '1' (ones) and bad pixels – by '0' (zeros). Mask is applied as a last correction. The bad masked pixel amplitudes are substituted by the masked_value
.parameter | default value | description |
---|---|---|
| "DetInfo(:Camera)" | source of data |
|
| key for input image |
| "calibrated" | output key for calibrated image saved in event |
|
| input file name for pedestals, applied if the file name is specified |
|
| input file name for background, applied if the file name is specified |
|
| input file name for gain, applied if the file name is specified |
|
| input file name for mask, applied if the file name is specified |
|
| input file name for RMS, applied if the file name is specified |
| 0. | amplitude value (in ADU) substituted for masked pixels |
| 3. | threshold as a number of sigmas from file |
| 0. | constant low level threshold in ADU |
| 0. | the amplitude substituted in pixels below threshold |
| false | if true - low level |
| 0 | the window in image, which is used for background normalization |
| 10 | the window in image, which is used for background normalization |
| 0 | the window in image, which is used for background normalization |
| 10 | the window in image, which is used for background normalization |
| 0 | verbosity:
|
The pedestal, background, gain, mask, and N*RMS corrections are applied if associated file name is specified.
The constant low level threshold is applied if do_threshold
is true.
Corrections are not applied by default or with empty file names.
See also Example for Module ImgAlgos::ImgCalib.
Functionality:
source
and key_in
parameters to get the input image (as ndarray<
const
T,2>
object), where T stands for uint16_t, int, float, uint8_t, or double,fname_map_bins
, which has a size of image,number_of_bins
bin (from 0 to number_of_bins-1
),fname_int_bins
.parameter | default value | description |
---|---|---|
| "DetInfo(:Camera)" | source of data |
|
| key for input image |
|
| input file name for map of bin indexes |
|
| output file name for intensity(bin,event) |
| 10 | number of bins for output intensities, if map has bins \>= |
| 0 | verbosity:
|
This module averages the image pixel amplitude in r-phi bins, normalizes it per single pixel and subtracts the average amplitude from each pixel. Image is obtained from event by its source
and inkey
values. The output corrected image is saved in the event with outkey
keyword. Carthesian to polar coordinate transformation is done with respect to center coordinates xcenter
, ycenter
. The central region (r<rmin) and edges of the image (r>rmax) pixels can be removed from further consideration by setting rmin
and rmax
. The number of radial bins is defined as an int(rmax-rmin)
. The number of angular bins is set by n_phi_bins
.
parameter | default value | description |
---|---|---|
| "DetInfo()" | input source of data |
|
| key for input data, by default use raw data |
| "rad_corrected" | output key for further image processing |
| 850 | x coordinate of the image center |
| 850 | y coordinate of the image center |
| 10 | radius minimal image is not processed for r<rmin |
| 1000 | radius maximal image is not processed for r>rmax |
| 12 | number of angular sectors for the background averaging |
| 0 | test event for print/save |
| 0 | module verbosity:
|
See also Example for Module ImgAlgos::ImgRadialCorrection.
The ImgAlgos::ImgPixAmpFilter is a filter for event selection.
This filter counts the number of image pixels in the specified window with amplitude exceeding the threshold
. If the number of high-amplitude pixels exceed the numPixMin
, the event is passed for further analysis.
The algorithm performance was tested for CSPad images. This algorithm consumes up to 15 ms/event on psana0205 for full CSPad (1650x1650) window size. For smaller window consumed time is negligible, comparing to the image reconstruction time, which is ~70 ms/event (for cspad_mod.CsPadCalib
and CSPadPixCoords::CSPadImageProducer
) on psana0205.
parameter | default value | description |
---|---|---|
| "DetInfo(:Cspad)" | source of data |
| "Image2D" | key for input image data |
| 10 | minimal threshold on pixel amplitude |
| 100 | minimal number of pixels with amplitude above the threshold |
| true | On/Off the filter |
| 0 | minimal column number |
| 100000 | maximal column number |
| 0 | minimal row number |
| 100000 | maximal row number |
| 0 | verbosity:
|
Remarks:
key
("Image2D") stands for the CSPadPixCoords::Image2D<double>
image object. Other key
names work for the ndarray<
const
double,2>
image object.xmin
, xmax
, ymin
, ymax
(in pixels) defines the window in the image for pixel counting. Default values means the entire image range.threshold
, numPixMin
, and the window extents have to be adjusted for particular experiment.This algorithm was motivated by users of amo42112:
1. Select the pixels in the window xmin
, xmax
, ymin
, ymax
with amplitudes above the threshold_low
.
2. Optionally smears image for selected pixels, using 2-d matrix of weights over pixels from -smear_radius
to +smear_radius
around each smeared pixel amplitude. The matrix of weights is defined by the 2-d Gaussian function of width sigma
. If sigma=0
smearing is not applied.
3. Find peaks as pixels with absolute-maximal amplitude above the threshold_high
in the center of the matrix -peak_radius
to +peak_radius
.
4. Put the vector or ndarray of found peaks in the event with key peaksKey
or peaks_nda
respectively. Each entry of this vector has an object of the struct Peak
, containing x, y positions, peak pixel amplitude, the total amplitude in the matrix, defined by the peak_radius
, and the number of pixels in the matrix above threshold_low
:
struct Peak{ double x; double y; double ampmax; // amplitude in the peak maximum double amptot; // total amplitude in the range of {{peak_radius}} unsigned npix; // number of pixels in the range of {{peak_radius}} } ; |
parameter | default value | description |
---|---|---|
| "DetInfo()" | source of data |
|
| key for input image data |
| "peaks" | key for output list of peaks as std::vector<Peak> (if empty - do not save) |
peaks_nda | key for output list of peaks ndarray<float,2> with shape=[npeaks,5]. Is empty by default (if empty - do not save) | |
| 10 | low threshold on pixel amplitude |
| 100 | high threshold on pixel amplitude |
| 1.5 | width of the Gaussian for smearing; =0-no smearing |
| 3 | radius in pixel for smearing - radial size of matrix of weights |
| 3 | radius in pixel for peak finding - radial size of the region to search for local maximum |
| 0 | minimal column number |
| 100000 | maximal column number |
| 0 | minimal row number |
| 100000 | maximal row number |
| 0 | event number to save images and print info for test purpose |
| true | On/Off algorithm |
| 0 | module verbosity:
|
Remarks:
peaks_nda
is non-empty.See also Example for Module ImgAlgos::ImgPeakFinder.
This module use results and should work after the ImgAlgos::ImgPeakFinder. It gets the vector of peaks for the source
and key
, loops over all founded peaks and counts the number of peaks above the thresholds threshold_peak
and threshold_total
. If the selection_mode
is "SELECTION_ON" and the number of found peaks exceeds the n_peaks_min
the event is passed for further analysis/processing,
the table of found peaks may be saved in file with prefix defined by the fname
parameter.
parameter | default value | description |
---|---|---|
| "DetInfo()" | source of data |
| "peaks" | key for input list of peaks, should be the same as |
| "SELECTION_ON" | three possible options:
|
| 0 | threshold on peak amplitude |
| 0 | threshold on total peak intensity (in the matrix around peak) |
| 1 | minimal number of peaks above all thresholds |
|
| file name prefix; by default the prefix is empty and file is not saved |
| 0 | filter verbosity:
|
See also Example for Module ImgAlgos::ImgPeakFilter.
This module finds peaks in the ndarray<
image object. Algorithm and the list of parameters are the same as described is section Module ImgAlgos::CSPadArrPeakFinder. The only difference is that the image size is defined by the const
double,2>ndarray<
object.const
double,2>
See also Example for Module ImgAlgos::ImgPeakFinderAB.
ImgHitFinder is created by request for amo74213. It works pretty similar to ImgCalib, but the threshold algorithms are extended and background subtraction is removed.
It gets the raw image from data and process it as follows:
Functionality:
source
and key_in
parameters to get the input raw image (as ndarray<
const
T,2>
object), where T stands for uint16_t, int, float, uint8_t, or double,fname_peds
, fname_gain
, fname_mask
, and fname_thre
, if their names are specified,the specified by the file name corrections are applied per-pixel to raw data image as follows:
A_cor = A_raw (1) - pedestal | if the file name is specified in the parameter "fname_peds" (2) * gain | if the file name is specified in the parameter "fname_gain" (3) apply mask | if the file name is specified in the parameter "fname_mask" (4) apply threshold | if the file name is specified in the parameter "fname_thre" |
key_out
as double
(or unsigned for pixel map) type.Details:
fname_mask
file should be marked by '1' (ones) and bad pixels – by '0' (zeros). The bad masked pixel amplitudes are substituted by the masked_value
.parameter | default value | description |
---|---|---|
| "DetInfo(:Camera)" | source of data |
|
| key for input image |
| "img-hits" | output key for calibrated image saved in event |
|
| input file name for pedestals, applied if the file name is specified |
|
| input file name for mask, applied if the file name is specified |
|
| input file name for gain, applied if the file name is specified |
|
| input file name for threshold, applied if the file name is specified and |
| 0. | amplitude value (in ADU) substituted for masked pixels |
| 1 | threshold mode: 0 - is not applied; 1 - constant level threshold defined by |
| def | threshold parameter - its meaning depends on |
| 0. | the amplitude substituted in pixels below threshold |
| def | the amplitude substituted in pixels below threshold, by default - amplitude above threshold is not substituted |
| 1 | the window in image, which is used to make hit map |
| def | the window in image, which is used to make hit map, by default - use all rows-1 |
| 1 | the window in image, which is used to make hit map |
| def | the window in image, which is used to make hit map, by default - use all columns-1 |
| 0 | verbosity:
|
Corrections are not applied by default or with empty file names.
See also Example for Module ImgAlgos::ImgHitFinder.
This module is motivated by the discussion with Josef Frisch, Ryan Coffee, Nick Hartmann. In xppi0412 etc. experiments they need to extract two spectra from Opal1000 camera image for signal and reference, evaluate their relative difference, and find peak position in the differential spectrum.
Module mgAlgos::ImgSpectra
works as follows:
ndarray<
const
double,2>
image object from event,ndarray<
const
double,2>
object with shape (3,<number-of-columns>) in the event.parameter | default value | description |
---|---|---|
| "DetInfo(:Opal1000)" | source of data |
| "img" | key for input image data |
| "spectra" | key for output array with spectra |
| 510 | signal band central row at column=0 |
| 550 | reference ... |
| 0 | signal band tilt angle |
| 0 | reference ... |
| 10 | signal band width in number op rows |
| 10 | reference ... |
| 0 | module verbosity:
|
See also Example for Module ImgAlgos::ImgSpectra.
Works after module ImgAlgos::ImgSpectra.
This module is designed as an example, in order to show how to get access to spectral array.
In particular, method ImgSpectraProc::getSpectra(...)
shows how to get pointer to data, and method ImgSpectraProc::printSpectra(...)
iterates over array and selectively prints its elements.
parameter | default value | description |
---|---|---|
| "DetInfo(:Opal1000)" | source of data |
| "spectra" | key for input image data |
| 0 | module verbosity:
|
See also Example for Module ImgAlgos::ImgSpectraProc.
Aka depricated modules: CSPadPixCoords::CSPadImageGetTest, CSPadPixCoords::SaveImageInFile, and ImgAlgos::SaveImageInFile. |
Module ImgSaveInFile receives from the event the image object using source
and key
parameters and saves it in the ftype
format with prefix file name fname
for event(s) specified by the parameters eventSave
or saveAll
. Currently implemented file formats: txt
, bin
, tiff
, and png
.
ImgSaveInFile works after the CSPadImageProducer, CSPadInterpolImageProducer, CameraImageProducer, PnccdImageProducer, etc., which produce image object in formats CSPadPixCoords::Image2D<T>
or ndarray<
, where the const
T,2>T
stands for one of the data types, double, float, int, uint8_t, or uint16_t.
parameter | default value | description |
---|---|---|
| "CxiDs1.0:Cspad.0" | source of data |
| "Image2D" | input image key |
| 0 | event number to save the CSPad image file |
| false | save or not all selected events |
| "txt" | output file format; currently implemented formats |
| "img" | prefix of the output file name. Entire file name is formed as |
| 0 | verbosity:
|
Saving in PNG currently works for uint8_t and uint16_t formats only... |
See also Example for Module ImgAlgos::ImgSaveInFile
This module is a part of complex algorithm, described in Command Line Interface For Time Correlation Analysis.
This module is designed for parallel image processing for correlation analysis.
Functionality:
ndarray<
const
T,2>
object,nfiles_out
equal parts,<fname_prefix>-<fname-common>-b<block-number>.<file_type>
,<fname_prefix>-<fname-common>-med.txt
,<fname_prefix>-<fname-common>-time.txt
.parameter | default value | description |
---|---|---|
| "DetInfo(:Princeton)" | source of data |
| "img" | input image key |
| "my-exp" | output files common prefix |
| "bin" | output files type: "bin", "txt" |
| true | add time stamp to the output file names |
| 8 | number of output files (or number of parts to split image), it works tested and works for |
| 1 | pixel amplitude threshold in image amplitude units |
| 1 | default pixel amplitude if it is below |
| 0 | verbosity:
|
See also Example for Module ImgAlgos::ImgVsTimeSplitInFiles.
Further processing of the files
<fname_prefix>-<fname-common>-b<block-number>.<file_type>
<fname_prefix>-<fname-common>-med.txt
<fname_prefix>-<fname-common>-time.txt
is implemented in stand-alone c++ module
ImgAlgos/app/corana.cpp (or ImgAlgos/test/corana.cpp) |
Note, the application in the |
where
<fname_data>
is one of the data files: <fname_prefix>-<fname-common>-b<block-number>.<file_type>
, which needs to be available;<fname_tau>
is a file with a list of indexes of tau for evaluation of correlations. By default or if the file is missing, the list of indexes will be generated automatically, and for book-keeping is saved in the file <fname_prefix>-<fname-common>-tau.txt
;<basedir>
is a directory for all data files, which is current by default;<logfile>
is an output log-file, or standard output by default.This module is reduced from ImgAlgos::ImgVsTimeSplitInFiles.
Functionality is restricted to:
fname
.print summary parameters for parser, for example:
ImgTimeStampList: Summary for parser BATCH_RUN_NUMBER 0020 BATCH_NUMBER_OF_EVENTS 75 BATCH_FRAME_TIME_INTERVAL_AVE 8.086934 BATCH_FRAME_TIME_INTERVAL_RMS 0.120381 BATCH_FRAME_TIME_INDEX_MAX 74 |
parameter | default value | description |
---|---|---|
| "tstamp-list.txt" | output files common prefix |
| 0 | verbosity:
|
ImgAlgos::UsdUsbEncoderFilter (ImgAlgos > V00-03-43) module is created by request of Thomas Kroll for experiment with mobile rack in SACLA .
This psana module contains an example of how to get UsdUsb::DataV1 object and time-stamps from different data sources for event synchronization purpose.
Functionality:
To work with time-code objects it uses helper class TimeCode. It loads input timing information from file defined by the ifname
and store it in the std::vector<TimeCode> member object. For each event the TimeCode object is defined from data; time stamp from PSEvt::EventId, and unique code from UsdUsb::DataV1. TheTimeCode object from data is compared with information loaded from the input file. If the TimeCode object from data is consistent with one of the records in the file event is passed for further processing, otherwise discarded.
parameter | default value | description |
---|---|---|
| "DetInfo(:USDUSB)" | source of data |
| 1 | filter mode: 0-off, 1-on, -1-on in inverted decision mode |
| input file name, is empty by default | |
|
| output file name, is empty by default |
bitmask | 63 | bitmask on UsdUsb code, by default use 6 bits |
| 0 | verbosity:
|
Input (ifname
) and output (ofname
) files have any number of records of the same format. Each record consists of four integer numbers:
Example of the input/output file content:
1373280273 293197261 51 1 1373280273 301532011 52 2 1373280273 309867156 54 3 1373280273 318206221 58 4 1373280273 326553079 61 5 ... |
See also Example for Module ImgAlgos::UsdUsbEncoderFilter.
This module is intended for CorAna
project.
ImgIntMonCorr gets the image and intensity monitor data, evaluate the normalization factor, applies this factor to the image intensity, and saves the corrected image in the event.
Functionality:
source
and key_in
parameters to get the input image (as ndarray<
const
double,2>
object),fname_imon_cfg
,key_out
. The type of output data is the same as the type of input data.parameter | default value | description |
---|---|---|
| "DetInfo(:Camera)" | source of data |
| "calibrated" | key for input image |
| "imon_corrected" | key for output calibrated image saved in event |
|
| input file name with intensity monitors' configuration |
| 0 | verbosity:
|
fname_imon_cfg
file content per line: source name, short name, on/off bits for 4-channels, normalization and selection, minimal, maximal and averaged intensities:
BldInfo(FEEGasDetEnergy) FEEGasDetEnergy 1 1 1 1 0 1 3.000000 7.500000 5.250000 BldInfo(XCS-IPM-02) XCS-IPM-02 1 1 1 0 1 0 10.000000 13.000000 11.500000 BldInfo(XCS-IPM-mono) XCS-IPM-mono 1 1 1 0 0 1 14.500000 16.000000 15.250000 DetInfo(XcsBeamline.1:Ipimb.4) Ipimb.4 1 1 1 1 0 0 -1.000000 -1.000000 1.000000 DetInfo(XcsBeamline.1:Ipimb.5) Ipimb.5 1 1 1 1 0 0 -1.000000 -1.000000 1.000000 |
This module is intended for CorAna
project.
It gets the 5 intensity monitors data (4 channels for each) and saves them in the text or binary file file_data
. Comments (or header) for this file is saved separately in file_header
. It also prints the summary parameters for parser, for example:
IntensityMonitorsData: Summary for parser BATCH_RUN_NUMBER 0020 BATCH_NUMBER_OF_EVENTS 75 |
parameter | default value | description |
---|---|---|
| "BldInfo(FEEGasDetEnergy)" | source name for intensity monitor |
| "BldInfo(XCS-IPM-02)" | source name for intensity monitor |
| "BldInfo(XCS-IPM-mono)" | source name for intensity monitor |
| "DetInfo(XcsBeamline.1:Ipimb.4)" | source name for intensity monitor |
| "DetInfo(XcsBeamline.1:Ipimb.5)" | source name for intensity monitor |
| "txt" | file type may be "txt" or "bin" |
| "intensity-monitor-data.txt" | file name for data from intensity monitors |
| "intensity-monitor-comments.txt" | file name for comments |
| 0 | verbosity:
|
This module saves the CSPad data array formatted as [5920=4*8*185][388] in output file for each passes event.
parameter | default value | description |
---|---|---|
| "DetInfo(:Cspad)" | input source of data |
|
| key for input data, for example, it might be "calibrated" |
| "cspad-arr" | out file name prefix for saved array |
| 0 | module verbosity:
|
This module averages the CSPad data array and saves two files for averaged and rms values in CSPad format [5920=4*8*185][388]. In contrast to the cspad_mod.CsPadPedestals, the input data can be specified with a key
, that allows to average CSPad array for already pre-processed data, for example "calibrated"
. This feature can be used to evaluate the averaged signal or background event. Implemented algorithm of averaging allows to eliminate large statistical fluctuations in the pixel amplitude spectrum. In advanced case averaging may have up to three stages, depending on configuration parameters:
evts_stage1
is averaged without any constrains, the preliminary averaged and rms values are defined for each pixelevts_stage1
data are collected only for abs(amplitude-average0) < gate_width1
. At the end of this stage the preliminary averaged and rms values are defined for each pixel.evts_stage1 + evts_stage2
data are collected only for abs(amplitude-average1) < gate_width2
. At the end of this stage the preliminary averaged and rms values are defined for each pixel and saved in the files specified by the avefile
and rmsfile
parameters, respectively.This type of averaging algorithm may be useful for pedestal defenition in case of large amplitude fluctuations.
parameter | default value | description |
---|---|---|
| "DetInfo(:Cspad)" | input source of data |
|
| key for input data, for example, it might be "calibrated" |
| "cspad-ave.dat" | out file with averaged amplitudes |
| "cspad-rms.dat" | out file with rms |
| 1<<31U | number of events before stage 1 |
| 100 | additional number of events before stage 2 |
| 0 | gate_width for stage 1 |
| 0 | gate_width for stage 2 |
| 0 | module verbosity:
|
Default version of the configuration parameters works the same way as cspad_mod.CsPadPedestals. In this case module gets raw events and stage 0 continues for entire input data sample.
See also Example for Module ImgAlgos::CSPadArrAverage.
Alternative to the cspad_mod.CsPadCalib
algorithm for the common mode correction.
Takes the CSPad data array with subtracted pedestals, evaluate the average amplitude for each 2x1 section for amplitudes below the threshold, and subtract it from all pixel amplitudes. This algorithm consumes about 30 ms/event on psana0205.
parameter | default value | description |
---|---|---|
| "DetInfo(:Cspad)" | source of data for CSPad |
| "calibrated" | key for input data, by default expects data from |
| "cm_subtracted" | output key for the next data processing stage |
| 30 | threshold to exclude the large pixel amplitudes in average |
This module uses the CSPad array, specified by the configuration parameters source
and inputKey
, subtracts the background, defined in the file bkgd_fname
, and saves the resulting array in the event with outputKey
. The subtracted background array is normalized on the sum of pixel amplitudes in the quad section norm_sector
, which can be set from 0 to 7.
The shape of the CSPad array in the file
|
parameter | default value | description |
---|---|---|
| "DetInfo(:Cspad)" | input source of data |
|
| key for input data, by default use raw data |
| "bkgd_subtracted" | output key for the next data processing stage |
| "cspad_background.dat" | file with CSPad array [4*8*185][388] of averaged background |
| 0 | CSPad sector in quad from 0 to 7 |
| 0 | module verbosity:
|
See also Example for Module ImgAlgos::CSPadBkgdSubtract.
This module uses the CSPad array, defined by the configuration parameters source
and inkey
, apply the mask from file mask_fnname
and saves the masked data with key outkey
. For masked pixels the amplitude will be replaced by the value from masked_amp
.
The file mask_fnname
has the same structure as files for pedestals and background with dimensions [4*8*185][388]. Masked pixels are indicated by 0-th in this file. This file can be generated, for example, from the averaged background file, using amplitude threshold. This can be done with auxiliary python script MakePixelMask.py
as explained in Example for Module ImgAlgos::CSPadMaskApply.
parameter | default value | description |
---|---|---|
| "DetInfo(:Cspad)" | input source of data |
|
| key for input data, by default use raw data |
| "bkgd_subtracted" | output key for the next data processing stage |
| "cspad_mask.dat" | file with CSPad mask array [4*8*185][388] |
| 0 | this amplitude will replace the amplitude in each masked pixel |
| 1 | control bits for applied mask:
|
| 0 | module verbosity:
|
This module works on CSPad data array shaped as [5920=4*8*185][388], uses the "median algorithm" to evaluate the signal and noise for each pixel, evaluates S/N ratio for each pixel, counts the fraction of events where S/N > SoNThr
, and writes the same shape arrays for pixel mask and status information in the maskfile
and statusfile
, respectively. The statusfile
contains for each pixel the fraction of events where S/N > SoNThr
. This module presents a part of features implemented in the module ImgAlgos::CSPadArrPeakFinder.
parameter | default value | description |
---|---|---|
| "DetInfo(:Cspad)" | input source of data |
|
| key for input data, for example, it might be "calibrated" |
| "cspad-pix-status.dat" | out file with pixel status: fraction of noisy images |
| "cspad-pix-mask.dat" | out file with pixel mask |
| 3 | radial parameter of the area for median algorithm |
| 1 | radial band width of the area for median algorithm |
| 3 | S/N threshold for each pixel to be considered as noisy |
| 0.1 | fraction of noisy images above which pixel is masked in the |
| 0 | module verbosity:
|
See also Example for Module ImgAlgos::CSPadArrNoise.
Module ImgAlgos::CSPadArrPeakFinder
is a psana-based implementation of the "median algorithm" for peak finding in CSPad data array shaped as [5920=4*8*185][388]. This algorithm was first implemented in myana/Cheetah by Anton Barty and Co. The "median algorithm" assumes that the amplitude level of background and noise for each pixel can be estimated as a mean and RMS of the surrounding pixels, located in the ring with parameters rmin
and dr
around the pixel in question. The threshold SoNThr_noise
on signal over noise (S/N) ratio allow to asset the pixel amplitude as a large noise fluctuation. Statistics of pixels above the S/N threshold accumulated over many images can be used to form the noisy-pixel mask. For example, if the fraction of images where pixel exceeds the S/N threshold grater than certain value (frac_noisy_imgs
=0.9), the pixel is considered as noisy. The permanent bad pixel mask (see module ImgAlgos::CSPadMaskApply) and dynamically evaluated noisy pixel mask are used to get rid of bad pixels and improve the image quality. Healthy pixels with S/N above threshold (SoNThr_signal
about 3-5) are treated as potential signals. Using recursive flood-filling algorithm the groups of connected signal pixels can be found and considered as a candidate for a diffraction peaks. Peak finding algorithm uses the amplitude, S/N thresholds, and limits on number of pixels in the connected region (parameters peak_amp_tot_thr
, peak_SoN_thr
, peak_npix_min
, and peak_npix_max
) in order to define the peak. Finally, the event is selected or discarded depending on number of found peaks and total amplitude threshold, defined by the parameters event_npeak_min
, event_npeak_max
, and event_amp_tot_thr
, respectively.
Description of implemented algorithm:
beginJob(...)
method:hot_pix_mask_file
(if its name is specified in the configuration file),event(...)
method the main part of "median algorithm" is implemented:m_stat
- number of events with |S/N| > SoNThr
,m_signal
- signal amplitude, or 0(zero) for masked pixels,m_proc_status
- sets 255 for S/N > SoNThr
or 0(zero) for masked pixels.m_proc_status
and m_signal
to find peaks:v_peaks
of struct Peak
, using peak_npix_min
, peak_npix_max
, and peak_amp_tot_thr
parameters,v_peaks
, count total amplitude and the number of peaks in the event.event_npeak_min
, event_amp_tot_thr
, and selection_mode
parameters.m_stat
array and frac_noisy_imgs
parameter. When to start and for how many events to update the mask is defined by the nevents_mask_update
and nevents_mask_accum
parameters, respectively.m_signal
in file for selected events, depending on out_file_bits
parameter.v_peaks
in the evt with key=key_peaks_out
.endJob(...)
method, depending on bit status in out_file_bits
:hot_pix_mask_out_file
frac_noisy_evts_file
parameter | default value | description |
---|---|---|
| "DetInfo(:Cspad)" | input source of data |
|
| key for input data, for example, it might be "calibrated" |
|
| key for output signal array. If the string non-empty, the array is added in the datagram for each event (before selection). |
| "peaks" | key for vector of found peaks in the selected event |
| key for ndarray<const float,2>, shape=[npeaks,12] of found peaks in the selected event | |
| "cspad-pix-mask-in.dat" | in read the pixel mask from file |
| "cspad-pix-mask-out.dat" | out write current pixel mask in the file |
| "cspad-pix-frac-out.dat" | out file with per-pixel fraction of noisy images |
| "./cspad-ev-" | out file with signal CSPad array. Time stamp is added. |
| 3 | radial parameter of the area for median algorithm |
| 1 | radial band width of the area for median algorithm |
| 3 | S/N threshold for each pixel to be considered as noisy |
| 5 | S/N threshold for each pixel to be considered as noisy |
| 0.9 | fraction of noisy images above which pixel will be masked |
| 4 | minimal number of connected pixels for the good peak |
| 25 | maximal number of connected pixels for the good peak |
| 0. | threshold on total signal amplitude of the group of connected pixels, if =0:OFF |
| 7. | threshold on peak S/N (S and N are sums over connected pixels) |
| 10 | threshold on minimal number of good peaks for the event selection |
| 10000 | threshold on maximal number of good peaks for the event selection |
| 0. | threshold on total signal amplitude in ADU of all good peaks for the event selection, if =0:OFF |
| 0 | number of skipped events before each mask re-evaluation cycle |
| 50 | number of events for the mask re-evaluation |
| SELECTION_ON | selection mode, other allowed values are: SELECTION_OFF, or SELECTION_INV |
| 0 | control on writing of files:
|
| 0 | module verbosity:
|
See also Example for Module ImgAlgos::CSPadArrPeakFinder.
This module is intended for analysis of the results obtained in the peak finding algorithm implemented in the ImgAlgos::CSPadArrPeakFinder
module.
key
parameter and prints it.fname_root
parameter - NOT AVAILABLE SINCE V00-03-50 - get rid of root...parameter | default value | description |
---|---|---|
| "DetInfo(:Cspad)" | input source of data |
| "peaks" | key for input data for peaks found in event |
| "file.root" | name of the output file with root histograms and ntuples - NOT AVAILABLE SINCE V00-03-50 - get rid of root... |
| 0 | module verbosity:
|
See also Example for Module ImgAlgos::CSPadArrPeakAnalysis.
This package contains python modules which work with both frameworks pyana and psana. Functionality of these modules resembles modules from C++ package ImgAlgos. The difference between two frameworks at code level is explained in Migration from pyana to psana.
This module gets data from evt
store for CSAPD or CSPAD2x2 depending on unique detector name in parameter source
, produce numpy array of full scale shape (4,8,185,388) or (185, 388, 2) of specified in dtype
type, and saves it in the evt
store with unique name key_out
. In case of missing 2x1 sections, their pixel amplitudes substituted by the value form val_miss
.
parameter | default value | type | description |
---|---|---|---|
| *-*|Cspad-* | string | address of Detector-Id|Device-Id |
| int | string | output array data type. Implemented types: int, int8, int16, int32, uint8, uint16, uint32, float, double. |
| cspad_array | string | unique keyword for output array identification |
| 0 | float | intensity value substituted for missing 2x1 sections in data |
| 1 | int | module verbosity:
|
See also Examples for package pyimgalgos.
This module gets from evt
store the numpy array identified by key_in
of full scale shape (4,8,185,388) or (185, 388, 2) for CSPAD or CSPAD2x2, respectively, and produces 2-d image numpy array, taking into account geometry calibration parameters specified by the path calib_dir
. The output image array is saved in the evt
store with unique name, specified by parameter key_out
.
parameter | default value | type | description |
---|---|---|---|
| '' (empty) | string | path to calibration directory for ex.: |
| *-*|Cspad-* | string | address of Detector-Id|Device-Id |
| cspad_array | string | keyword for input numpy array, shape=(4, 8, 185, 388) - for cspad or (185, 388, 2) - for cspad2x2 |
| cspad_image | string | unique keyword for output 2-d image numpy array |
| 1 | int | module verbosity:
|
See also Examples for package pyimgalgos.
This module gets from evt
store the 2-d image numpy array identified by
, crop it using range of row and column parameters, and saves cropped 2-d image numpy array in the source
and key_inevt
store with unique key_out
.
parameter | default value | type | description |
---|---|---|---|
| *-*|Cspad-* | string | address of Detector.Id:Device.Id |
| mage_in | string | keyword for input 2-d image numpy array |
| image_out | string | keyword for output 2-d image numpy array |
rowmin | 0 | int | row minimal to crop image (dafault = 0 - for full size) |
rowmax | -1 | int | row maximal to crop image (dafault = -1 - for full size) |
colmin | 0 | int | column minimal to crop image (dafault = 0 - for full size) |
colmax | -1 | int | column maximal to crop image (dafault = -1 - for full size) |
| 1 | int | module verbosity:
|
This module gets from evt
store 2-d image numpy array for specified unique name in key_in
and saves it in the file with name given by parameter ofname
. File extension defines the output file format. Experiment, run, and event numbers are added to the name of the output file. For example, for ofname = image.tiff
files will be created with names image-<experinent>-r####-ev######.tiff
, where symbols # stands for numbers.
parameter | default value | type | description |
---|---|---|---|
| *-*|Cspad-* | string | address of Detector-Id|Device-Id |
| image | string | unique keyword for input 2-d image numpy array |
| /roi-img | string | output file name (type is selected by extension) supported formats: txt, tiff, gif, pdf, eps, png, jpg, jpeg, npy (default), npz |
mode | 0 | int | 0-save one event per event, >0-length of the ring buffer (or round robin) for event browser |
delay_sec | 0 | int | additional sleep time in sec between events for event browser |
| 1 | int | module verbosity:
|
In the event browser mode, mode>0, this module saves images in the files with names from the ring buffer. That images can be seen by the application plims
, see command options:
% plims -h
See also Examples for package pyimgalgos.
Is intended to print records showing job performance current and integrated from the beginning of job:
pyimgalgos.tahometer: run:0049 evt:000005 t[sec]: 2.575 dt[sec]: 2.575 n/t[1/sec]: 1.942 dn/dt[1/sec]: 1.942 |
parameter | default value | type | description |
---|---|---|---|
| 100 | int | interval in number of events to print current statistics |
| 1 | int | module verbosity:
|
See also Examples for package pyimgalgos.
This module is an example of how to get from the evt
store the 2-d numpy array of shape=[Npeaks,12] with peak info produced by ImgAlgos.CSPadArrPeakFinder.
The numpy array is identified by
. If the peak numpy array is available in the event it will be printed.source
and key_in
parameter | default value | type | description |
---|---|---|---|
| DetInfo(:Cspad.) | string | address of Detector.Id:Device.Id |
| peaks_nda | string | keyword for input 2-d numpy array |
| 1 | int | module verbosity:
|
See also Example for Module ImgAlgos::CSPadArrPeakAnalysis.
The translator package include the H5Output module which translates xtc to hdf5. For more information see the page Outdated: The XTC to HDF5 Translator
The psana_test package includes the psana module dump, some Python library code for testing, and a command line tool for providing a line oriented dump of xtc files. The psana_test package is primarily for psana developers to do software testing, however the dump module and xtclinedump can be generally useful to users.
The dump module will take a standard psana datasource and dump all the event, config, and epics data found. The entire contents of large arrays are not printed. However a checksum over all the array data is, as well as the min, 25th percentile, median, 75th percentile, and max over the data. The dump module does not serve as a good example of how to retrieve and work with objects from the event store – see the psana_examples package for this.
An example of running the module is
psana -n 2 -m psana_test.dump exp=xpptut13:run=179
This dumps the first two events of run 179 of the xpp tutorial data.
Below we annotate the output that psana_test.dump can produce. All annotations are preceded by a #
================== === begin job ==== # first the epics aliases are printed during begin job. Epics Aliases: total = 240 Be_xpos Be_ypos Be_zpos ... # next the epics pv, as they appear during beginJob. # This corresponds to the xtc configure transition. # At this point, these are ctrl pvs. Epics PV pvName=HX2:DVD:GCC:01:PMON pvid=106 dbrtype=34 isCtrl=1 pvName=HX2:DVD:GCC:01:PMON numElements=1 status=0 severity=0 units=T upper_disp_limit=1.0000e-02 lower_disp_limit=0.0000e+00 upper_alarm_limit=0.0000e+00 upper_warning_limit=0.0000e+00 lower_warning_limit=0.0000e+00 lower_alarm_limit=0.0000e+00 upper_ctrl_limit=1.0000e-02 lower_ctrl_limit=0.0000e+00 data=5.0000e-09 ... # After epics, we get the content of the psana env configStore. # The dump module is getting all keys from the configStore(), # then retrieving each object. Objects that have a xtc type id, or are # an numpy array will be printed. # For each object, we first get a string describing the even key: type=psana.ControlData.ConfigV2, src=ProcInfo(0.0.0.0, pid=7670) # then we get the data of the object. # most all data for a Psana object is obtained through accessor methods. # methods that return unsigned ints print in hex. # methods that return signed ints print in decimal. # methods that return floats print in scientific format with 4 decimals of precision. npvControls: 0x1 npvMonitors: 0x0 npvLabels: 0x0 events: 0x1E0 uses_duration: 0x0 uses_events: 0x1 # methods that return a compound type, like duration() that returns the compound type time # are printed as follows: duration: nanoseconds: 0x0 seconds: 0x0 # some methods return a python list. Each element in the list is printed separately: pvControls[0]: name: las_lensh index: 0xFFFFFFFF value: 0.0000e+00 array: 0x0 ... # epics config is not dumped, see the EPICS alias list for how Psana stores this data type=psana.Epics.ConfigV1, src=DetInfo(EpicsArch.0:NoDevice.0) epicsConfig not dumped # cspad config is an example which has methods that return ndarrays type=psana.CsPad.ConfigV4, src=DetInfo(XppGon.0:Cspad.0) ... numSect: 0x20 # some methods return a list of simple types, these are printed in one line roiMask [0]=0xFF [1]=0xFF [2]=0xFF [3]=0xFF numAsicsStored [0]=0x10 [1]=0x10 [2]=0x10 [3]=0x10 ... quads[0]: ... dp: # for an ndarray, we print the type, dimensions, adler32 checksum, and quartile # statistics (min, 25th percentile, median, 75th percentile, and max): pots: ndarray_uint8_1: dim=[ 80 ] adler32=0x231B31F5 min=0x0 25th=0x3F median=0xB0 75th=0xFF max=0xFF gm: gainMap: ndarray_uint16_2: dim=[ 185 x 194 ] adler32=0x18730001 min=0x0 25th=0x0 median=0x0 75th=0x0 max=0x0 ... =============================================================== === beginrun 0 === # typically, there is nothing new in beginrun =============================================================== === begincalibcycle run=0 step=0 === # having dumped the entire initial contents of epics and the config store, # the dump module will now only print changes to epics or the config. # That is it remembers how each epics pv and config object printed the last # time it saw it. With each new transition, it looks at all the epics pv and # config objects. If any change, they are dumped. # The control data changed in the calib cycle: type=psana.ControlData.ConfigV2, src=ProcInfo(0.0.0.0, pid=7670) npvControls: 0x1 npvMonitors: 0x0 npvLabels: 0x0 events: 0x1E0 uses_duration: 0x0 uses_events: 0x1 duration: nanoseconds: 0x0 seconds: 0x0 pvControls[0]: name: las_lensh index: 0xFFFFFFFF value: -4.9997e-01 array: 0x0 # next we see event data, printing the following: =============================================================== === event: run=0 step=0 event=0 seconds= 1362889345 nanoseconds= 770371931 fiducials= 19593 # at this point, all epics pv's are replaced with TIME pv's, not the stamp.sec, stamp.nsec below: Epics PV pvName=HX2:DVD:GCC:01:PMON pvid=106 dbrtype=20 isTime=1 numElements=1 status=0 severity=0 stamp.sec=731737344 stamp.nsec=134374000 data=5.2000e-09 ... # while most epics pv's have one value, there are some with more than one. If a EPICS pv has less than 20 values, they are all printed, otherwise the typical ndarray summary is printed. # now we get into regular event data type=psana.EvrData.DataV3, src=DetInfo(NoDetector.0:Evr.0) numFifoEvents: 0x2 fifoEvents[0]: timestampHigh: 0x4C89 timestampLow: 0x32A6 eventCode: 0x29 fifoEvents[1]: timestampHigh: 0x4C89 timestampLow: 0x2E4C eventCode: 0x8C type=psana.CsPad.DataV2, src=DetInfo(XppGon.0:Cspad.0) quads[0]: seq_count: 0x1 ticks: 0x329D fiducials: 0x4C89 sb_temp: ndarray_uint16_1: dim=[ 4 ] adler32=0x4FC00AC min=0x7 25th=0x8 median=0x9 75th=0x291 max=0x291 frame_type: 0x4 data: ndarray_int16_3: dim=[ 8 x 185 x 388 ] adler32=0xAD5ACF7F min=0 25th=1281 median=1346 75th=1475 max=16383 virtual_channel: 0x0 lane: 0x0 tid: 0x0 acq_count: 0x85 op_code: 0x85 quad: 0x0 sectionMask: 0xFF quads[1]: seq_count: 0x1 ticks: 0x329D fiducials: 0x4C89 sb_temp: ndarray_uint16_1: dim=[ 4 ] adler32=0x2DB006A min=0x7 25th=0xA median=0xA 75th=0x34B max=0x34B frame_type: 0x4 data: ndarray_int16_3: dim=[ 8 x 185 x 388 ] adler32=0xD28441BE min=0 25th=1316 median=1374 75th=1504 max=16383 virtual_channel: 0x0 lane: 0x0 tid: 0x0 acq_count: 0x85 op_code: 0x85 quad: 0x1 sectionMask: 0xFF |
If a source alias has been defined, it will show up when the event key is printed:
type=psana.Pimax.FrameV1, src=DetInfo(AmoEndstation.0:Pimax.0) alias=pimax
Several options allow you to control the output of psana_test.dump
The most useful are
include = term1 term2
exclude = term1 term2
These are used to filter the key strings. For example, running
psana -m psana_test.dump -o psana_test.dump.include=10k exp=mob30114:run=145
Would only dump event keys that had 10k in them, effectively giving you only Epix::Config10KV1 and psana.Epix.ElementV1 since these are the only types coming from the source DetInfo(NoDetector.0:Epix10k.0).
Other options one could set are:
epics = False do not print epics
aliases = False do not print the EPICS alias list
dump_aliases=True follow EPICS aliases to print the EPICS pv's they point to
dump_sml=True dump the small data type (if found, psana should automatically replace these proxies)
regress_dump=True do not print the DAQ assigned pvId when printing EPICS
dump_beginjob_evt=False do not dump begin job data
output_file = filename write output to filename
config = False do not print the contents of the configStore, only regular event data
counter = False do not print the counter string that labels event numbers and calib cycle numbers
header = False
indent = 4 change the indent from the default of 2 to 4
Two functions are provided in the Python psana_test package that allow Python scripts to turn Psana objects into strings. A Python script could include the following function to build dictionaries describing the state of the event, configStore, and epicsStore:
from psana_test import obj2str, epicsPvToStr def getPsanaState(event, configStore, epicsStore): evtDict = {} cfgDict = {} epicsDict = {} for key in event.keys(): if key.type() is None: continue obj = event.get(key.type(), key.src(), key.key()) if (obj is None): continue if not hasattr(obj,'TypeId'): continue evtDict[str(key)]=obj2str(obj) for key in configStore.keys(): if key.type() is None: continue obj = configStore.get(key.type(), key.src()) if (obj is None): continue if not hasattr(obj,'TypeId'): continue cfgDict[str(key)]=obj2str(obj) for pvName in epicsStore.pvNames(): pv = epicsStore.getPV(pvName) if not pv: continue epicsDict[pvName] = epicsPvToStr(pv) return evtDict, cfgDict, epicsDict |
xtclinedump is a command line tool to dump xtc and datagram header information in a line oriented style. By keeping the output for each header to one line, it makes it easy to use grep to filter the output. The command can be run by
xtclinedump dg xtcfile.xtc or xtclinedump xtc xtcfile.xtc |
The first just dumps datagram headers, the latter dumps xtc headers. There are some additional options, how much of the xtc payloads to print, and if you want parsed output for epics. A help string is available by typing xtclinedump with no arguments. Except for the non-default epics argument, xtclinedump does no parsing of the xtc payloads, it simply prints the first few bytes in hex. For reading through payloads, the intel architechture uses little endian, so 0x00040000 = 1024,
dump_sml dump small data type