About

This page provides a list of existing modules for psana framework. Only the modules that are included in the standard analysis releases appear on this page.

...

This page provides examples for selected modules in Psana Module Examples.

Source Code Access

You can find the source code at SLAC (e.g. for module CSPadPixCoords::PixCoordsTest) in these two directories:

/reg/g/psdm/sw/releases/ana-current/CSPadPixCoords/src/PixCoordsTest.cpp (.cpp for c++ source code, or .py for python source code)
/reg/g/psdm/sw/releases/ana-current/CSPadPixCoords/include/PixCoordsTest.h (c++ include file)

Package psana

Psana package include several simple modules which do generic tasks that do not need knowledge of the event data types.

...

See also Example for Package ImgPixSpectra.

Package ImgAlgos

This packages contains a few psana modules for analysis and image processing.

Module ImgAlgos::Tahometer

...

See also Example for Module ImgAlgos::Tahometer.

Module ImgAlgos::TimeStampFilter

This module passes only the events if their time stamp is in the requested range.
The range of allowed time stamps is defined by the configuration parameters.

...

See also Example for TimeStampFilter and XtcOutputModule.

Module ImgAlgos::EventNumberFilter

...

1. If the first and/or last event numbers are defined, then the filter will select events in this range only.

2. If the evtstring is defined, only listed events of event ranges will be selected. For example, the evtstringparameter can be defined as

   Code Block
   bgColor #F7F7ED
   2,5,11-15,20-25,29,30

   that means the list of events:

   Code Block
   bgColor #F7F7ED
   2 5 11 12 13 14 15 20 21 22 23 24 25 29 30

   In the evtstring parameter the comma "," and sign minus "-" as a dash are the only allowed separators. Blank spaces are also allowed. Other characters may abort the program. The evtstring mode has higher priority than the 1st mode.
   The filterIsOn allows easy turn on/off this filter in *.cfg file.

Module ImgAlgos::EventCounterFilter

...

2 3 6 7 9 11 15
17 28 32 ...

Module ImgAlgos::

...

EventCodeFilter

Functionality:

...

ImgAlgos::EventCodeFilter (ImgAlgos > V00-03-80) module is created by request of Silke Nelson for dark image selection based on EvrData eventCode.

Functionality: Filter selects events which have requested evtcode in Psana::EvrData::DataV4, 3, ... data objects in the list of Psana::EvrData::FIFOEvent objects. Events which do not have the event code are skipped

...

.

Example of configuration parameters

...

for this module:

modules = ... ImgAlgos.EventCodeFilter ...

[ImgAlgos.EventCodeFilter]
evtcode    = 41
mode       = 1
print_bits = 5

Module ImgAlgos::

...

AndorImageProducer

Functionality:

• gets Epix Andor data from the event store for specified source and key_in parameters,
• puts the ndarray<const T,2> object with camera image in the event store using specified key_out parameter, where the output type T is defined by the outtype parameter.

...

Module ImgAlgos::EpixNDArrProducer

...

Module ImgAlgos::PnccdNDArrProducer

Functionality:

• gets pnCCD Epix data from Psana::PNCCD::FramesV1 object from the event store for specified source and key_in parameters,
• puts the ndarray<const TOUTT,3> object of shape 4x512x512 2> object with image in the event store using specified key_out parameter, where the output type T is defined by the outtype parameter.

See also Example for Module ImgAlgos::EpixNDArrProducer.

Module ImgAlgos::

...

PnccdNDArrProducer

Functionality:

• gets from the event store the object with pnCCD data of typefrom Psana::PNCCD::FullFrameV1 containing four [512][512] frames with T=uint16_t, or
• ndarray<const T,3> , where shape=[4][512][512], T=unsigned short, float, double, int, or int16,
FramesV1 object from the event for specified source and inkey key_in parameters,
• puts the ndarray<const TTOUT,2> object with pnccd [1024+gap_rows][1024+gap_cols] image in the event using specified outimgkey 3> object of shape 4x512x512 specified key_out parameter.

...

Module ImgAlgos::PnccdImageProducer

...

This module takes care about any generic camera image.
Functionality:

• gets any camera image data Camera::FrameV1 from the event store the object with pnCCD data of type
  • Psana::PNCCD::FullFrameV1 containing four [512][512] frames with T=uint16_t, or
  • ndarray<const T,3> , where shape=[4][512][512], T=unsigned short, float, double, int, or int16,
  for specified source and key_in inkey parameters,

• puts the ndarray<const T,2> object with camera image in the event using specified type outtype and key_out parameterspnccd [1024+gap_rows][1024+gap_cols] image in the event using specified outimgkey parameter.

on/off the amplitude offset using configuration data (not applied for outtype = asdata)

parameter	default value	description
source	"DetInfo(:CamerapnCCD)"	source of data
key_ininkey		key for input datakey_out
outimgkey	"pnccdimg"	output key for image saved in event
outtype	"asdata"	out data type: implemented values: asdata (default, uint16_t), float, double, int and int16.
gap_rows	0	gap between top and bottom segments in number of pixels
gap_cols	0	gap between left and right segments in number of pixels
gap_value	0	Image effective pixel intensity value in the gap	subtract_offset	true
print_bits	0	verbosity: =0 - print nothing +1 - input pars +2 - time stamp for each eventinitial portion of pnccd array +4 - summary at the endJobconfiguration pars +8 16 - details warning about missing data format conversion and applied offset +16 - configuration data for each beginCalibCycle

See also Example for Module ImgAlgos::CameraImageProducerPnccdImageProducer.

Module ImgAlgos::

...

CameraImageProducer

This module works with any generic camera image stored in data type Camera::FrameV#Functionality:

• gets the Princeton or Pimax any camera image data PrincetonCamera::FrameV1 /2 or Pimax::FrameV1 from the from the event store for specified source and key_in parameters,

• puts the ndarray<const T,2> object with camera image in the event store using specified type outtype and key_out parameters.

   

  Note
  Special treatment for fccd960: if  outtype is not "asdata" the gain factor depending on gain bits is applied, See FCCD-Detector.

See also Example for Module ImgAlgos::PrincetonImageProducerCameraImageProducer.  AnchorAcqirisArrProducerAcqirisArrProducer

Module ImgAlgos::

...

PrincetonImageProducer

Functionality:

• gets the Princeton or Pimax camera image data Princeton::FrameV1/2 or Pimax::FrameV1 from the event store for specified Gets acqiris configuration and data from Acqiris::ConfigV1 and Acqiris::DataDescV1,  objects using parameters source and key_in;
• produces ndarray<const double,2> of shape[] = {nbrChannels, nbrSamples} for waveforms and time stamps;
• saves configuration data in the file defined by fname_prefix;
• parameters,
• puts the ndarray<const T,2> object with camera image in the event using specified type outtype and key_out parameterssaves waveforms and time stamps in the event store with keys key_wform and key_wtime.

...

See also Example for Module ImgAlgos::

...

PrincetonImageProducer.

...

...

Module ImgAlgos::

...

AcqirisArrProducer

• Gets Acqiris waveforms from event store as ndarray<const double,2> object acqiris configuration and data from Acqiris::ConfigV1 and Acqiris::DataDescV1,  objects using parameters source and key_in;
• performs waveform selection controlled by parameters: thresholds, is_postive_signal, do_inverse_selection, in the range depending on local event numbers skip_events and proc_events;

• produces ndarray<const double,2> of shape[] = {nbrChannels, nbrSamples} for waveforms and time stamps;
• saves configuration data in the file defined by fname_prefix;
• saves waveforms and time stamps in the event store with keys key_wform and key_wtimeafter number of events proc_events or at the end of job (whatever happens first), saves array of averaged waveforms in the text file with name constructed from fname_ave_prefix and in the event store using parameters source and key_average.

 Example for Module ImgAlgos::

...

AcqirisArrProducer

...

...

Module ImgAlgos::

...

AcqirisAverage

• Gets Acqiris waveforms from event store as ndarray<const double,2> object using parameters source and key_in;
• processes events performs waveform selection controlled by parameters: thresholds, is_postive_signal, do_inverse_selection, in the range depending on local event numbers skip_events and proc_events;

• at the 1st processed event loads the fname_base_line file with baseline  ndarray<const double,2>;
• subtract baseline from waveforms;
• save corrected waveforms in the event store as ndarray<const double,2> object after number of events proc_events or at the end of job (whatever happens first), saves array of averaged waveforms in the text file with name constructed from fname_ave_prefix and in the event store using parameters source and key_outaverage.

 Example for Module ImgAlgos::AcqirisCalibAcqirisAverage

Module ImgAlgos::

...

AcqirisCalib

• Gets Acqiris waveforms from event store as ndarray< ndarray<const double,2> object object using parameters parameters source,  and key_wform, key_wtime;
• runs constant-fraction discriminator algorithm on all acqiris channels using user-specified per-channel parameters;
• in;
• processes events in the range depending on local event numbers skip_events and proc_events;
• at the 1st processed event loads the fname_base_line file with baseline  ndarray<const double,2>;
• subtract baseline from waveforms;
• save corrected waveforms in the event store as ndarray<const double,2> object using parameters source and key_out.

  saves edges into the event as a set of ndarray<double,1>;

   

...

...

...

...

...

...

...

...

...

...

Key for output ndarray<double,1> with calibrated waveforms. This key will have the acqiris channel number (1 thru 20) appended to the end of it, and the data for that channel will be added to the event only if edges were found.

 Example for Module ImgAlgos::AcqirisCalib

Module ImgAlgos::AcqirisCFD

• Gets Acqiris waveforms from event store as ndarray<const double,2> object using parameters source, key_wform, key_wtime;
• runs constant-fraction discriminator algorithm on all acqiris channels using user-specified per-channel parameters;

• saves edges into the event as a set of ndarray<double,1>;

   

  parameter	default value	description
  source	"DetInfo(:Acqiris)"	Source of data.
  key_wform	"acqiris_wform"	Key for input ndarray with waveforms (either raw, or subtracted using AcqirisCalib module) from evt store.
  key_wtime	"acqiris_wtime"	Key for input ndarray with waveform times from evt store.
  key_edges	"acqiris_edges_"	Key for output ndarray<double,1> with calibrated waveforms. This key will have the acqiris channel number (1 thru 20) appended to the end of it, and the data for that channel will be added to the event only if edges were found.
  baselines	""	A list of baseline values (one per channel) to subtract from the waveform in volts.
  fractions	""	A list of fractions (one per channel) between 0 and 1. The edge-time reported will be at the time when the pulse is at this fraction of the peak value.
  thresholds	""	A list of threshold values (one per channel) in volts that indicate a new edge should be found. If this value is less than the baseline, then the algorithm will look for negative pulses, otherwise it will look for positive pulses.
  deadtimes	""	A list of deadtimes (one per channel) in seconds. After each edge the algorithm will ignore any new hits in this time interval.
  leading_edges	""	A list of 0/1 values (one per channel) indicating whether edge-times are desired for leading edges (1) or falling edges (0).

 Example for Module ImgAlgos::AcqirisCFD

Module ImgAlgos::NDArrImageProducer

This module converts any (detector-associated) ndarray in to image.

For any available in the event store ndarray<const T,NDim>, defined by the source and key_in parameters, and which size is equal to the number of pixels in the detector,  this module creates 2-d image ndarray<const TOUT,2> and saves it in the event store with key_out. TOUT is defined by the type_out. Pixel geometry array is retreived from the calibration file of  "geometry" type from default standard or specified in calibdir directory.

Configuration parameters

...

""

...

A list of baseline values (one per channel) to subtract from the waveform in volts.

...

""

...

A list of fractions (one per channel) between 0 and 1. The edge-time reported will be at the time when the pulse is at this fraction of the peak value.

...

""

...

A list of threshold values (one per channel) in volts that indicate a new edge should be found. If this value is less than the baseline, then the algorithm will look for negative pulses, otherwise it will look for positive pulses.

...

""

...

A list of deadtimes (one per channel) in seconds. After each edge the algorithm will ignore any new hits in this time interval.

...

""

...

A list of 0/1 values (one per channel) indicating whether edge-times are desired for leading edges (1) or falling edges (0).

 Example for Module ImgAlgos::AcqirisCFD

Module ImgAlgos::NDArrImageProducer

This module converts any (detector-associated) ndarray in to image.

For any available in the event store ndarray<const T,NDim>, defined by the source and key_in parameters, and which size is equal to the number of pixels in the detector,  this module creates 2-d image ndarray<const TOUT,2> and saves it in the event store with key_out. TOUT is defined by the type_out. Pixel geometry array is retreived from the calibration file of  "geometry" type from default standard or specified in calibdir directory.

Configuration parameters:

Additional parameters which can be used for special applications like alignment:

...

Additional parameters which can be used for special applications like alignment:

See also Example for Module ImgAlgos::NDArrImageProducer.

Module ImgAlgos::NDArrAverage

This module averages over events the per-element data of the image array (

Module ImgAlgos::NDArrAverage

This module averages over events the per-element data of the image array (ndarray<const T,NDim>, where T is implemented for almost all types: int, int16, uint, float, double etc., NDim≤5) and saves files for sum, averaged, rms values, mask, and, the hot pixel map. Input ndarray can be specified by the source and key parameters. Averaging may have up to three stages, depending on configuration parameters:

...

...

• and saves the corrected ndarray<const doubleTOUT,NDim> NDim> in the event with key key_out, where TOUT is controlled by the parameter outtype, which can be set to double(default), float, int, and int16.

• In ImgAlgos V00-02-01 implemented detectors: CsPad, CsPad2x2, Pnccd, Princeton, Andor, Opal1000, Opal4000

...

...

Doxygen documentation for interface: CalibParsStore,  GenericCalibPars: CalibParsStore,  GenericCalibPars

 See also Example for Module ImgAlgos::NDArrCalib,

Test of the NDArrCalib module for pnCCD.

Module ImgAlgos::NDArrDropletFinder

Finds "droplets" (wide peaks) in data ndarray and saves their list in output ndarray. This is a re-implementation of algorithm ImgPeakFinder for ndarray of 2-d segments with functionality as follows.
1. Selects pixels in the windows defined by the list of parameters:  segment, rowmin,  rowmax, colmin,  colmax with amplitudes above the threshold_low.
2. Optionally smears image for selected pixels using 2-d matrix of weights in the range -smear_radius to +smear_radius in both dimensions around smeared pixel. The matrix of weights is defined by the 2-d Gaussian function of width sigma. If sigma=0 smearing is not applied.
3. Finds peaks as pixels with absolute-maximal amplitude above the threshold_high in the center of the matrix covering the range -peak_radius to +peak_radius  in both dimensions.
4. Puts the ndarray of found peaks in the event store with key key_droplets. Each row of this ndarray has parameters from struct Droplet, containing segment number,  row, column of the droplet center, peak pixel amplitude, the total amplitude inside peak_radius the region, and the number of pixels in this region above threshold_low:

  struct Droplet{
    unsigned seg;
    double   row;
    double   col; 
    double   ampmax;  // amplitude in the peak maximum
    double   amptot;  // total amplitude in the range of peak_radius
    unsigned npix;    // number of pixels in the range of peak_radius
  };

Remarks:

• if print_bits is not zero - warning messages will be printed.
• Saves table of droplets/peaks as  ndarray<float,2> with shape=[ndroplets,6]  (see struct Droplet) if the keyword  peaks_droplets is non-empty.
• A set of threshold parameters should be different in cases when smearing is applied (sigma is not zero) or not.

See  See also Example for Module ImgAlgos::NDArrCalib,

Test of the NDArrCalib module for pnCCD.

NDArrDropletFinder

Module ImgAlgos::PixCoordsProducer

...

This module is a part of complex algorithm, described in Command Line Interface For Time Correlation Analysis.

This module is designed for parallel image processing for correlation analysis.
Functionality:

...

This package contains python modules which work with both frameworks pyana and psana. Functionality of these modules resembles modules from C++ package ImgAlgos. The difference between two frameworks at code level is explained in Migration from pyana to psana.

Module pyimgalgos.cspad_arr_producer

...

The translator package include the H5Output module which translates xtc to hdf5. For more  information see the page Outdated: The XTC - to - HDF5 Translator

Package psana_test

Anchor
psana_test psana_test

The psana_test package includes the psana module dump, some Python library code for testing, and a command line tool for providing a line oriented dump of xtc files. The psana_test package is primarily for psana developers to do software testing, however the dump module and xtclinedump can be generally useful to users.

module dump

The dump module . This will take a standard psana datasource and dump all the event, config, and epics data found. The entire contents of large arrays are not printed. However a checksum over all the array data is, as well as the min, 25th percentile, median, 75th percentile, and max over the data. The dump module does not serve as a good example of how to retrieve and work with objects from the event store – see the psana_examples package for this. The psana_test package is primarily for psana developers to do software testing.

...

Running psana_test dump

An example of running the module is

...

epics = False       do not print epics
aliases = False     do not print the EPICS alias list
dump_aliases=True   follow EPICS aliases to print the EPICS pv's they point to
dump_sml=True       dump the small data type (if found, psana should automatically replace these proxies)
regress_dump=True   do not print the DAQ assigned pvId when printing EPICS printing EPICS
dump_beginjob_evt=False   do not dump begin job data
output_file = filename   write output to filename
config = False      do not print the contents of the configStore, only regular event data
counter = False     do not print the counter string that labels event numbers and calib cycle numbers
header = False
indent = 4          change the indent from the default of 2 to 4

...

from psana_test import obj2str, epicsPvToStr

def getPsanaState(event, configStore, epicsStore):
    evtDict = {}
    cfgDict = {}
    epicsDict = {}
    for key in event.keys():
        if key.type() is None: continue
        obj = event.get(key.type(), key.src(), key.key())
        if (obj is None): continue
        if not hasattr(obj,'TypeId'):  continue
        evtDict[str(key)]=obj2str(obj)
    for key in configStore.keys():
        if key.type() is None: continue
        obj = configStore.get(key.type(), key.src())
        if (obj is None): continue
        if not hasattr(obj,'TypeId'):  continue
        cfgDict[str(key)]=obj2str(obj)
    for pvName in epicsStore.pvNames():
        pv = epicsStore.getPV(pvName)
        if not pv: continue configStore.keys():
        epicsDict[pvName] = epicsPvToStr(pv)if key.type() is None: continue
    return evtDict, cfgDict, epicsDict

Package TimeTool

An example of how to use the TimeTool can be found Here.

Modules for analyzing recorded data from a timetool camera setup. The timetool camera measures the time difference between laser and FEL in one of two methods:

1. spatial encoding, where the X-rays change the reflectivity of a material and the laser probesthat change by the incident angle of its wavefront; or
2. spectral encoding, where the X-rays change the transmission of a material and the chirped laser probes it by a change in the spectral components of the transmitted laser.

Below the package modules are described. The package includes sample configuration files that describe all the options. From a psana release directory, users are encouraged to add the TimeTool package to obtain the latest source. For instance:

newrel ana-current myrel
cd myrel
kinit  obj = configStore.get(key.type(), key.src())
        if (obj is None): continue
   # get ticket to check outif package from svn repository
addpkg TimeTool
sit_setup
scons
# now examine the files in TimeTool/data:  sxr_timetool.cfg  sxr_timetool_setup.cfg  timetool_setup.py  xpp_timetool.cfg  xpptut.cfg

 timetool_setup.py is a python script to calculate the digital filter weights.

Module Analyze

a module that analyzes the camera image by projecting a region of interest onto an axis and dividing by a reference projection acquired without the FEL.  The resulting projection is processed by a digital filter which yields a peak at the location of the change in reflectivity/transmission.  The resulting parameters are written into the psana event. Starting with ana-0.13.3, a TimeTool::DataV1 object will be placed in the event in addition to the set of doubles that have been placed in the past. A temporary solution for C++ to Python conversion was adding the option

put_ndarrays=True

to the config file. This wrote the doubles as ndarrays, but this will not be necessary with ana-0.13.3.

Module Check

a module that retrieves results from the event for either the above module or from data recorded online.

Module Setup

not hasattr(obj,'TypeId'):  continue
        cfgDict[str(key)]=obj2str(obj)
    for pvName in epicsStore.pvNames():
        pv = epicsStore.getPV(pvName)
        if not pv: continue
        epicsDict[pvName] = epicsPvToStr(pv)
    return evtDict, cfgDict, epicsDict

xtclinedump

xtclinedump is a command line tool to dump xtc and datagram header information in a line oriented style. By keeping the output for each header to one line, it makes it easy to use grep to filter the output. The command can be run by

xtclinedump dg xtcfile.xtc
or
xtclinedump xtc xtcfile.xtc

The first just dumps datagram headers, the latter dumps xtc headers. There are some additional options, how much of the xtc payloads to print, and if you want parsed output for epics. A help string is available by typing xtclinedump with no arguments. Except for the non-default epics argument, xtclinedump does no parsing of the xtc payloads, it simply prints the first few bytes in hex. For reading through payloads, the intel architechture uses little endian, so 0x00040000 = 1024,a module that calculates the reference autocorrelation function from events without FEL for use in the digital filter construction.

References

• Psana User Manual - Old
• Psana Reference Manual - Old
• psana - Module Examples
• Migration from pyana to psana
• psana - Migration from pyana

dump_sml            dump small data type