Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Include Page
PSDM:PageMenuBegin
PSDM:PageMenuBegin
Table of Contents
Include Page
PSDM:PageMenuEnd
PSDM:PageMenuEnd

About

This page provides a list of existing modules for psana framework. Only the modules that are included in the standard analysis releases appear on this page.

...

  • gets any camera image data Camera::FrameV1 from the event store for specified source and key_in parameters,
  • puts the ndarray<const T,2> object with camera image in the event store using specified type outtype and key_out parameters.

     

    Note

    Special treatment for fccd960: if  outtype is not "asdata" the gain factor depending on gain bits is applied, See FCCD-Detector.

parameter

default value

description

source

"DetInfo(:Camera)"

source of data

key_in

 

key for input data

key_out

"pnccdimg"

output key for image saved in event

outtype

"asdata"

out data type: implemented values: asdata (default, uint16_t), float, double, int and int16.

subtract_offset

true

on/off the amplitude offset using configuration data (not applied for outtype = asdata)

print_bits

0

verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - time stamp for each event
  • +4 - summary at the endJob
  • +8 - details about data format conversion and applied offset
  • +16 - configuration data for each beginCalibCycle

...

parameter

default value

description

source

DetInfo(:Opal1000)

input source of data

key

 

key for input data, for example, it might be "calibrated"

sumfile

""

out file with sum of amplitudes, saved if the name is not empty

avefile

""

out file with averaged amplitudes, saved if the name is not empty

rmsfile

""

out file with rms, saved if the name is not empty

maskfile""out file with pixel mask with 0/1-for bad/good pixels, saved if the name is not empty

hotpixfile

""

out file with pixel bit-words: 0/1/2/4/8 for good/hot/saturated/cold/cold-rms, saved if the name is not empty

maxfile""out file with maximal value per pixel over events
ftypetxtout file type: txt (default), metatxt, bin

thr_rms_min

0.

minimal threshold on rms (in ADU);  If rms lower than this threshold - pixel is cold-rms.

thr_rms_ADU

10000.

maximal threshold on rms (in ADU); =0 - use auto-evaluated threshold. If rms exceeds this threshold - pixel is hot.

thr_min_ADU

-100000.

threshold on minimal intensity (in ADU); if ave exceeds this threshold - pixel is good

thr_max_ADU

100000.

threshold on maximal intensity (in ADU); if ave exceeds this threshold - pixel is bad

evts_stage1

1000000

number of events before stage 1

evts_stage2

0

additional number of events before stage 2

gate_width1

0

gate_width for stage 1

gate_width2

0

gate_width for stage 2

print_bits

0

module verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - event record
  • +4 - beginning of 3 stages
  • +8 - processed statistics at the end of each stage
  • +16 - records for saved files
  • +32 - summary with keywords for parser
  • +64 - statistics of bad pixels
  • +128 - mean and rms in 3 iterations of the threshold auto-evaluation algorithm and evaluated threshold
  • +256 - warning if ndarray shape is not defined yet

...

parameter

default value

description

source

DetInfo(:Camera)

source of data

key_in

 

key for input ndarray<const T,NDIM>

key_out

calibrated

output key for calibrated image saved in event

outtypedoubleoutput ndarray data type can be set to double(default), float, int, and int16.

do_peds

false

true: pedestals subtracted if available in calib store

do_cmod

false

true: common mode correction is evaluated and applied [Ref.]

do_stat

false

true: bad/hot pixels in pixel_status are masked

do_mask

false

true: mask is applied if the file fname_mask is available (1/0 = good/bad pixels)

do_bkgd

false

true: normalized background is subtracted if the file fname_bkgd is available

do_gain

false

true: pixel_gain correction is applied if available in calib store

do_nrms

false

true: per-pixel threshold is applied if pixel_rms  is available in calib store

do_thre

false

true: low level threshold in ADU is applied

fname_bkgd

 

input file name for background, applied if the file name is specified

fname_mask

 

input file name for mask, applied if the file name is specified

masked_value

0.

intensity value (in ADU) substituted for masked pixels

threshold_nrms

3.

threshold as a number of sigmas to pixel_rms parameters

threshold

0.

common low level threshold in ADU

below_thre_value

0.

intensity substituted for pixels below threshold

bkgd_ind_min

0

minimal index in flatten ndarray, which is used for background normalization

bkgd_ind_max

100

maximal index in flatten ndarray, which is used for background normalization

bkgd_ind_inc

2

index increment in flatten ndarray, which is used for background normalization

print_bits

0

verbosity:

  • =0 - print nothing
  • +1 - input pars
  • +2 - calibration parameters
  • +4 - common mode algorithm parameters
  • +8 - ndarray parameters; type, ndim, shape, etc.
  • +16- time stamp for each event
  • +32 - first 10 elements of the raw image data
  • +64 - first 10 elements of the calibrated image data

...

parameter

default value

description

source

"DetInfo()"

source of data

key

 

key for input data ndarray, default is empty - raw data

key_droplets

 

key for output list of peaks as std::vector<AlgDroplet::Droplet> (default is empty - do not save)

key_smeared  

threshold_low

10

low threshold on pixel amplitude

threshold_high

100

high threshold on pixel amplitude

sigma

1.5

width of the Gaussian for smearing; =0-no smearing

smear_radius

3

radius in pixel for smearing - radial size of matrix of weights

peak_radius

3

radius in pixel for peak finding - radial size of the region to search for local maximum

low_value 0value substituted for pixels with intensity below threshold and outside window

windows

 

list of windows, each window is defined by 5 parameters; segment-index, rowmin, rowmax, colmin, colmax, separated by space. Default is empty - process all segments

mask path to the file with mask, by default (empty) mask is not used
masked_value0value substituted for masked pixels (0-masked, non-zero - good pixel)

fname_prefix

 

Common prefix for saved files. If non-empty - save files with image and list of droplets for each event with found droplets. The file name is formed as <prefix>-r####-e######-<suffix>.txt, where hash stands for number (0-9), suffix may be raw, smeared, or peaks. Default is empty - do not save files.

print_bits

0

module verbosity:

  • =0 - print nothing,
  • +1 - input pars in the beginJob(...),
  • +2 - summary in the endJob(...),
  • +4 - number of droplets/peaks in the event,
  • +8 - array of peak parameters in the event,
  • +16 - print info about saved files (if they are saved)
  • +64 - info messages from smearing and droplet finding algorithms, 
  • +128 - debugging messages from smearing and droplet finding algorithms,
  • +256 - details for debugging; messages from windows parser, window parameters accounting for segment limits,
  • != 0  - all warning messages

...

This module is a part of complex algorithm, described in Command Line Interface For Time Correlation Analysis.

This module is designed for parallel image processing for correlation analysis.
Functionality:

...

This package contains python modules which work with both frameworks pyana and psana. Functionality of these modules resembles modules from C++ package ImgAlgos. The difference between two frameworks at code level is explained in Migration from pyana to psana.

Module pyimgalgos.cspad_arr_producer

...

The translator package include the H5Output module which translates xtc to hdf5. For more  information see the page Outdated: The XTC - to - HDF5 Translator

Package psana_test
Anchor
psana_test
psana_test

...

epics = False       do not print epics
aliases = False do not print the EPICS alias list
dump_aliases=True follow EPICS aliases to print the EPICS pv's they point to
dump_sml=True dump the small data type (if found, psana should automatically replace these proxies)
regress_dump=True do not print the DAQ assigned pvId when printing EPICS
dump_beginjob_evt=False do not dump begin job data
output_file = filename write output to filename
config = False do not print the contents of the configStore, only regular event data
counter = False do not print the counter string that labels event numbers and calib cycle numbers
header = False
indent = 4 change the indent from the default of 2 to 4

...

The first just dumps datagram headers, the latter dumps xtc headers. There are some additional options, how much of the xtc payloads to print, and if you want parsed output for epics. A help string is available by typing xtclinedump with no arguments. Except for the non-default epics argument, xtclinedump does no parsing of the xtc payloads, it simply prints the first few bytes in hex. For reading through payloads, the intel architechture uses little endian, so 0x00040000 = 1024,

Package TimeTool

An example of how to use the TimeTool can be found Here.

Modules for analyzing recorded data from a timetool camera setup. The timetool camera measures the time difference between laser and FEL in one of two methods:

  1. spatial encoding, where the X-rays change the reflectivity of a material and the laser probesthat change by the incident angle of its wavefront; or
  2. spectral encoding, where the X-rays change the transmission of a material and the chirped laser probes it by a change in the spectral components of the transmitted laser.

Below the package modules are described. The package includes sample configuration files that describe all the options. From a psana release directory, users are encouraged to add the TimeTool package to obtain the latest source. For instance:

Code Block
newrel ana-current myrel
cd myrel
kinit             # get ticket to check out package from svn repository
addpkg TimeTool
sit_setup
scons
# now examine the files in TimeTool/data:  sxr_timetool.cfg  sxr_timetool_setup.cfg  timetool_setup.py  xpp_timetool.cfg  xpptut.cfg

 timetool_setup.py is a python script to calculate the digital filter weights.

Module Analyze

A module that analyzes the camera image by projecting a region of interest onto an axis and dividing by a reference projection acquired without the FEL.  The resulting projection is processed by a digital filter which yields a peak at the location of the change in reflectivity/transmission.  The resulting parameters are written into the psana event. The type of the parameter depends on the release. Starting with ana-0.13.10, a TimeTool::DataV2 object in put in the event store. ana-0.13.3 put a TimeTool::DataV1 object in the event store. This is the preferred method to retrieve the data that TimeTool.Analyze writes. The module also writes its output as a set of doubles, and can optionally be written as a set of ndarrays to help with C++ to Python conversion. To enable this, set the option

put_ndarrays=True

to the config file. However this is not neccessary in releases after ana-0.13.3.

Controlling Laser/Beam Logic

TimeTool.Analyze is often used on experiments where both the laser and beam fire at different times. TimeTool.Analyze does the following based on what it determines about the laser and beam:

  • laser on, beam off: builds a reference/background based on just the laser. The user may configure TimeTool.Analyze to load the reference from a file, in case no "beam off" data was acquired in the run.

  • laser on, beam on: when it has a reference of just the laser background, computes its results and puts them in the Event.
  • laser off: nothing

The laser on/off beam on/off logic is typically determined based on evr codes, and looking at energy in the beam monitors (ipmb data)  - which evr codes and ipmb's is configurable. However for some experiments, users need to override this logic and make their own decision. Starting in ana-0.13.17, this can be done as follows

  • configure TimeTool.Analyze to get laser and/or beam logic from strings in the Event
  • Write a Psana Module that puts "on" or "off" in the Event for the laser and/or beam based on your own logic
  • Load this Psana Module before loading TimeTool.Analyze

The new parameters to tell TimeTool.Analyze to get laser/beam logic are "beam_on_off" and "laser_on_off". For example, if you do

Code Block
# in a config file
[TimeTool.Analyze]
beam_on_off_key=beam_on_off
laser_on_off_key=laser_on_off

then TimeTool.Analyze will bypass it's own logic for determining if the laser as well as the beam is on or off, and get if from variables in the event that are strings, with the keys "beam_on_off" and "laser_on_off" (you can set those to whatever you like, and you need not specify both if you only want to control the beam logic, or laser logic, respectively).

Next one needs to write a Psana Module (not a standard Python script) that adds these variables into the event. A good reference for Psana Modules is psana - User Manual. Note - this link is different then the links that discuss writing Python scripts, such as  psana - Python Script Analysis Manual. The Psana module will have to add the variables for every event - once you specify a value for beam_on_off_key, or laser_on_off_key, those keys need to be present for all events. An example Psana Module written in Python might be

Code Block
languagepython
class MyMod(object):
    def event(self, evt, env):
        evt.put("on","beam_on_off")
        evt.put("off","laser_on_off")

Now, assuming this Psana Module called MyMod was in a Package called MyPkg (so it resied in a file in your test release, MyPkg/src/MyMod.py) if one were to set the psana modules option like so

Code Block
[psana]
modules=MyPkg.MyMod,TimeTool.Analyze

then TimeTool.Analyze would treat the beam as on and the laser as off for every event.

Module Check

a module that retrieves results from the event for either the above module or from data recorded online.

Module Setup

a module that calculates the reference autocorrelation function from events without FEL for use in the digital filter construction.

References

References

 

 

 

 

 

 

 

 

 

 

dump_sml            dump small data type