Page History

...

This module is a part of complex algorithm, described in Command Line Interface For Time Correlation Analysis.

This module is designed for parallel image processing for correlation analysis.
Functionality:

...

Module ImgAlgos::CSPadArrSaveInFile

Wiki MarkupThis module saves the CSPad data array formatted as \ [5920=4*8*185\]\[388\] in output file for each passes event.
Module configuration parameters:

• source (default: "DetInfo(:Cspad)") – input source of data
• key (default: "") – key for input data, for example, it might be "calibrated"
• outfile (default: "cspad-arr") – out file name prefix for saved array
• print_bits (default: 0) – module verbosity:
  • =0 - print nothing,
  • +1 - input pars,
  • +2 - event ID,
  • +4 - time stamp,
  • +8 - saved file names.

Module ImgAlgos::CSPadArrAverage

Wiki MarkupThis module averages the CSPad data array and saves two files for averaged and rms values in CSPad format \ [5920=4*8*185\]\[388\]. In contrast to the [cspad_mod.CsPadPedestals|PCDS:Psana Module Catalog#Modulecspadmod.CsPadPedestals], the input data can be specified with a {{key}}, that allows to average CSPad array for already pre-processed data, for example {{"calibrated"}}. This feature can be used to evaluate the averaged signal or background event. Implemented algorithm of averaging allows to eliminate large statistical fluctuations in the pixel amplitude spectrum. In advanced case averaging may have up to three stages, depending on configuration parameters:

• 0-stage: the 1st portion of events from 0 to evts_stage1 is averaged without any constrains, the preliminary averaged and rms values are defined for each pixel
  at the end of this stage.
• 1-stage: starting from the event evts_stage1 data are collected only for abs(amplitude-average0) < gate_width1. At the end of this stage the preliminary averaged and rms values are defined for each pixel.
• 2-stage: starting from the event evts_stage1 + evts_stage2 data are collected only for abs(amplitude-average1) < gate_width2. At the end of this stage the preliminary averaged and rms values are defined for each pixel and saved in the files specified by the avefile and rmsfile parameters, respectively.

...

This module uses the CSPad array, specified by the configuration parameters source and inputKey, subtracts the background, defined in the file bkgd_fname, and saves the resulting array in the event with outputKey. The subtracted background array is normalized on the sum of pixel amplitudes in the quad section norm_sector, which can be set from 0 to 7.

   shared_ptr<Psana::CsPad::ConfigV2> config2 = env.configStore().get(m_str_src);
   unsigned mask = config2->roiMask(quad_number); // should be in the range from 0 to 255

...

• source (default: "DetInfo(:Cspad)") – input source of data
• inputKey (default: " " ) – key for input data, by default use raw data
• outputKey (default: "bkgd_subtracted" ) – output key for the next data processing stage
• Wiki Markup{{bkgd_fname}} (default: "cspad_background.dat" ) -- – file with CSPad array \ [4*8*185\]\[388\] of averaged background
• norm_sector (default: 0) – CSPad sector in quad from 0 to 7
• print_bits (default: 0) – module verbosity:
  • =0 - print nothing,
  • +1 - input pars,
  • +2 - event ID,
  • +4 - normalization factor for each event,
  • +8 - part of the background array.

...

This module uses the CSPad array, defined by the configuration parameters source and inkey, apply the mask from file mask_fnname and saves the masked data with key outkey. For masked pixels the amplitude will be replaced by the value from masked_amp.

Wiki MarkupThe file {{mask_fnname}} has the same structure as files for pedestals and background with dimensions \ [4*8*185\]\[388\]. Masked pixels are indicated by 0-th in this file. This file can be generated, for example, from the averaged background file, using amplitude threshold. This can be done with auxiliary python script {{MakePixelMask.py}} as explained in [Example for Module ImgAlgos::CSPadMaskApply|PCDS:Psana Module Examples#Example for Module ImgAlgos::CSPadMaskApply].

Module configuration parameters:

• source (default: "DetInfo(:Cspad)" ) – input source of data
• inkey (default: " " ) – key for input data, by default use raw data
• outkey (default: "bkgd_subtracted" ) – output key for the next data processing stageunmigrated-wiki-markup
• {{mask_fname}} (default: "cspad_mask.dat" ) -- file with CSPad mask array \) – file with CSPad mask array [4*8*185\]\[388\]
• masked_amp (default: 0) – this amplitude will replace the amplitude in each masked pixel
• mask_control_bits (default: 1) – control bits for applied mask:
  • =0 - do not apply any mask,
  • +1 - apply mask from file,
  • +2 - mask two long edges of 2x1,
  • +4 - mask two short edges of 2x1,
  • +8 - mask two short rows in the middle of 2x1 (rows with wide pixels).
• print_bits (default: 0) – module verbosity:
  • =0 - print nothing,
  • +1 - input pars (in beginJob),
  • +2 - event ID (in event),
  • +4 - mask statistics (in beginJob),
  • +8 - part of the mask array (in beginJob).

Module ImgAlgos::CSPadArrNoise

Wiki MarkupThis module works on CSPad data array shaped as \ [5920=4*8*185\]\[388\], uses the "median algorithm" to evaluate the signal and noise for each pixel, evaluates S/N ratio for each pixel, counts the fraction of events where {{S/N > SoNThr}}, and writes the same shape arrays for pixel mask and status information in the {{maskfile}} and {{statusfile}}, respectively. The {{statusfile}} contains for each pixel the fraction of events where {{S/N > SoNThr}}. This module presents a part of features implemented in the module [ImgAlgos::CSPadArrPeakFinder|PCDS:Psana Module Catalog#Module ImgAlgos::CSPadArrPeakFinder].

Module configuration parameters:

...

Module ImgAlgos::CSPadArrPeakFinder

...

Module {{ImgAlgos::CSPadArrPeakFinder}} is a psana-based implementation of the "median algorithm" for peak finding in CSPad data array shaped as \ [5920=4*8*185\]\[388\]. This algorithm was first implemented in [myana|PCDS:myana user guide]/[Cheetah|https://github.com/antonbarty/cheetah] by Anton Barty and Co. The "median algorithm" assumes that the amplitude level of background and noise for each pixel can be estimated as a mean and RMS of the surrounding pixels, located in the ring with parameters {{rmin}} and {{dr}} around the pixel in question. The threshold {{SoNThr_noise}} on signal over noise (S/N) ratio allow to asset the pixel amplitude as a large noise fluctuation. Statistics of pixels above the S/N threshold accumulated over many images can be used to form the noisy-pixel mask. For example, if the fraction of images where pixel exceeds the S/N threshold grater than certain value ({{frac_noisy_imgs}}=0.9), the pixel is considered as noisy. The permanent bad pixel mask (see module [ImgAlgos::CSPadMaskApply|PCDS:Psana Module Catalog#Module ImgAlgos::CSPadMaskApply]) and dynamically evaluated noisy pixel mask are used to get rid of bad pixels and improve the image quality. Healthy pixels with S/N above threshold ({{SoNThr_signal}} about 3-5) are treated as potential signals. Using recursive flood-filling algorithm the groups of connected signal pixels can be found and considered as a candidate for a diffraction peaks. Peak finding algorithm uses the amplitude, S/N thresholds, and limits on number of pixels in the connected region (parameters {{peak_amp_tot_thr}}, {{peak_SoN_thr}}, {{peak_npix_min}}, and {{peak_npix_max}}) in order to define the peak. Finally, the event is selected or discarded depending on number of found peaks and total amplitude threshold, defined by the parameters {{event_npeak_min}}, {{event_npeak_max}}, and {{event_amp_tot_thr}}, respectively.

Description of implemented algorithm:

• in the constructor and beginJob(...) method:
  • enter input parameters,
  • (re)set the initial mask of noisy pixels from file hot_pix_mask_file (if its name is specified in the configuration file),
  • do necessary initialization of work arrays.
• in the event(...) method the main part of "median algorithm" is implemented:
  unmigrated-wiki-markup
  • fill \ [4\]\[8\]\[185\]\[388\] per-pixel arrays:
    • m_stat - number of events with |S/N| > SoNThr,
    • m_signal - signal amplitude, or 0(zero) for masked pixels,
    • m_proc_status - sets 255 for S/N > SoNThr or 0(zero) for masked pixels.
  • use arrays m_proc_status and m_signal to find peaks:
    unmigrated-wiki-markup
    • iterate over \ [185\]\[388\] 2x1 pixels and find the connected regions (using recursive flood-filling algorithm)
    • create vector of peaks v_peaks of struct Peak, using peak_npix_min, peak_npix_max, and peak_amp_tot_thr parameters,
  • loop over v_peaks, count total amplitude and the number of peaks in the event.
  • decide if the event selected or not based on event_npeak_min, event_amp_tot_thr, and selection_mode parameters.
  • periodically dynamically re-generate the mask, based on m_stat array and frac_noisy_imgs parameter. When to start and for how many events to update the mask is defined by the nevents_mask_update and nevents_mask_accum parameters, respectively.
  • save m_signal in file for selected events, depending on out_file_bits parameter.
  • put the vector with peaks v_peaks in the evt with key=key_peaks_out.
• in the endJob(...) method, depending on bit status in out_file_bits :
  • save current hot-pixel mask in the file hot_pix_mask_out_file
  • save current fraction of events with noisy/signal pixels in the file frac_noisy_evts_file

...