Data access example

from psana import DataSource
#ds = DataSource(exp='tmoc00318',run=10, dir='/cds/data/psdm/prj/public01/xtc') # on pcds
ds = DataSource(exp='tmoc00318',run=10, dir='/sdf/data/lcls/ds/prj/public01/xtc')
orun = next(ds.runs())
det = orun.Detector('epix100')
for evt orun.events():
    raw = det.raw.raw()

Direct detector interface methods

Hidden for AMI mask methods

For implementation of AMI most of the area detector interface methods are hidden - name starts with underscore "_". In analysis code these methods can be used directly:

DTYPE_MASK = np.uint8 # OR: from psana.detector.UtilsMask import DTYPE_MASK, DTYPE_STATUS

o = det.raw
mask = o._mask_default(dtype=DTYPE_MASK)
mask = o._mask_calib()
mask = o._mask_calib_or_default(dtype=DTYPE_MASK)
mask = o._mask_from_status(status_bits=0xffff, stextra_bits:(1<<64)-1, gain_range_inds=(0,1,2,3,4), dtype=DTYPE_MASK, **kwa)
mask = o._mask_neighbors(mask, rad=9, ptrn='r')
mask = o._mask_edges(width=0, edge_rows=1, edge_cols=1, dtype=DTYPE_MASK, **kwa)
mask = o._mask_center(wcenter=0, center_rows=1, center_cols=1, dtype=DTYPE_MASK, **kwa)
mask = o._mask(status=True, status_bits=0xffff, stextra_bits:(1<<64)-1, gain_range_inds=(0,1,2,3,4),\
               neighbors=False, rad=3, ptrn='r',\
               edges=True, width=0, edge_rows=10, edge_cols=5,\
               center=True, wcenter=0, center_rows=5, center_cols=3,\
               calib=False,\
               umask=None,\
               force_update=False, dtype=DTYPE_MASK)

Usage of mask in methods det.calib and det.image

kwa = {'status':True, 'status_bits':0xffff, 'stextra_bits':(1<<64)-1, 'gain_range_inds':(0,1,2,3,4),\
       'neighbors=False, 'rad':3, 'ptrn':'r',\
       'edges':True, 'width':0, 'edge_rows':10, 'edge_cols':5,\
       'center':True, 'wcenter':0, 'center_rows':5, 'center_cols':3,\
       'calib':False,\
       'umask':None,\
       'force_update':False}
o =  = det.raw
a = o.calib(evt, cmpars=(7,2,100,10), **kwa)
a = o.image(evt, nda=None, **kwa)

Cumulative mask example

epix10ka_raw_calib_image mask -e ueddaq02 -d epixquad -r211 -o fig-ueddaq02-r211-epixquad-mask.png -P

umask = np.ones((4, 352, 384), dtype=np.uint8)
umask[3,100:120,160:200] = 0

mask = det.raw._mask(status=True, status_bits=0xffff, stextra_bits:(1<<64)-1, gain_range_inds=(0,1,2,3,4),\
                     neighbors=True, rad=5, ptrn='r',\
                     edges=True, edge_rows=10, edge_cols=5,\
                     center=True, center_rows=5, center_cols=3,\
                     calib=False,\
                     umask=umask,\
                     force_update=False, dtype=DTYPE_MASK)
mask += 1 # for visibility of the mask 0 and 1 relative to image background

Class Mask

Alternative usage of mask methods is implemented in class Mask. Class Mask is a wrapper of hidden for AMI mask methods, which can be used to set dedicated mask parameters separately from all other detector interface optional parameters.

Instantiation of the Mask object

Object of the class Class Mask is instantiated for requested list of optional parameters,  evaluates and caches combined mask in det._mask_ property. If this property is None - mask is re-evaluated. 

from psana.detector.mask import Mask, DTYPE_MASK

m = Mask(det,\
         status=True, status_bits=0xffff, stextra_bits:(1<<64)-1, gain_range_inds=(0,1,2,3,4),\
         neighbors=True, rad=5, ptrn='r',\
         edges=True, width=0, edge_rows=10, edge_cols=5,\
         center=True, wcenter=0, center_rows=5, center_cols=3,\
         calib=True,\
         umask=test_umask(det),\
         force_update=False, dtype=DTYPE_MASK)

m = Mask(det) # minimal version.

Class Mask access methods

All other methods of the class Mask are shotcuts to det._mask* methods without "_".

m = Mask(det)
m.set_mask(**kwa) # forced update of cached mask, **kwa - the same list of keyword arguments as in class Mask w/o det.
mask = m.mask(**kwa) # returns cached mask, **kwa - the same list of keyword arguments as in class Mask w/o det.
mask = m.mask_default()
mask = m.mask_calib_or_default() # if available returns mask from "pixel_mask" calibration type, otherwise array of ones. 
mask = m.mask_from_status(status_bits=0xffff, stextra_bits:(1<<64)-1, gain_range_inds=(0,1,2,3,4), dtype=DTYPE_MASK)
mask = m.mask_edges(width=0, edge_rows=1, edge_cols=1, dtype=DTYPE_MASK)
mask = m.mask_center(wcenter=0, center_rows=1, center_cols=1, dtype=DTYPE_MASK)
mask = m.mask_neighbors(mask, rad=9, ptrn='r')

A few test examples can be found in psana/detector/testman/test_mask.py

Implementation notes

  • Default combined mask parameters (status=True, neighbors=False, edges=False, center=False, calib=False, umask=None) are set to define mask for pixel_status only.
  • Mask defined as m = Mask(det) and property det._mask_ is cached for combined/cumulative mask retrieved by the method mask=m.mask().
  • Method m.set_mask(**kwa) or its equivalent mask = m.mask(force_update=True, ...) can be used to update cached mask, if necessary.
  • Method m.mask(...) applies mask_neighbors after mask_from_status only. It this is not enough, mask needs to be constructed using separate m.mask_*(...) methods.
  • Most of methods mask = m.mask_*(...) do not cache intermediate results and consume time for specific mask evaluation. To optimize computing time in analysis it would be wise to evaluate mask once per run only.
  • Module psana.detector.UtilsMask contains useful methods for manual mask building, for example merge_masks(...).
  • By default mask is used in common mode correction, but it is not applied to data in det.raw.calib(...)

Mask conversion from image to ndarray

API

Examples of conversion image-like 2-d mask to data-like 3-d ndarray

Access geometry parameters 
from psana.pscalib.geometry.GeometryAccess import GeometryAccess, img_from_pixel_arrays
geo = GeometryAccess(<geometry-file-name>)
ix, iy = geo.get_pixel_coord_indexes(**kwargs)
mask_nda = geo.get_pixel_mask(mbits=0xffff) # returns mask associated with segment geometry
mask2d = img_from_pixel_arrays(ix, iy, W=mask_nda)

# similar methods are available in the detector interface.
# modify mask2d - add ROI mask bad pixels/regions with mask editor or programatically.
Examples of conversion image-like 2-d mask to data-like 3-d ndarray
from psana.detector.UtilsMask import *
m = convert_mask2d_to_ndarray_using_pixel_coord_indexes(mask2d, ix, iy)
m = convert_mask2d_to_ndarray_using_geo(mask2d, geo, **kwargs) # NOTE: kwargs are the same as in geo.get_pixel_coord_indexes(**kwargs)
m = convert_mask2d_to_ndarray_using_geometry_file(mask2d, gfname, **kwargs)

See for detail UtilsMask.py

CLI

roicon -h

output after the command > roicon -h 
(ps-4.5.26) [dubrovin@psanagpu110:~/LCLS/con-lcls2]$ roicon 1 -g /cds/group/psdm/detector/data_test/geometry/geo-epix10ka2m-16-segment.data -i test-2d-mask.npy -t  -h
OR
(ps-4.5.26) [dubrovin@psanagpu110:~/LCLS/con-lcls2]$ roicon 1 -g /sdf/group/lcls/ds/ana/detector/data_test/geometry/geo-epix10ka2m-16-segment.data -i test-2d-mask.npy -t  -h
usage: 

1) Construct 2-d image (or mask-of-segments) from ndarray with image shaped as data using appropriate geometry file
         roicon 1 -g <geometry-file> [-a (input)<ndarray-shaped-as-data-fname>] [-i <image-(output)file>] [-c <control-bitword>]
  ex1:   roicon 1 -g <geometry-fname>
  ex2:   roicon 1 -g <geometry-fname> -a <ndarray-shaped-as-data-fname>
  test:  roicon 1 -g /cds/group/psdm/detector/data_test/geometry/geo-epix10ka2m-16-segment.data -i test-2d-mask.npy -t

2) (TBD) Create ROI mask using mask editor "med" (DO NOT FORGET to save mask in file!)
         roicon 2 [-i <image-(input)file>] [-m <roi-mask-(output)file>]
  ex1,2: roicon 2
  ex3:   roicon 2 -i image.npy -m roi-mask.npy

3) Convert ROI mask to ndarray with mask shaped as data
         roicon 3 -g <geometry-file> [-m <roi-mask-(input)file>] [-n ndarray-with-mask-(output)-file] [-c <control-bitword>]
  ex1,2: roicon 3 -g <geometry-fname>
  test:  roicon 3 -g /cds/group/psdm/detector/data_test/geometry/geo-epix10ka2m-16-segment.data -m test-2d-mask.npy -n test-3d-mask.npy

Conversion between 2-d and 3-d masks.

positional arguments:
  args                  process number: 1-construct image, 2-run mask editor on image, 3-convert image mask to ndarray; default = 1

optional arguments:
  -h, --help            show this help message and exit
  -g GFNAME, --gfname GFNAME
                        geometry file name, default = <geometry-fname>
  -a AFNAME, --afname AFNAME
                        input ndarray file name, default = None
  -i IFNAME, --ifname IFNAME
                        image file name, default = mask-img.txt
  -m MFNAME, --mfname MFNAME
                        ROI mask file name, default = mask-roi.txt
  -n NFNAME, --nfname NFNAME
                        ndarray mask file name, default = mask-nda.txt
  -c CBITS, --cbits CBITS
                        mask control bits, =0-none, +1-edges, +2-middle, etc..., default = 65535
  -v, --verb            verbosity, default = False
  -t, --dotest          add a couple of rings to the 2-d mask for test purpose, default = False
  -F FIGPREFIX, --figprefix FIGPREFIX
                        figure file name prefix under <dirrepo>/figs, None - do not save, default = fig
  -o DIRREPO, --dirrepo DIRREPO
                        repository for logs and output files, default = ./work-roicon
  -L LOGMODE, --logmode LOGMODE
                        logging mode, one of CRITICAL FATAL ERROR WARN WARNING INFO DEBUG NOTSET TRACE, default = INFO
  -k KWARGS, --kwargs KWARGS
                        str python code evaluated to dict and passed to geo.get_pixel_coord_indexes(**kwargs), default = {}
(ps-4.5.26) [dubrovin@psanagpu110:~/LCLS/con-lcls2]$

Mask Editor

Test commands and plots

Test commands
# on pcds:
roicon 1 -g /cds/group/psdm/detector/data_test/geometry/geo-epix10ka2m-16-segment.data -i test-2d-mask.npy -t
# or on s3df:
roicon 1 -g /sdf/group/lcls/ds/ana/detector/data_test/geometry/geo-epix10ka2m-16-segment.data -i test-2d-mask.npy -t
roicon 2 ... runs Mask Editor - TBD
roicon 3 -g /sdf/group/lcls/ds/ana/detector/data_test/geometry/geo-epix10ka2m-16-segment.data -m test-2d-mask.npy -n test-3d-mask.npy

Content of the files for "assembled" image and "not-assembled" 3-d array

  • test-2d-mask.npy shape:(1641, 1639), generated for geo-epix10ka2m-16-segment.data with a couple of artificially added rings
  • test-3d-mask.npy (16, 352, 384) reshaped to 2d shape:(16*352, 384)

References

  • No labels