Python Structure

Produced by psana/src/dgram.cc.

Accessed from psana event with a pattern like:

evt._dgrams[nDgram].detname[nSegment].drpClassName.attr1.attr2...

  • detname is defined by the hutch. For example: "xppcspad" or "xpphsd"
  • nSegment is an integer, denoting which portion of a larger detector this is (e.g. cspad panel number)
  • drpClassName is defined by the DRP segment-level software via the "Alg" parameter to xtcdata/xtcdata/xtc/ShapesData.hh:Names class.  For example: "raw", "fex", "roi1", "roi2"

A complication:  the different detector segments can be split over several datagrams (offline event-build) or be all in one datagram (online event-build)

Some Examples and Special Cases

From a conversation with Mona Uervirojnangkoorn on Aug. 27, 2020.

for epics scans:
dgram[0].scans[0].raw.motor2
det = Detector('motor2')
val = det(evt) (timestamp in the event is used to do lookup in stepstore)
could do:
val = det(step) (and get timestamp from the step to do stepstore lookup)


for epics values:
SIOC::SYS0::VARNAME maps to SIOC__SYS0__VARNAME (maybe)
dgram[0].epics[0].raw.SIOC::SYS0::VARNAME
Detector('SIOC::SYS0::VARNAME')
dgram[0].epics[0].raw.SIOC__SYS0__VARNAME
det = Detector('motor1')
val = det(evt)
for "jump" each detector owns a stepstore and the detector uses it to to return correct value.


for normal detectors:
dgram[0].tmoopal[0].raw
Detector('tmoopal')


proposal for configuration scans ... try to put it in the main detector interface:

det = Detector('tmoopal')
dgram[0].tmoopal[0].cfgscan.user.blacklevel (maybe cfgscan is similar to raw/fex: optionally present)
val = det.cfgscan.user.blacklevel(evt or step)
jumping works
"cfgscan" would be a detector base-class attribute only if being scanned
(no attribute if not being scanned!)

Implementation

1) Detectors, scan and epics, own EnvStore and their variables (e.g. motor1) an be accessed by det('motor1').

2) Detectors with cfgscan (configs[0].detName[0].cfgscan) also own EnvStore and the corresponding values (segment dependent) can be accessed by det.cfgscan.user.black_level(evt) for example.

This is done by EnvStore going through configs in __init__. Scan and epics EnvStores are hard-coded and any other detectors, if they have cfgscan in DrpClass level will also be there.

DetectorImpl uses EnvStore to decide if this detector should be added with hierarchical attributes so 2) can be achieved.

Naming

In general xtcdata/xtc/ShapesData.hh::Name names should be python compatible containing only A-Z,a-z,0-9,_.  Unfortunately there are exceptions that make general naming conventions difficult to enforce in "attributes" xtcdata/xtc/ShapesData.hh::Name.  Note that python-compatibility is enforced for the "detector name" in the higher-level ShapesData.hh::Names.  The exceptions for "Name:"

  • ENUMDICT and ENUMVAL types have special handling that requires a ":".
  • Attributes (e.g. attr1, attr2 in evt._dgrams[nDgram].detname[nSegment].drpClassName.attr1.attr2...) are created by adding "." in the names.
  • I think if no alias is specified for epics variables then the ugly epics name is used (often having ":", ".", maybe others, but I haven't seen any)
    • feels like we should prevent "." in the xtc name for epics vars, given the fatal error below where "." is incorrectly interpreted as an attribute delimeter
  • in the case of step-scans I think the epics variables have no python-compatible alias name. Need to avoid "." here too

For epics vars with ":" psana handles it correctly by using getattr() for epics vars.  But that character does not work for names that are not epics/enumdict/enumval.

For epics vars with a "." we get the fatal error shown here in the ancillary "epicsinfo" object which has a dictionary mapping the xtc Name to the ugly/real epics name.  This because the portion of the name after the "." is interpreted as a hierarchical attribute ("container" object) by psana's dgram.cc.

(ps-4.1.3) psanagpu102:~$ detnames exp=tmoc00118,run=232
Traceback (most recent call last):
  File "/cds/home/c/cpo/git/lcls2/install/bin/detnames", line 11, in <module>
    load_entry_point('psana', 'console_scripts', 'detnames')()
  File "/cds/home/c/cpo/git/lcls2/psana/psana/app/detnames.py", line 71, in detnames
    names = myrun.detinfo.keys()
  File "/cds/home/c/cpo/git/lcls2/psana/psana/psexp/run.py", line 159, in detinfo
    info[(detname,det_xface_name)] = _enumerate_attrs(getattr(self.Detector(detname),det_xface_name))
  File "/cds/home/c/cpo/git/lcls2/psana/psana/psexp/run.py", line 118, in Detector
    setattr(det,drp_class_name,drp_class(det_name, drp_class_name, self.dsparms.configinfo_dict[det_name], self.dsparms.calibconst[det_name], env_store, var_name))
  File "/cds/home/c/cpo/git/lcls2/psana/psana/detector/misc_detectors.py", line 23, in __init__
    values = getattr(names,n).split('\n')
AttributeError: 'container.Container' object has no attribute 'split'
(ps-4.1.3) psanagpu102:~$

Proposal:

  • all pvnames should have "." replaced with "_" in ShapesData.hh::Name to avoid confusion with hierarchy delimiter.  This should be done by the step-scan software and epicsArch.  We should not abort, but epics scans need to support "." but xtc naming does not.  PvaDetector doesn't need it because it already has to provide an xtc-compatible name, I believe.
  • at xtc-level enforce use of python-compatible characters plus ":" (for epics step-scans which have no alias) and "." for attribute hierarchies.  If we support ":" for epics scans, might as well support it for epicsArch as well.
  • No labels