Table of Contents
Initial Discussion

want:
- give the drp a psana-python script
- drive that psana-python script by calling psana_set_dgram(Dgram*) (would replace the file reading)

...

needed for epixhr
- we should make psana (det.calib and det.image) usable in drp
happens in the worker cores (right after dma engine, before time-ordered messages for teb/meb):
- worry about GIL with threads
may need easier tools to append to xtc from python (extracted features)
configured from configdb?
can consider compiling python (e.g. with cython) for speed improvement
- can gain 20%-30%? (potentially small improvement compared to GIL)
- cpo: treat as an optimization
really depending on mikhail's area-detector xface being able to handle variable number of segments
problem: currently we can't make a DataSource
- Mona can probably implement one
- problem: How do we scale to large numbers of cores fetching calibration constants? (this is a problem for shmem too)

Thoughts on a DRP DataSource

dgrams flow through in memory, one at a time (a little like shmem)
the dgrams are in circular buffers (a little like shmem) ("pebble": per-event-buffers-with-boundaries-listed-explicitly)
like shared memory, we don't actually free the dgram memory when we're done
shmem doesn't work, because shmem drops events if the code can't keep up
need a new pattern similar to shmem

(optional, e.g. if we run 'drp -p "myroi.py <arg1> <arg2>"') run in each worker thread (one subprocess per worker):
- send detector raw-data datagram to psana-python subprocess via a zmq bi-directional pipe (perhaps). Performance matters
- receive potentially two datagrams back:
  - datagram with fex data
  - datagram with trigger data (e.g. number of photons) to send to the teb for both the trigger/monitoring decision
- to save memory the received fex datagram would replace the raw datagram in pebble memory (requires a memory copy)
each teb process would have optional python to process the trigger-datagram produced by the worker (the python may not be psana, depending on whether or not we send all the phase 2 transitions to the teb? ... don't remember)
we would start with fex and do the trigger later

...