Mar 14, 2022 discussion with Chuck, Mona, Jyoti, Kristjan, Georgi, Jana, Silke, 

• will integrate with andor (1D) (most important, VLS, fall 2022), rixccd (1D) (not far behind andor), epixm (2D) (least important), opal-replacement, maybe exit-slit-manta
  • some 1D will need to be 2D for droplet algorithms
• need to know precisely which shots contribute to the images
  • will depend on how precisely camera shuttering is controllable (expect epixm to be precise, others to be imprecise)
    • if a camera shutter is not precisely controllable (andor, rixccd) we will use DAQ sequencing which is less efficient use of the beam. have to run in "burst mode". need to check with Matt.  georgi wants high efficiency ~90%.  Longer exposures improve efficiency, but can introduce problems.
    • precisely controllable shutters (e.g. epixm) can be done with readout-groups instead of daq sequences
• typically will want to sum over the high rate detectors, e.g. to get an I-zero, but for ATM need to learn distribution (e.g. RMS)
• not clear how we handle the psana parallelization with integrating detectors

Discussion of Various Detectors

March 21, 2022

first use case (fall 2022?): andor + hsd.  rixccd later.  then epixM.

import questions for each detector:
1) precise shutter? (know exactly which shot start/ends the integration period)
2) imprecise shutter?
   - use burst mode if so
   - how efficient?  georgi wants 90+%
3) can we expose the camera while it's reading out?
   - use burst mode if so

timestamping:
- should be done on the shutter-close

psana:
- should "batch" events in units of the slow detector
  - this means all batching detectors must have sync'd shutters up to
    integer multiple.  e.g. camera A integrating at 2Hz, camera B can
    integrate at 1Hz, as long as they are "sync'd"
- current "destination callback" in psana is likely not sufficient because
  only supports 1EB core, and for MHz we need multiple EB cores

what about deadtime?
- camera still sees signal but we can't record deadtime
- may need to throw these images away because they can't be analyzed
  perfectly
- may need to run in a regime with no deadtime
- dream: could deadtime trigger DMPBSY?
- they can calculate percentage of shots lost by comparing #hsd shots
  seen in xtc to expected number in burst

lcls1 experience:
- cam integrates for 30 minutes
  - times will be shorter (1Hz or 120Hz) for LCLS2 MHz
  - attach to "shutter closing" timestamp
- some cameras were software controlled (listened to evr multicasts)
- andor can listen to sequencer code
  - send shutter-open eventcode
  - wait a bit for andor to respond (4 or 5 fiducials?)
  - burst ends
  - special event code in sequencer for shutter-close

e.g. efficiency for 120Hz FVB

- impact from the sequencer itself minimal
  o previously waited 5*8.4ms between camera shutter-open and burst-start
    but this could be optimized for LCLS2.
  o andor shutter-open ("exposure start") is done all in hardware
  o andor has a "clear mode" but then can't run at 120Hz
  o since it's all hardware we likely could reduce it significantly
- expect inefficiency to be dominated by readout (3ms?)
- andor key idea: last burst shot (or a programmed sequence event after the last burst shot, which is cleaner, but this has the wrong timestamp?) will send the TTL to trigger the readout (have to delay this appropriately by timing it in).  if we use the last burst shot, have to delay it a little bit so readout starts after the pulse arrives: one does this in two ways: (1) delay with a TPR setting, or (2) delay readout in the camera with the "exposure time" setting (perhaps preferred because it works in single-shot or low-beam-rate mode without a sequence)
  o plus: works with the current IOC
- next burst runs automatically.  insert enough time to cover the readout time
  plus jitter (but jitter should be microseconds?).  we think the jitter
  on the transfer time (due to the OS) doesn't matter:  in principle next
  exposure could overlap transfer of data over usb.
- readout time depends on all the usual params: ADC speed, ROI, FVB...
- there is a PV that estimates the camera readout time (this is
  "Acquisition Period" from SDK on the upper right of main panel (was 0.018s
  in 32 partial-vertical-binning mode).  FVB is 0.006s.  This is an
  over-estimate because it includes usb data-transfer time.  Maybe
  the SDK has a number that doesn't include data-transfer time.  Otherwise
  we have to try to calculate it ourselves somehow

- so for 1ms readout ~1/8.4=12% dead, and 3ms is 36% dead at
  120Hz.  If we integrate longer deadtime decreases, obviously.
- set exposure time to 0 because the camera is always exposing (maybe
  a couple of internal clocks).  program in readout event code in usual
  place.
- someone (bruce or dan) needs to integrate Kukhee's timing system module
  into IOC.

burst mode:
- dump with new kicker magnet (DMPBSY) (settable per-pulse)
- not generate at all (settable per-pulse, may impact stability)

- BYKIK (event code 161) may not work
- goes to TPG on accelerator side (like EVG)

sequencer example:
https://github.com/slac-lcls/lcls2/blob/master/psdaq/psdaq/seq/burst.py

rixccd:
- roughly same as andor
- has extra complication: "batched" output, but this is the same for integrated
  running and non-integrated running.

epixm:
- do we need burst mode or can we run continuously? (does it have a "precise
  shutter")
- old epixes had a AcqWidth.  clear had to be kludged for 10k (ghost images)
  maybe means it's always exposing?
  - ask detector group
  - ** what happens if xrays hit the sensor while it's reading out?
    o this could force us into burst-mode with sequences
- if it is a precise shutter set eventcode and acqWidth
- might we need to change the timestamping in the camera to always

  be at the DAQ trigger instead of the RUN trigger.  could be a big
  change.
  - ** we should raise this issue with TID-AIR
  - related to the issue of temperature stability where Matt talked
    with Dionisio about separate RUN/DAQ triggers
- time is getting short on epixM and epixHR
- maybe RUN triggers go all the time.  the DAQ triggers are the usual
  readout-group event-codes.  set acqWidth precisely.