...

• (done) Move to Rogue 6: v 1.0.0
• (done) current configuration time is long at 34s
  • some progress on this: now 10s according to Ric (check that we're down to 2-3 seconds that rogue sees?)
• fiber-power monitoring on the detector side and kcu1500
  • not there yet on march 22, 2024
  • (done for the detector as of v1.1.5 of epix-hr-m-320k)
• • .  Now ~5s according to Alex on Sept. 16, 2024?
• (to be checked) Alex thought that the configuration is not re-done if it hasn't changed in the database.  Ric doesn't think this is the case.
• (done) fiber-power monitoring in both kcu and epixM devGui
  • for kcu1500: "python ./devGui.py --dev /dev/datadev_0 --pciePgpEn 1 --boardType XilinxKcu1500"
  • done for the detector as of v1.1.5 of epix-hr-m-320k
• (done) have to manually lock the lanes between ASICs and (done?) have to manually lock the lanes between ASICs and managing FPGA by running 1000 events: feels awkward-ish
  • what does epixHR do?
• not all lanes in an ASIC lock (can perhaps be fixed with improved delay settings)
  • most have 3-5 lanes that don't function or are unstable (out of 24) for unit 0xbf
• data is currently scrambled (not natural order)
  • ric is descrambling ASICS in software, but would like to move to firmware
• (done) remove epixViewer imports in _Root.py
• on May 11, 2024 lost timing link from epixM to xpm7 (had to power cycle).  Include Julian's latest link fixes in your firmware.
• need eye-scanning for all transceiver links
• add batcherEventBuilder to kcu1500
  • use Lcls2EpixHrXilinxKcu1500Pgp4_10Gbps, which contains a BEB
  • make the ePixM devGui compatible
• (done) remove 8 bytes of null padding between timing header elements
• fix set-registers-before-each-charge-injection-event issue
  • Implement FPGA registers to rearm ASICs on each event when test register is true?
• (done) Split prepareChargeInjection() into 2 functions, the first taking first and last column (as now) and the second taking a 384 element numpy array (e.g., setupChargeInjection(self, asicIndex, lane_selected, pulserValue))
• (done) Since the scan work, normal data taking runs now see dropped and short frames from ASIC 0
• DAQ has no environmental monitoring support as yet
  • needs board re-spin (boards in layout on March 22, 2024)
• will epixM automatically increase charge with each injection pulse like epixHR?  Dionisio thinks probably not (will double-check with Lorenzo).
• (done?) zmq server port gone with rogue6?  needs to be re-added?
• need to know common-mode "bank" info
  • each lane?
  • other structures for ADC?
• v1.1.5 of epix-hr-m-320k introduced per detector instance yaml files, which implies that if a detector is swapped out, either:
  • the configDb for the given detector alias must be reinitialized, or
  • a different detector alias and configDb instance must be set up
    • requires corresponding .cnf changes

YAML Re-Ingestion Issue When Swapping Detectors

slack conversation with Dawood/Alex on Aug 21 2024

Does it make sense to you that the yaml files need to be re-ingested after a detector swap, and if so can you clarify why we can’t use the same yaml for all detectors?

Dawood Alnajjar
We cannot use the same yaml because there are ASIC specific configurations, and we are unable to get rid of this still and ASICs vary between carriers and within one carrier

...

• • Dawood thinks that the LaneDiagnostics function could be removed.  Ric says it is commented out on 9/16/24
• (done) eliminate manual asic-specific configurations (e.g. delay) in yaml.  v1.1.9.1 contains lane recover logic that guarantees that guarantees daq receives a full frame (and per-event metadata): automatic delay determination (surf does this in some unknown way) instead of manually determined delays in the yaml.
  
  • Dawood says the manual delays might still be better
• (workaround for the daq) not all lanes in an ASIC lock (can perhaps be fixed with improved delay settings)
  • still an issue, but won't kill the DAQ
• (done) blinking asics in ami.  fixed by increasing timeouts.
• data is currently scrambled (not natural order)
  • ric is descrambling ASICS in software, but would like to move to firmware
  • in progress on 9/16/24
• (done) remove epixViewer imports in _Root.py
• on May 11, 2024 lost timing link from epixM to xpm7 (had to power cycle).  Include Julian's latest link fixes in your firmware.
  • v1.1.9.1 has the latest Julian link fixes, but during the beamtime in early September 2024, Alex had to do a TxLinkReset on the timing link.  Might be an indication of a problem (not 100% clear, since beam times are chaotic)
• need Julian's eye-scanning for all transceiver links (important)
  • have seen significant BER's at 10Gbps
• add batcherEventBuilder to kcu1500
  • use Lcls2EpixHrXilinxKcu1500Pgp4_10Gbps, which contains a BEB
  • make the ePixM devGui compatible
• (done) remove 8 bytes of null padding between timing header elements
• fix set-registers-before-each-charge-injection-event issue
  • Implement FPGA registers to rearm ASICs on each event when test register is true?
  • how it will work: specify 2 columns only.  all columns in between get charge.  and each trigger increases the amount of charge.  Dawood will try to have the charge be incremented by a user-settable amount (vs. fixed amount)
• (done) Split prepareChargeInjection() into 2 functions, the first taking first and last column (as now) and the second taking a 384 element numpy array (e.g., setupChargeInjection(self, asicIndex, lane_selected, pulserValue))
• (done) Since the scan work, normal data taking runs now see dropped and short frames from ASIC 0
• DAQ has no environmental monitoring support as yet
  • needs board re-spin (boards in layout on March 22, 2024)
  • still in progress 9/16/24
• (done) (fixed by Dawood in late Aug 2024, according to Alex) need new config object when swapping detectors to pick up per-detector delay settings (Same as 4 bullets down?)
• (done) zmq server port gone with rogue6?  needs to be re-added?
• (done: ric has received format for both header/trailer from Dawood and is getting them into xtc) With v1.1.9.1 there is a trailer appended to the data.  Alex indicated the group wants both the header and the trailer recorded in addition to the image data.  They each seem to be 48 bytes long.  We need the format of the these structures, since has per-event information about bad lanes, for example.

YAML Re-Ingestion Issue When Swapping Detectors

slack conversation with Dawood/Alex on Aug 21 2024

Does it make sense to you that the yaml files need to be re-ingested after a detector swap, and if so can you clarify why we can’t use the same yaml for all detectors?

Dawood Alnajjar
We cannot use the same yaml because there are ASIC specific configurations, and we are unable to get rid of this still and ASICs vary between carriers and within one carrier

Can you provide a little more detail about what is asic-specific?  Delays? Others?

Dawood Alnajjar
and some others. I will make you a list

christopher o'grady
For us, perhaps those would ideally go in the calibration database (not configuration db).  Then when a detector with a new “serial number” appeared the daq would look up those “delay calibration constants” specific to that detector (calib lookups are done with the serial number) and would be automatically applied.

Dawood Alnajjar
jujst checked. Only delays

Alex Batyuk
Do we have carrierID working that we can assign yaml files to the carriers?

Dawood Alnajjar
not yet. There is still an issue with the carrier ID reading
so far we are using the digital board serial number
I agree. The carrier ID would be ideal

Alex Batyuk
digitalID is only valid for the digital board, and the delays etc. belong to the asics.. (edited) 

Dawood Alnajjar
yes, correct

Link Quality

Matt added firmware to do a Julian-style bathtub curve for epixm pgp data links and ran it like this

ssh drp-det-cmp001
source ~detopr/scripts/setup_env.sh
python ~detopr/git/lcls2_081624/psdaq/psdaq/eyediag/pgp-drp.py --link 0 --bathtub --target 1.e-10

Miscellaneous Info

Asic readout order:

The ASICs are in this format. ASIC 0 and 1 rotated 180 degree (viewed looking at the sensor):

0 1

3 2

Currently running on drp-neh-cmp003 and using (perhaps incorrectly) tdetsim.service.

• (done) change to kcu.service

Auto-ranging, cannot be run in fixed-range mode (maybe Lorenzo is thinking about this?)

GitHub repo: https://github.com/slaclab/epix-hr-m-320k (currently using branch tempRelease)

currently has 4 ASICs in a 2x2 configuration (one piece of silicon).  the size of the ASIC is 192*384 columns (more than twice as big as EpixHR ASIC)

each ASIC is its own lane

firmware defaults to LCLS-II timing

Need to ask Chris Kenney or Lorenzo about precise panel geometries so Mikhail can support the geometry in psana2

need this setting in devGui under "Root":

Image Added

3 means 168MHz clock, and the 4 1's initialize (includes a reset plus configuration) the ASICs, which includes loading config files: the config files that are used are in config/ePixHRM320k_ASIC_u1_PLLBypass.yml (and u2,u3,u4 for other ASICs).  Currently the 4 configurations are identical, apart from a module name.  Also configures the firmware of a single "managing" FPGA (e.g. batching event-builder).

• As of v1.1.5, InitASIC() is called with 4, 1, 1, 1, 1 where 4 means 'use a default on-board clock configuration'.
• As of v1.1.6, some .yml filenames have changed.  Some are now detector specific and so include the digital board's serial number.

Issue: the serial links between the managing FPGA and the ASICs don't always lock until a number of frames have been transmitted (may want to fix this more robustly in the long-term).  Short-term workaround: call root.hwTrigger(frames, rate) for ~1000 frames at rate 1000 (1kHz).

Currently the data needs to be descrambled in software.  There is a plan to eventually descramble in firmware.

Plan: run the software/notebook/maximumRateTest.ipynb

Introductory script (likely out of date):

### Setup the library ###
import pyrogue as pr
import os, sys
import matplotlib.pyplot as plt
import time
import datetime
import numpy as np
import math
import pprint
import inspect 
top_level=f'{os.getcwd()}/../'
rootTopLevel = top_level+'script/'
pr.addLibraryPath( rootTopLevel )

import setupLibPaths

import ePix320kM as devBoard

args = None

# ONLY RUN ONCE!
# Defining root
root =  devBoard.Root(   
        top_level  = rootTopLevel,
        dev        = '/dev/datadev_0',
        pollEn     = False,
        initRead   = True,
        serverPort = 9099,
        pciePgpEn  = True,
)
root.start()

# example showing a read
AxiVersion = root.Core.AxiVersion
print ( '###################################################')
print ( '#             Firmware Version                    #')
print ( '###################################################')
AxiVersion.printStatus()
print ( '###################################################')

# Useful short names
APP  = root.App
AXIV = root.Core.AxiVersion
ASICTOP = APP.AsicTop
TRIG = ASICTOP.TriggerRegisters
ASIC0 = APP.Mv2Asic[0]
ASIC1 = APP.Mv2Asic[1]
ASIC2 = APP.Mv2Asic[2]
ASIC3 = APP.Mv2Asic[3]
HSDAC = APP.Dac.FastDac 
PKREG = [None] * 4
PKREG[0] = ASICTOP.DigAsicStrmRegisters0
PKREG[1] = ASICTOP.DigAsicStrmRegisters1
PKREG[2] = ASICTOP.DigAsicStrmRegisters2
PKREG[3] = ASICTOP.DigAsicStrmRegisters3
BATCHER0 = ASICTOP.BatcherEventBuilder0
BATCHER1 = ASICTOP.BatcherEventBuilder1
BATCHER2 = ASICTOP.BatcherEventBuilder2
BATCHER3 = ASICTOP.BatcherEventBuilder3
DEBUG0 = root._dbg[0]
DEBUG1 = root._dbg[1]
DEBUG2 = root._dbg[2]
DEBUG3 = root._dbg[3]
DATARCV0 = root.DataReceiver0
DATARCV1 = root.DataReceiver1
DATARCV2 = root.DataReceiver2
DATARCV3 = root.DataReceiver3
FULLRATERCV0 = root.fullRateDataReceiver[0]
FULLRATERCV1 = root.fullRateDataReceiver[1]
FULLRATERCV2 = root.fullRateDataReceiver[2]
FULLRATERCV3 = root.fullRateDataReceiver[3]
DAC = APP.Dac
REGCTRL = ASICTOP.RegisterControlDualClock

# Configure clock to 168 MHz and configures all ASICS
root.InitASIC([3,1,1,1,1])

# disable some software rogue data receivers
root.disableAndCleanAllFullRateDataRcv()
root.enableDataRcv(False)
root.enableDataDebug(False)

#run some triggers and exercise lanes and locks
frames = 5000
rate = 1000

root.hwTrigger(frames, rate)

#get locked lanes
root.getLaneLocks()

#Enable data receivers and run some triggers
root.enableDataRcv(True)
root.enableAllAsics(True)
root.Trigger() # one event via software trigger

# Obtain descrambled single frame data from ASICs from DataReceiver. Data receiver is down sampled.
root.printDataReceiverStatus()
    
frame = [None for i in range(4)]
for asicIndex in range(4):
    frame[asicIndex] = getattr(root, f"DataReceiver{asicIndex}").Data.get()

#frame dimensions
for asicIndex in range(root.numOfAsics):
    print(np.shape(frame[asicIndex]))

#plot image
plt.subplots(2,2,figsize=(17,17))
for asicIndex in range(root.numOfAsics):
    if asicIndex == 3 :
        plt.subplot(2,2,3)
    elif asicIndex == 2 :
        plt.subplot(2,2,4)
    else :
        plt.subplot(2,2,asicIndex+1)
    
    if np.shape(frame[asicIndex])[0] != 1 :
        plt.imshow(frame[asicIndex])
        plt.xlabel("ASIC {}".format(asicIndex))
        plt.colorbar()
    else :
        plt.xlabel("ASIC {}: No data".format(asicIndex))

Teststand: (timing is left fiber, registers are on middle MPO8 fiber, data is on right MPO8 fiber)

Image Added

To lock the lanes:

App->AsicTop->TriggerRegisters→SetAutoTrigger (set to 1000 and hit enter)

numberTrigger set to 5000

StartautoTrigger (exec)

Do "Read All" at the bottom and check that AcqCount and DaqCount are 5000

StopTriggers

View state of locks with App→SspMonGrp[0:3] look at "Locked" register. Total of 24 lanes in each asic (should be 0xffffff).  Can still run even if not all links are locked: disable lanes that are not functioning.  Saw between 0 and 6 lanes not locked for asics 0-3.  Mostly reproducible failing lane numbers.

App->AsicTop→DigAsicStrmRegisters[0:3] set "DisableLane" to turn off non-locked lanes.  A single unlocked lane that is not disabled will prevent frames from being transmitted by firmware.

Enable the software receivers via DataReceiver[0:3] with RxEnable.  Hit "Trigger" to generate one software trigger.  In principle can view an image with DisplayViewer[0:3], but saw a pydm error when we tried this.

To run with XPM triggers, go to the usual TriggerEventManager→TriggerEventBuffer[1] and set "Partition" to the readout group: this is the DAQ trigger.  TriggerEventBuffer[0] is the Run Trigger.

To see trigger counts go to AsicTop→TriggerRegisters and EXEC "SetTimingTrigger" and then do a "Read All" to watch for counters to increment (AcqCount and DaqCount).  Did two things: set TriggerEventBuffer[0] (the run trigger) to also fire on the readout group and set AsicTop->TriggerRegisters→PgpTrigEn to True and then we saw it count.  AcqCount counts run triggers, and the DaqCount counts daq triggers.  Can see it in the DataReceiver "FrameCount"

Dawood writes: Chris, I just noticed that the batchers have the register Bypass set to 1. That needs to become 0 when we using timing from the timing fiber for all 4 ASICS.

Fiber Requirements for Prototype

For MFX beam time Phil requested 6 fiber pairs plus spares, eventually 24:

cpo has a message from Omar dated Oct. 2, 2023 that says there are 6 fiber pairs between room 208 and mfx.

Dionisio wrote about the fiber pair count requirement for the prototype detector:
Looking at the git repo https://github.com/slaclab/epix-hr-m-320k I would think that the minimum number of fibers is 7 but @Dawood Alnajjar please confirm this

Lane[0].VC[0] = Data[0]
Lane[1].VC[0] = Data[1]
Lane[2].VC[0] = Data[2]
Lane[3].VC[0] = Data[3]

Lane[5].VC[0] = SRPv3
Lane[5].VC[1] = software trigger (ssiCmd)
Lane[5].VC[2] = XVC
-Lane[6].VC[0] = slow monitoring[1:0]
-    [1] = Power and Communication Board
-    [0] = Digital Board


Lane[11] = LCLS-II Timing

Running devGui

Using XilinxKcu1500Pgp4_10Gbps mcs files from pgp-pcie-apps GitHub repo

(need to run from the "script" directory at the moment) "python script/devGui.py --pciePgpEn 1 --boardType XilinKcu1500".  or to run with via the zmq port: "python devGui.py --serverPort 9200 --pciePgpEn 1" (zmq port defaults to 9100, but that port was in use in the test stand).  or to ignore the data streams: "python devGui.py --boardType XilinxKcu1500 --dev /dev/datadev_0 --pciePgpEn 1 --justCtrl 1"

For kcu1500 devGui use pgp-pcie-apps directory and run "python scripts/PgpMonitor.py --numLane 8 --boardType XilinxKcu1500".  Currently (April 1, 2024) this still requires rogue5 in ps-4.6.1.

Pedestal Scans and Charge Injection

(documents from Dionisio, Lorenzo and Dawood)

ePixM: gain modes, Charge Injection, Charge injection with python using helper functions

(from Conny on Jan. 24 2024)

As you mentioned, this detector only has one gain mode, which is an outranging mode. However, the idea is to create two additional “fixed gain modes” currently referred to as soft fixed modes. These modes are created by changing the switching point of the auto-ranging mode. So for the SL, the switching point is moved below the baseline and we are always switched, and for SH the switching point is all the way up the dynamic range, so the detector never switches.  As such, unlike the current generation of epix detectors which moves between gain modes by changing the tr_bit and the pixel config matrix value (ePixM do not contain registers that change the functionality of the pixels),  the soft gain modes will be moved into by changing the following

...

Link Quality

Matt added firmware to do a Julian-style bathtub curve for epixm pgp data links and ran it like this

ssh drp-det-cmp001
source ~detopr/scripts/setup_env.sh
python ~detopr/git/lcls2_081624/psdaq/psdaq/eyediag/pgp-drp.py --link 0 --bathtub --target 1.e-10

Miscellaneous Info

Asic readout order:

The ASICs are in this format. ASIC 0 and 1 rotated 180 degree (viewed looking at the sensor):

0 1

3 2

Currently running on drp-neh-cmp003 and using (perhaps incorrectly) tdetsim.service.

• (done) change to kcu.service

Auto-ranging, cannot be run in fixed-range mode (maybe Lorenzo is thinking about this?)

GitHub repo: https://github.com/slaclab/epix-hr-m-320k (currently using branch tempRelease)

currently has 4 ASICs in a 2x2 configuration (one piece of silicon).  the size of the ASIC is 192*384 columns (more than twice as big as EpixHR ASIC)

each ASIC is its own lane

firmware defaults to LCLS-II timing

Need to ask Chris Kenney or Lorenzo about precise panel geometries so Mikhail can support the geometry in psana2

need this setting in devGui under "Root":

Image Removed

3 means 168MHz clock, and the 4 1's initialize (includes a reset plus configuration) the ASICs, which includes loading config files: the config files that are used are in config/ePixHRM320k_ASIC_u1_PLLBypass.yml (and u2,u3,u4 for other ASICs).  Currently the 4 configurations are identical, apart from a module name.  Also configures the firmware of a single "managing" FPGA (e.g. batching event-builder).

• As of v1.1.5, InitASIC() is called with 4, 1, 1, 1, 1 where 4 means 'use a default on-board clock configuration'.
• As of v1.1.6, some .yml filenames have changed.  Some are now detector specific and so include the digital board's serial number.

Issue: the serial links between the managing FPGA and the ASICs don't always lock until a number of frames have been transmitted (may want to fix this more robustly in the long-term).  Short-term workaround: call root.hwTrigger(frames, rate) for ~1000 frames at rate 1000 (1kHz).

Currently the data needs to be descrambled in software.  There is a plan to eventually descramble in firmware.

Plan: run the software/notebook/maximumRateTest.ipynb

Introductory script (likely out of date):

### Setup the library ###
import pyrogue as pr
import os, sys
import matplotlib.pyplot as plt
import time
import datetime
import numpy as np
import math
import pprint
import inspect 
top_level=f'{os.getcwd()}/../'
rootTopLevel = top_level+'script/'
pr.addLibraryPath( rootTopLevel )

import setupLibPaths

import ePix320kM as devBoard

args = None

# ONLY RUN ONCE!
# Defining root
root =  devBoard.Root(   
        top_level  = rootTopLevel,
        dev        = '/dev/datadev_0',
        pollEn     = False,
        initRead   = True,
        serverPort = 9099,
        pciePgpEn  = True,
)
root.start()

# example showing a read
AxiVersion = root.Core.AxiVersion
print ( '###################################################')
print ( '#             Firmware Version                    #')
print ( '###################################################')
AxiVersion.printStatus()
print ( '###################################################')

# Useful short names
APP  = root.App
AXIV = root.Core.AxiVersion
ASICTOP = APP.AsicTop
TRIG = ASICTOP.TriggerRegisters
ASIC0 = APP.Mv2Asic[0]
ASIC1 = APP.Mv2Asic[1]
ASIC2 = APP.Mv2Asic[2]
ASIC3 = APP.Mv2Asic[3]
HSDAC = APP.Dac.FastDac 
PKREG = [None] * 4
PKREG[0] = ASICTOP.DigAsicStrmRegisters0
PKREG[1] = ASICTOP.DigAsicStrmRegisters1
PKREG[2] = ASICTOP.DigAsicStrmRegisters2
PKREG[3] = ASICTOP.DigAsicStrmRegisters3
BATCHER0 = ASICTOP.BatcherEventBuilder0
BATCHER1 = ASICTOP.BatcherEventBuilder1
BATCHER2 = ASICTOP.BatcherEventBuilder2
BATCHER3 = ASICTOP.BatcherEventBuilder3
DEBUG0 = root._dbg[0]
DEBUG1 = root._dbg[1]
DEBUG2 = root._dbg[2]
DEBUG3 = root._dbg[3]
DATARCV0 = root.DataReceiver0
DATARCV1 = root.DataReceiver1
DATARCV2 = root.DataReceiver2
DATARCV3 = root.DataReceiver3
FULLRATERCV0 = root.fullRateDataReceiver[0]
FULLRATERCV1 = root.fullRateDataReceiver[1]
FULLRATERCV2 = root.fullRateDataReceiver[2]
FULLRATERCV3 = root.fullRateDataReceiver[3]
DAC = APP.Dac
REGCTRL = ASICTOP.RegisterControlDualClock

# Configure clock to 168 MHz and configures all ASICS
root.InitASIC([3,1,1,1,1])

# disable some software rogue data receivers
root.disableAndCleanAllFullRateDataRcv()
root.enableDataRcv(False)
root.enableDataDebug(False)

#run some triggers and exercise lanes and locks
frames = 5000
rate = 1000

root.hwTrigger(frames, rate)

#get locked lanes
root.getLaneLocks()

#Enable data receivers and run some triggers
root.enableDataRcv(True)
root.enableAllAsics(True)
root.Trigger() # one event via software trigger

# Obtain descrambled single frame data from ASICs from DataReceiver. Data receiver is down sampled.
root.printDataReceiverStatus()
    
frame = [None for i in range(4)]
for asicIndex in range(4):
    frame[asicIndex] = getattr(root, f"DataReceiver{asicIndex}").Data.get()

#frame dimensions
for asicIndex in range(root.numOfAsics):
    print(np.shape(frame[asicIndex]))

#plot image
plt.subplots(2,2,figsize=(17,17))
for asicIndex in range(root.numOfAsics):
    if asicIndex == 3 :
        plt.subplot(2,2,3)
    elif asicIndex == 2 :
        plt.subplot(2,2,4)
    else :
        plt.subplot(2,2,asicIndex+1)
    
    if np.shape(frame[asicIndex])[0] != 1 :
        plt.imshow(frame[asicIndex])
        plt.xlabel("ASIC {}".format(asicIndex))
        plt.colorbar()
    else :
        plt.xlabel("ASIC {}: No data".format(asicIndex))

Teststand: (timing is left fiber, registers are on middle MPO8 fiber, data is on right MPO8 fiber)

Image Removed

To lock the lanes:

App->AsicTop->TriggerRegisters→SetAutoTrigger (set to 1000 and hit enter)

numberTrigger set to 5000

StartautoTrigger (exec)

Do "Read All" at the bottom and check that AcqCount and DaqCount are 5000

StopTriggers

View state of locks with App→SspMonGrp[0:3] look at "Locked" register. Total of 24 lanes in each asic (should be 0xffffff).  Can still run even if not all links are locked: disable lanes that are not functioning.  Saw between 0 and 6 lanes not locked for asics 0-3.  Mostly reproducible failing lane numbers.

App->AsicTop→DigAsicStrmRegisters[0:3] set "DisableLane" to turn off non-locked lanes.  A single unlocked lane that is not disabled will prevent frames from being transmitted by firmware.

Enable the software receivers via DataReceiver[0:3] with RxEnable.  Hit "Trigger" to generate one software trigger.  In principle can view an image with DisplayViewer[0:3], but saw a pydm error when we tried this.

To run with XPM triggers, go to the usual TriggerEventManager→TriggerEventBuffer[1] and set "Partition" to the readout group: this is the DAQ trigger.  TriggerEventBuffer[0] is the Run Trigger.

To see trigger counts go to AsicTop→TriggerRegisters and EXEC "SetTimingTrigger" and then do a "Read All" to watch for counters to increment (AcqCount and DaqCount).  Did two things: set TriggerEventBuffer[0] (the run trigger) to also fire on the readout group and set AsicTop->TriggerRegisters→PgpTrigEn to True and then we saw it count.  AcqCount counts run triggers, and the DaqCount counts daq triggers.  Can see it in the DataReceiver "FrameCount"

Dawood writes: Chris, I just noticed that the batchers have the register Bypass set to 1. That needs to become 0 when we using timing from the timing fiber for all 4 ASICS.

Fiber Requirements for Prototype

For MFX beam time Phil requested 6 fiber pairs plus spares, eventually 24:

cpo has a message from Omar dated Oct. 2, 2023 that says there are 6 fiber pairs between room 208 and mfx.

Dionisio wrote about the fiber pair count requirement for the prototype detector:
Looking at the git repo https://github.com/slaclab/epix-hr-m-320k I would think that the minimum number of fibers is 7 but @Dawood Alnajjar please confirm this

Lane[0].VC[0] = Data[0]
Lane[1].VC[0] = Data[1]
Lane[2].VC[0] = Data[2]
Lane[3].VC[0] = Data[3]

Lane[5].VC[0] = SRPv3
Lane[5].VC[1] = software trigger (ssiCmd)
Lane[5].VC[2] = XVC
-Lane[6].VC[0] = slow monitoring[1:0]
-    [1] = Power and Communication Board
-    [0] = Digital Board


Lane[11] = LCLS-II Timing

Running devGui

Using XilinxKcu1500Pgp4_10Gbps mcs files from pgp-pcie-apps GitHub repo

(need to run from the "script" directory at the moment) "python script/devGui.py --pciePgpEn 1 --boardType XilinKcu1500".  or to run with via the zmq port: "python devGui.py --serverPort 9200 --pciePgpEn 1" (zmq port defaults to 9100, but that port was in use in the test stand).  or to ignore the data streams: "python devGui.py --boardType XilinxKcu1500 --dev /dev/datadev_0 --pciePgpEn 1 --justCtrl 1"

For kcu1500 devGui use pgp-pcie-apps directory and run "python scripts/PgpMonitor.py --numLane 8 --boardType XilinxKcu1500".  Currently (April 1, 2024) this still requires rogue5 in ps-4.6.1.

Pedestal Scans and Charge Injection

(documents from Dionisio, Lorenzo and Dawood)

ePixM: gain modes, Charge Injection, Charge injection with python using helper functions

(from Conny on Jan. 24 2024)

As you mentioned, this detector only has one gain mode, which is an outranging mode. However, the idea is to create two additional “fixed gain modes” currently referred to as soft fixed modes. These modes are created by changing the switching point of the auto-ranging mode. So for the SL, the switching point is moved below the baseline and we are always switched, and for SH the switching point is all the way up the dynamic range, so the detector never switches.  As such, unlike the current generation of epix detectors which moves between gain modes by changing the tr_bit and the pixel config matrix value (ePixM do not contain registers that change the functionality of the pixels),  the soft gain modes will be moved into by changing the following registers (Ric says these registers are settable per-asic):

...

Sometimes a group is used implicitly, as with the ePixM's (and ePixHR's) Run Trigger group.  Its parameters are not governed by the configDb instance, but instead are manipulated directly using the groupca GUI and a script.  The Beam Test Trigger Setup section of the EpixHR page describes one such script and how to run it.  With groupca, one then enters values on the appropriate group tab to set up rates, event codes, etc., as shown in the first screen-shot below:

Image RemovedImage Removed

Note that which tabs groupca displays is governed by the launch line for it in the .cnf file.  On the Groups/EventCodes tab of the xpmpva GUI (second screen-shot above), one can see what event codes already exist and what their parameters are.  Once the settings are dialed in, switch to groupca's Events tab to verify that the Run Trigger's group is running (L0InpRate not equal to 0).  If it isn't running, click the Run checkbox to start it.  Unfortunately, this will also start the other group (DAQ Trigger), which may confuse the DAQ.  To recover, either restart the DAQ with:

procmgr restart epixM.cnf

or move the Target State in the Control GUI to RESET.  Then Apply the detector selection again.

The Control GUI is built around a Finite State Machine.  The configDb values are applied to the hardware when the DAQ goes through the Configure transition.  Remember to re-Configure (unwind the state machine to CONNECTED or below) when you make a change.

Once the parameters have been set up as desired, a run can be taken.  Decide whether you want to run with or without recording and click the Record button on the Control GUI, as appropriate (must be done when the state machine is in a state lower than BeginRun).  Then move the Target State to RUNNING to start the run.  When done, move the Target State to PAUSED (or lower, e.g., CONFIGURED) to stop the run. 

Alternatively, there are scripts with which to run a scan.  Available scans are timing, pedestal and charge injection.  These can be run by first putting the DAQ in the ALLOCATED state (procmgr start followed by Applying the detectors chosen in the Select partition pop-up) and running one of the following scripts:

python ~rixopr/git/lcls2_040324/psdaq/psdaq/cas/epixhr_timing_scan.py -p 3 -C drp-neh-ctl002 --hutch tst --config BEAM --detname epixm_0 --scantype timing --events 2000 --record 1
python ~rixopr/git/lcls2_040324/psdaq/psdaq/cas/epixm_pedestal_scan.py -p 3 -C drp-neh-ctl002 --hutch tst --config BEAM --detname epixm_0 --scantype pedestal --events 2000 --record 1
python ~rixopr/git/lcls2_040324/psdaq/psdaq/cas/epixm_chargeinj_scan.py -p 3 -C drp-neh-ctl002 --hutch tst --config BEAM --detname epixm_0 --scantype chargeinj --events 2000 --record 1

See the -h option of these scripts for usage information.  Note that the timing scan can be done with the ePixHR script while the pedestal and charge injection scripts are specific to the ePixM.

When you're done with the DAQ, it can be closed down with:

., as shown in the first screen-shot below:

Image AddedImage Added

Note that which tabs groupca displays is governed by the launch line for it in the .cnf file.  On the Groups/EventCodes tab of the xpmpva GUI (second screen-shot above), one can see what event codes already exist and what their parameters are.  Once the settings are dialed in, switch to groupca's Events tab to verify that the Run Trigger's group is running (L0InpRate not equal to 0).  If it isn't running, click the Run checkbox to start it.  Unfortunately, this will also start the other group (DAQ Trigger), which may confuse the DAQ.  To recover, either restart the DAQ with:

procmgr restart epixM.cnf

or move the Target State in the Control GUI to RESET.  Then Apply the detector selection again.

The Control GUI is built around a Finite State Machine.  The configDb values are applied to the hardware when the DAQ goes through the Configure transition.  Remember to re-Configure (unwind the state machine to CONNECTED or below) when you make a change.

Once the parameters have been set up as desired, a run can be taken.  Decide whether you want to run with or without recording and click the Record button on the Control GUI, as appropriate (must be done when the state machine is in a state lower than BeginRun).  Then move the Target State to RUNNING to start the run.  When done, move the Target State to PAUSED (or lower, e.g., CONFIGURED) to stop the run. 

Alternatively, there are scripts with which to run a scan.  Available scans are timing, pedestal and charge injection.  These can be run by first putting the DAQ in the ALLOCATED state (procmgr start followed by Applying the detectors chosen in the Select partition pop-up) and running one of the following scripts:

python ~rixopr/git/lcls2_040324/psdaq/psdaq/cas/epixhr_timing_scan.py -p 3 -C drp-neh-ctl002 --hutch tst --config BEAM --detname epixm_0 --scantype timing --events 2000 --record 1
python ~rixopr/git/lcls2_040324/psdaq/psdaq/cas/epixm_pedestal_scan.py -p 3 -C drp-neh-ctl002 --hutch tst --config BEAM --detname epixm_0 --scantype pedestal --events 2000 --record 1
python ~rixopr/git/lcls2_040324/psdaq/psdaq/cas/epixm_chargeinj_scan.py -p 3 -C drp-neh-ctl002 --hutch tst --config BEAM --detname epixm_0 --scantype chargeinj --events 2000 --record 1

See the -h option of these scripts for usage information.  Note that the timing scan can be done with the ePixHR script while the pedestal and charge injection scripts are specific to the ePixM.

When you're done with the DAQ, it can be closed down with:

procmgr stop epixM.cnf

Header and trailer format

The header and trailer seem to each be 48 bytes long.  However, only the following bytes are defined, according to Dawood:

Byte[8] = ASIC Number
{Byte[7] , Byte[6], Byte [5], Byte[4]} = Frame number

{Byte [3], Byte[2], Byte[1], Byte[0] }= AutoFillMask
{Byte [7], Byte[6], Byte[5], Byte[4] }= FixedMask (laneDisable)

The following detector interface example shows how to access these fields:

from psana import DataSource
ds = DataSource(exp='ascdaq18', run=491, dir='/cds/data/drpsrcf/temp/claus/xtc', max_events=10)
myrun = next(ds.runs())
det = myrun.Detector('epixm')
for evt in myrun.events():
    img = det.raw.image(evt)
    # check for missing data
    if img is not None:
        print(f'frameNo: {det.raw.frameNo(evt)},',
              f'asicNo: {det.raw.asicNo(evt)},',
              f'autoFillMask: {det.raw.autoFillMask(evt)},',
              f'fixedMask: {det.raw.fixedMask(evt)}')

procmgr stop epixM.cnf