...

Fundamental Numbers

• The accelerator provides exactly 910,000 buckets in exactly 0.98 seconds
• The accelerator patterns repeat on this 0.98s interval
• The ratio of these two numbers is what gives rise to the 929kHz (~1MHz) machine rate.  Equivalently: accelerator rate is 13/14MHz, bucket period is 14/13us.
• timing eventcodes arrive ~100us prior to beam
• The 71428Hz rate is special: common between LCLS1 and LCLS2

Presentation

Recording of Matt's presentation on the timing system.

Link to Matt's presentation.

A presentation from Matt:

https://docs.google.com/presentation/d/1QumkWlhA9Q8PW40hlyMkr0HZwBB4UBOK/edit?usp=sharing&ouid=118434024476101633307&rtpof=true&sd=true

Patterns

It's a common question to ask what are the rates that the accelerator can generate.  The table below lists the rates that are natural for the accelerator.  These are the periodic rates that repeat exactly at the ~0.98 second period of the timing/MPS system.  Of course, we can generate other rates as well that aren't exactly periodic by doing the sort of thing you described - dropping shots or irregular sequences.  This has been useful for the OPCPA laser development in deciding the exact frequencies they will support.

If a base accelerator rate is chose (e.g. 33kHz) then other rates having the same prime factors as the accelerator rate are allowed:

(python3_env) [weaver@lcls-dev3 lcls2-timing-patterns]$ python lcls/fixed_rate_table.py

Matt provides this example of how you can determine if various trigger rates overlap: 16kHz (16581 in the table) has factors (2,2,2,7).  102Hz has factors (2,2,5,5,7,13).  It's missing one factor of 2, so it only overlaps 1/2 of the time.  51Hz has (2,2,2,5,5,7,13), so it contains all the factors from 16kHz and thus overlaps 100%.

If a base accelerator rate is chosen (e.g. 33kHz) then other rates having the same prime factors as the accelerator rate are allowed, according to this table:

(python3_env) [weaver@lcls-dev3 lcls2-timing-patterns]$ python lcls/fixed_rate_table.py

 rate, Hz  | factor | factors
 928571          1   1
 464285 rate, Hz  | factor | factors
 928571          1   1
 464285          2   (2,)
 232142          4   (2, 2)
 185714          5   (5,)
 132653          7   (7,)
 116071          8   (2, 2, 2)
  92857         10   (2, 5)
  71428         13   (13,)
  66326         14   (2, 7)
  58035         16 2   (2, 2, 2, 2)
  46428 232142         20 4   (2, 2, 5)
  37142 185714         25 5   (5, 5)
  132653 35714         26 7   (27, 13)
  33163 116071         28 8   (2, 2, 72)
  2653092857         3510   (52, 75)
  2321471428         4013   (2, 2, 2, 513,)
  1857166326         5014   (2, 5, 57)
  1785758035         5216   (2, 2, 2, 132)
  1658146428         5620   (2, 2, 2, 75)
  1428537142         6525   (5, 135)
  1326535714         7026   (2, 5, 713)
  1160733163         8028   (2, 2, 2, 2, 5)
  10204 7)
  26530           9135   (75, 137)
  23214  9285         10040   (2, 2, 52, 5)
   892818571          10450   (2, 25, 2, 135)
   829017857          11252   (2, 2, 2, 2, 713)
  16581  7428         12556   (52, 2, 52, 57)
   714214285        130   (2, 65   (5, 13)
  13265  6632         14070   (2, 2, 5, 7)
  11607  5306         17580   (52, 2, 2, 52, 75)
  10204  5102         18291   (2, 7, 13)
   4642 9285        200 100   (2, 2, 2, 5, 5)
   4464 8928        208 104   (2, 2, 2, 2, 13)
   3714 8290        250 112   (2, 52, 2, 52, 57)
   3571 7428        260 125   (25, 25, 5, 13)
   3316 7142        280 130   (2, 2, 2, 5, 713)
   2857 6632        325 140   (52, 2, 5, 137)
   2653 5306        350 175   (2, 5, 5, 7)
   2551 5102        364 182   (2, 2, 7, 13)
   2321 4642        400 200   (2, 2, 2, 2, 5, 5)
   2040 4464        455 208   (52, 2, 2, 72, 13)
   1857 3714        500 250   (2, 2, 5, 5, 5)
   1785 3571        520 260   (2, 2, 2, 5, 13)
   1658 3316        560 280   (2, 2, 2, 2, 5, 7)
   1485 2857        625 325   (5, 5, 5, 513)
   1428 2653        650 350   (2, 5, 5, 137)
   1326 2551        700 364   (2, 2, 57, 5, 713)
   1275 2321        728 400   (2, 2, 2, 72, 5, 135)
   1061 2040        875 455   (5, 57, 5, 713)
   1020 1857        910 500   (2, 2, 5, 75, 135)
    1785 928       1000 520   (2, 2, 2, 5, 5, 513)
    1658 892       1040 560   (2, 2, 2, 2, 5, 137)
    742 1485       1250 625   (2, 5, 5, 5, 5)
    1428 714       1300 650   (2, 2, 5, 5, 13)
    1326 663       1400 700   (2, 2, 2, 5, 5, 7)
    1275 637       1456 728   (2, 2, 2, 2, 7, 13)
    571 1061       1625 875   (5, 5, 5, 137)
    1020 530       1750 910   (2, 5, 57, 5, 713)
    510928       18201000   (2, 2, 2, 5, 75, 135)
    464892       20001040   (2, 2, 2, 2, 5, 5, 513)
    408742       22751250   (2, 5, 5, 75, 135)
    371714       25001300   (2, 2, 5, 5, 5, 513)
    357663       26001400   (2, 2, 2, 5, 5, 137)
    331637       28001456   (2, 2, 2, 2, 57, 5, 7)13)
    285571       32501625   (2, 5, 5, 5, 13)
    265530       35001750   (2, 2, 5, 5, 5, 7)
    255510       36401820   (2, 2, 2, 5, 7, 13)
    212464       43752000   (52, 2, 2, 52, 5, 5, 75)
    204408       45502275   (2, 5, 5, 7, 13)
    185371       50002500   (2, 2, 2, 5, 5, 5, 5)
    178357       52002600   (2, 2, 2, 2, 5, 5, 13)
    142331       65002800   (2, 2, 2, 52, 5, 5, 137)
    132285       70003250   (2, 2, 2, 5, 5, 5, 713)
    127265       72803500   (2, 2, 25, 25, 5, 7, 13)
    114255       81253640   (52, 2, 52, 5, 57, 13)
    106212       87504375   (2, 5, 5, 5, 5, 7)
    102204       91004550   (2, 2, 5, 5, 7, 13)
     92185        100005000   (2, 2, 2, 2, 5, 5, 5, 5)
     81178        113755200   (52, 52, 2, 2, 5, 75, 13)
    142  71       130006500   (2, 2, 2, 5, 5, 5, 13)
     66132        140007000   (2, 2, 2, 2, 5, 5, 5, 7)
     57127        162507280   (2, 2, 52, 52, 5, 57, 13)
    114  53       175008125   (2, 2, 5, 5, 5, 5, 713)
     51106        182008750   (2, 25, 25, 5, 5, 7, 13)
     40102        227509100   (2, 52, 5, 5, 7, 13)
     35 92      26000 10000   (2, 2, 2, 2, 5, 5, 5, 135)
     28 81      32500 11375   (2, 2, 5, 5, 5, 57, 13)
     26 71      35000 13000   (2, 2, 2, 5, 5, 5, 5, 713)
     25 66      36400 14000   (2, 2, 2, 2, 5, 5, 75, 137)
     20 57      45500 16250   (2, 25, 5, 5, 5, 7, 13)
     16 53      56875 17500   (2, 2, 5, 5, 5, 5, 7, 13)
     14 51      65000 18200   (2, 2, 2, 5, 5, 57, 5, 13)
     13 40      70000 22750   (2, 2, 2, 2, 5, 5, 5, 57, 713)
     10 35      91000 26000   (2, 2, 2, 52, 5, 5, 75, 13)
      8 28     113750 32500   (2, 52, 5, 5, 5, 75, 13)
      26 7     130000 35000   (2, 2, 2, 2, 5, 5, 5, 5, 137)
      5 25     182000 36400   (2, 2, 2, 2, 5, 5, 5, 7, 13)
      4 20     227500 45500   (2, 2, 5, 5, 5, 5, 7, 13)
     16   2   56875   (5, 5, 5, 5, 7, 13)
     14     455000 65000   (2, 2, 2, 5, 5, 5, 5, 7, 13)
      13 1     910000 70000   (2, 2, 2, 2, 5, 5, 5, 5, 7, 13)

Event Codes

• 288 bits of "event codes":
  • some have well-defined meanings (in progress)
  • 16 highest bits are hutch specific for sequences
  • DAQ readout groups are "extra bits" included at end of timing frame
• timing frames have destinations: e.g. bykiks, and bykikh both go to "bsy" dump
• we will not be using event-codes for bykiks: use destinations instead

Rates

• timing arrives 100us prior to beam
• accelerator max rate is 13/14MHz=929kHz
• The 71428Hz rate is special: common between LCLS1 and LCLS2
• 928kHz is max rate, but accelerator pattern repeats every 910k buckets

Trigger Types

...

• sequences
• ac-line rate (1,5,10,30Hz...)
• fixed-rate patterns (1, 10, 100, 1000, 10000, 71kHz, 929kHz)

...

)
     10      91000   (2, 2, 2, 5, 5, 5, 7, 13)
      8     113750   (2, 5, 5, 5, 5, 7, 13)
      7     130000   (2, 2, 2, 2, 5, 5, 5, 5, 13)
      5     182000   (2, 2, 2, 2, 5, 5, 5, 7, 13)
      4     227500   (2, 2, 5, 5, 5, 5, 7, 13)
      2     455000   (2, 2, 2, 5, 5, 5, 5, 7, 13)
      1     910000   (2, 2, 2, 2, 5, 5, 5, 5, 7, 13)

Triggering Devices With Beam

To trigger on "every shot" with beam (the equivalent of LCLS1's eventcode 140) use "Mode=FixedRate:1MHz, Destination=Include:DumpSXR" (without the "destination" field I believe you will just get a pure 1MHz trigger).  See table below where DumpSXR destination corresponds to destination bit 4.   It is also important to set the "select" field to "Inclusive" (see below for daq configuration object GUI).

Another important method (used especially for low rate 1,10,100Hz triggers that are in-synch with the beam) is to use an event-code generated by ACR (see Control Sequence Bit for currently supported event codes).  cpo thinks event codes like "SC_SXR BSA" (e.g. 100Hz is eventcode 30) on that page would be commonly used by the hutches. NOTE: We have seen empirically that these event codes do not include a destination setting, so it is necessary to also select destination 4 with the "select" field set to "inclusive" with them.  Matt writes: "Event code 30 will follow the offset used by SXR, but it isn't aware of MPS mitigating the beam rate.  We use the destination 4 filter to also pickup when MPS has inhibited the beam."

Image Added

Some details:

• there are 288 bits of "event codes" available
  • some have well-defined meanings, like the low-rate ones described above (in progress)
  • 16 highest bits are hutch specific for sequences (272-287)
  • DAQ readout groups are "extra bits" included at end of timing frame
• timing frames have "destinations": e.g. bykiks, and bykikh both go to "bsy" dump
• unlike LCLS1 we will not be using event-codes to understand when bykiks has fired: use destinations instead, in particular DumpBSY for "dropped" or "background" shots
• some devices reference to "sequencer engineer number" and "sequence bit number" instead of "eventcode".  The formula to convert between the two: eventcode=(sequenceEngine#<<16 | sequencerBit#).

Trigger Destinations and Types

Timing receivers share a common logic module for generating triggers.  They consist of the logical AND of two components, a rate component and a beam destination component.

• Rate component.
  • sequences / event code
  • ac-line rate (1,5,10,30Hz...) + timeslot mask
  • fixed-rate patterns (1H, 10H, 100H, 1kH, 10kH, 71kH, 1MH)
• Beam destination component
  • "Dont Care" : beam or no beam, we don't care
  • "Inclusive" : specify a set/mask of destinations to which the beam must be destined; see the table below
  • "Exclusive" : specify a set/mask of destinations to which the beam must not be destined; see the table below.

Note: destination "DumpBSY" corresponds to dropped-shots.

Need to coordinate hutch sequence-pattern scripts with accelerator sequence-pattern (patterns repeat on 1H marker).

Sequencing Example

See https://github.com/slac-lcls/lcls2/blob/master/psdaq/psdaq/seq/rixexample.sh (high level example for bursts and period patterns) and Integrating Detectors in RIX.  See also Matt's documentation.

In this example the XPM is configured to insert event codes 272-287 into the timing stream.  Four event codes are generated by each sequence engine within the XPM.  The XPM has 4 sequence engines allowing 16 event codes total.

A current example for doing this is done by executing:

(first use Matt's high-level utility to generate two event codes that run at 10Hz and 100Hz (periods 91000 and 9100 respectively)

(ps-4.5.26) drp-srcf-mon001:psana$ periodicgenerator -p 91000 9100 -s 0 0 -d '10Hz' '100Hz' >& 10hz_100hz.py

(now program sequence engine 0 and start it using the python code generated above. since we're programming engine 0, the first two event codes will be 272,273 according to the above table)

(this command must be executed on a node that can access the XPM PV's, e.g. drp-srcf-*)

(ps-4.5.26) drp-srcf-mon001:psana$ seqprogram --seq 4:10hz_100hz.py --pv DAQ:NEH:XPM:3 --start
** engine 0 fname 10hz_100hz.py **
descset  None
seqcodes {0: '10Hz', 1: '100Hz'}
Remove -1
idx 2
Removing seq 2
[10, 1, 5, 3, 0, 0, 0, 0, 2, 0, 6, 2048, 0, 0, 0, 3, 2, 1, 3, 3, 0, 0, 2, 0, 6, 908, 0, 0, 0, 1, 5, 2, 0, 0, 0, 0, 3, 2, 1, 0, 8, 0, 0, 2, 0, 6, 2048, 0, 0, 0, 3, 2, 6, 3, 3, 0, 0, 2, 0, 6, 908, 0, 0, 0, 1, 2, 0, 0, 0, 0, 0]
Confirmed ninstr 10
Sequence  found at index 2
desc ['10Hz', '100Hz', '', '', '', '', '', '', '', '', '', '', 'burst', '', '', '']
seqcodes_pv epics:nt/NTTable:1.0 
    string[] labels [EventCode, Description, Rate, Enabled]
    structure value
        int[] EventCode [272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287]
        string[] Description [10Hz, 100Hz, , , , , , , , , , , burst, , , ]
        int[] Rate [11,102,0,0,0,0,0,0,0,0,0,0,0,0,0,0]
        byte[] Enabled [1,1,1,1,0,0,0,0,0,0,0,0,1,1,1,1]
    string descriptor 
    alarm_t alarm
        int severity 0
        int status 0
        string message 
    time_t timeStamp
        long secondsPastEpoch 1683163040
        int nanoseconds 339389440
        int userTag 0

(now query the xpm to see what is running in the 4 sequence engines, each having 4 event codes)

(ps-4.5.26) drp-srcf-mon001:psana$ pvget DAQ:NEH:XPM:3:SEQCODES
DAQ:NEH:XPM:3:SEQCODES 2023-05-03 18:17:31.339    
EventCode Description Rate Enabled
      272        10Hz   10       1
      273       100Hz  102       1
      274                0       1
      275                0       1
      276                0       0
      277                0       0
      278                0       0
      279                0       0
      280                0       0
      281                0       0
      282                0       0
      283                0       0
      284       burst    0       1
      285                0       1
      286                0       1
      287                0       1
(ps-4.5.26) drp-srcf-mon001:psana$ 

Note: the periodicgenerator "repeat" argument specifies the number of repeats of the slowest event-code.

There is a similar "traingenerator" script which you can read about in Matt's documentation.

Visualizing Sequences

This can be done with the same environment used to generate the sequences above.

cd ~tmoopr/daq
source setup_env.sh
seqplot --seq 0:codes.py 3:beam.py --time 2.0

codes.py and beam.py were created with these scripts in the psdaq package:

traingenerator -s 14000 -b 28 -n 32001 -r 2 -d "burst" -t 910000 > beam.py
periodicgenerator -p 91 91000 -s 0 0 -d '10kHz' '10Hz' --repeat 3 --notify > codes.py

The seqplot command brings up this window which shows the two periodic event codes (programmed into the first simulated sequence engine) and the one "beam burst" event code (programmed into the fourth simulated sequence engine):

Image Added

Sequence Start/Stop

Some technical details from a discussion on March 23, 2023

Since a sequence in a lower-level XPM can "overwrite" sequence eventcodes from a higher-level XPM it is important to be able to stop lower-level sequences.  Matt says one does this by setting DAQ:NEH:XPM:2:SEQENG:3:ENABLE (for example) to 0.  The current example where we have to careful of this is for XPM0 which the laser group uses for eventcodes 272,273,274.

Matt has added a Groups/EventCodes tab to the xpmpva tool.  It shows you the effect of overwriting and which xpm is the current source of eventcodes.

currently the XPM starts sequences on 1Hz boundary (e.g. every 910000 buckets). this slows down the daq scan steps, so support for a second mode is being considered:

(1) current seq mode (only program time-in-seconds or numL1Accepts)
    have to coordinate the start of an accelerator burst with the daq sequence
    - are there two sequences?  one for accelerator and one for daq?
      not clear how much control the accelerator will want (e.g. burst
      all the time for stability, but silke says can damage samples)
    - each step we start one sequence:
      o may also include control of accelerator, but if not
      o have to coordinate the start of the daq seq with accelerator seq
        o acc seq synced to 1Hz marker (everything repeats at 1Hz on
      accelerator side)
    o trickier to sync to rates faster than 1Hz
(2) future-firmware seq mode
    - make the daq enable/disable on a user-defined event code
      o would be faster on the enable since don't have to wait for 1Hz
        marker.  still have to understand the periodicity of the accelerator
    so the user-defined event codes get sent down at the right time
    but it does mean we don't need the slow 1Hz marker.  some things like
    laser electronic delay scanning may be faster than 1Hz so could benefit
    from not using the 1Hz marker.

andor integrates over 10 shots
6 andor images in a step

mode (1):
- program 6 andor-readout-group "L1Accepts".  might be set to the parent
  group now in control.py

the readout group that is counted for the sequence is currently managed
by these lines in control.py.  Currently defaults to the "platform" readout
group which is the primary readout group: need to make this more flexible
so we can change it to the integrating detector readout group:

self.pv_xpm_part_base = pv_base + ':XPM:%d:PART:%d' % (xpm_master, platform)
self.pva.pv_put(self.pva.pvStepEnd, self.readoutCumulative)

Generating a Low Rate Sequence

From Matt.

periodicgenerator --period 1820000 --start_bucket 91000 --description 'Example 0.5Hz sequence' > 0.5Hseq.py
(ps-4.6.1) bash-4.2$ seqplot --seq 0:0.5Hseq.py --time 4.0
eng 0 fn 0.5Hseq.py
0: FixedRateSync(929kHz) # occ(2048)
1: Branch to line 0 until ctr3=43
2: FixedRateSync(929kHz) # occ(888)
3: ControlRequest word 0x1 [0]
4: FixedRateSync(929kHz) # occ(2048)
5: Branch to line 4 until ctr3=843
6: FixedRateSync(929kHz) # occ(488)
7: Branch unconditional to line 0
start, stop: 0,3640000
engine exited request 0  instr 1  returnaddr None  frame 3642047  ccnt [0, 0, 0, 0]

Should show eventcode 272 at bucket 91,000 and every 1,820,000 (2 seconds) after that.

Image Added