Background

When setting up AMI graphs one of the more complex issues is gathering results from the multiple nodes/cores that are receiving the events.  This gather is done at a "heartbeat" rate (typically 1-10Hz) that uses the event timestamps to determine if enough time has passed so that a gather operation should happen.

...

Global Operations Commonly Used By Users

Average0D, Average1D, Average2D

Will average event data either over an infinite time ("infinite" checked in configuration) or a settable finite number of events (generated internally using the SumN pattern).  For the finite case be careful not to set N too large as it can consume a lot of memory.  The PickN produces an array of numbers/1D-arrays/2D-arrays which are, respectively, 1D/2D/3D arrays (with "time" being the added dimension).  AMI allows you to average over any of those axes, but users will typically want to average over axis=0 ("time"), unless you want to see an average projection (axis=1 or 2) vs. time.

This pattern is similar to an MPI "reduce" operation.

Binning, Binning2D

MeanVsScan, MeanWaveformVsScan

Global Operations Typically Used Internally By AMI

PickN

Use-case summary: Use PickN when you need to gather a precise number of events from multiple workers.  It is unusual for users to directly use this pattern: it is typically only used internally by AMI.  The ami graph output will update only when N events have been collected.  Note that there is some arithmetic rounding depending on the number of events requested and the number of workers running parallel.  This pattern (and RollingBuffer, below) are similar.

...

This pattern is similar to an MPI "gather" operation (gather a list of items).

RollingBuffer

Use-case summary: similar to PickN.  Use RollingBuffer like a circular buffer (or first-in-first-out buffer) that keeps a finite number of events.  Unlike PickN the ami graph output updates every event.

...

globalCollector: [6, 7, 8, 9, 10, 2, 3, 4, 5, 11]

Accumulator

An accumulator takes two functions as an argument, a reduction and a res_factory that is called on reset. It accumulates into self.res on each event in the following way: self.res = reduction(self.res, *args) at the end of a heartbeat workers and local collectors reset self.res by calling self.res_factory.

...

https://github.com/slac-lcls/ami/blob/master/ami/flowchart/library/Numpy.py#L506-L528

ReduceByKey

ReduceByKey takes a key and value and reduction function and reduces into a dictionary in the following way: if key in self.res: self.res[key] = reduction(self.res[key], value) else: self.res[key] = value

...

https://github.com/slac-lcls/ami/blob/master/ami/flowchart/library/Operators.py#L149-L184

Accumulator Example

Example includes:

• ReduceByKey
• Accumulator
• RollingBuffer
• PickN

...