Introduction

AMI may not always contain a box for performing the operation you want. In such cases AMI provides a PythonEditor box which allows users to enter custom python code allowing them to perform their own operations. A PythonEditor box can then be exported into a reusable box and loaded using the Manage Library functionality.

The export feature of PythonEditor will be the preferred method for most users to create a reusable box as it provides a good starting template to further customize a box and add a custom control gui. However, it is also possible to write a custom box from scratch.

PythonEditor

The first and easiest option is to use the PythonEditor box. This box allows a user to enter a custom python function which is applied to the inputs of the box to produce outputs. Each input connection is passed as an argument to the function and the function must return as many outputs as the box has. Inputs and outputs can be added by right clicking on the box and selecting Add input/Add output.  If you add outputs, return all outputs at the end of the function in a line like "return output1, output2".

It is possible to debug these boxes with print statements using "ami-local -b 1 -f interval=1".  The flags are worth considering: they slow down the event rate to 1Hz to avoid too high a rate of print statements.

When initially adding a PythonEditor box you will be prompted to add inputs and outputs. You can set the name of the terminals and their type. The names will be used in the auto-generated code template that appears in the editor window.

After adding inputs and outputs, clicking on the box will bring up the PythonEditor window.

The default code template of the PythonEditor uses a class based approach, defining a class called EventProcessor with methods that execute at the beginning of a run (begin_run), end of a run (end_run), beginning of a step (begin_step), end of a step (end_step), and on each event (on_event). Additionally, there is an empty __init__ method which can be useful for defining member variables which store data across execution of the various methods.  NOTE: if the "on_event" method here has no data to return (e.g. if filtering out events) it is important to return the "None" python object. 

class EventProcessor():

    def __init__(self):
        pass

    def begin_run(self):
        pass

    def end_run(self):
        pass

    def begin_step(self, step):
        pass

    def end_step(self, step):
        pass

    def on_event(self, cspad, *args, **kwargs):

        # return 1 output(s).  NOTE: return "None" if this event has no output (e.g. if filtering)
        return

Exporting a PythonEditor

Once you are done editing the code in the PythonEditor box you might wish to reuse the box in a different flowchart. The export functionality allows users to do that.

Clicking the Export button in the PythonEditor will bring up a window that allows users to set a Name and Docstring for the box. After clicking Ok a file dialog will appear and the box will be exported as a python file that can then be edited and imported using the manage library functionality.

PythonEditor Examples

Gaussian Fit

Example includes:

• Returns gaussian fit every 100 events (PythonEditor.1)

ami-local -f interval=0.1 -f repeat=True -l /cds/group/psdm/tutorials/ami2/tmo/gaussian_fit.fc psana://exp=rixx43518,run=45,dir=/cds/data/psdm/prj/public01/xtc

Writing A Reusable Box

The PythonEditor box is good for quick and dirty things. However, sometimes a user may wish to write a box that is reusable across multiple experiments without having to enter python code every time. AMI exposes an API allowing users to do that. Additionally, users may wish to do something more advanced such as implement their own configuration gui, a custom plot widget, or perform an accumulation, these are not possible to do using the PythonEditor box, but can be done by writing a custom box.

Recommend Approach: Exporting a PythonEditor Box

We strongly recommend writing a PythonEditor box and then using the "Export" button at the top of the PythonEditor window where you type code.  This will prompt you for a "box name" and then the name of a python file to save to.

This file can then be imported into AMI using the "Manage Library" button at the bottom left of the main AMI window:

NOTE: there is an issue if you want to store multiple exported boxes in the same .py "library":  the above "export" procedure creates some identical names for the different boxes in the same file, creating python errors.  So the .py library file must be edited by hand to avoid those name conflicts until we can fix this issue in AMI.

Add "constant" arguments to the box

In addition to the inputs, constant configuration parameters can be set by clicking on a box. For example the ROI parameter in the ROI box.

The following example shows how to add UI-based parameters to your custom node.

from ami.flowchart.library.common import CtrlNode


class EventProcessor():
	# The box implementation
	...


class myNode(CtrlNode):
    
	nodeName = "myNode"
    uiTemplate = [
		('a_random_checkbox', 'check', {'checked': False}), 	# a check box that defaults to False
		('a_random_spinbox', 'doubleSpin'), 					# a constant spinbox
		('a_random_int', 'intSpin', {'value': 10, 'min': 1}), 	# an integer defaulting to 10, with min value 1
		...
	]
    
	def __init__(self, name):
    ...

	def to_operation(self, **kwargs):
        proc = EventProcessor()
        proc.check_box_value = self.values['a_random_checkbox']

        return gn.Map(name=self.name()+"_operation", **kwargs,
                      func=proc.on_event,
                      begin_run=proc.begin_run,
                      end_run=proc.end_run,
                      begin_step=proc.begin_step,
                      end_step=proc.end_step)

• Updated the node to a CtrlNode instead of the simple Node. This is simply done by replacing the parent class Node with CtrlNode (and importing the relevant class).
• UI elements can then be added by using the uiTemplate list in the header portion of the class. The values of the UI elements are then accessible via a self.values dictionary within the class.
• If needed, you can pass these to your EventProcessor instance. It is also generally a good idea to give them default values in the EventProcessor.__init__ method.

Example of more ways of uiTemplate in the AMI repo, for example: https://github.com/slac-lcls/ami/blob/master/ami/flowchart/library/Operators.py

"From Scratch" Approach

(not recommended, except for experts)

NOTE: the nodename below should be a valid python variable name (e.g. no dashes).

To illustrate we look at the implementation of box called MyProjection, which implements a projection.

   import numpy as np
   import ami.graph_nodes as gn
   from ami.flowchart.library.common import CtrlNode
   from amitypes import Array1d, Array2d


   class MyProjection(CtrlNode):                                                                                       
                                                                                                                       
       """                                                                                                             
       Projection projects a 2d array along the selected axis.                                                         
       """                                                                                                             
                                                                                                                       
       nodeName = "MyProjection"                                                                                       
       uiTemplate = [('axis', 'intSpin', {'value': 0, 'min': 0, 'max': 1})]                                            
                                                                                                                       
       def __init__(self, name):                                                                                       
           super().__init__(name, terminals={'In': {'io': 'in', 'ttype': Array2d},                                     
                                             'Out': {'io': 'out', 'ttype': Array1d}})                                  
                                                                                                                       
      def to_operation(self, inputs, outputs, **kwargs):                                                                                
        axis = self.values['axis']                                                                                                                                                                                                            
        def func(arr):                                                                                                 
            return np.sum(arr, axis=axis)                                                                              
                                                                                                                       
        return gn.Map(name=self.name()+"_operation", func=func, inputs=inputs, outputs=outputs, **kwargs)    

Each box in AMI must inherit from the CtrlNode class which is available in the ami.flowchart.library.common module. A box is required to implement a docstring as it used for documentation when hovering in AMI. It must also define a nodeName class variable.

A uiTemplate is optional. If provided the node will auto-generate a configuration gui. The template consists of a list of tuples. The first entry of the tuple is the name of control, the second specifies the type (the list of supported types can be found here), the third entry is an optional dictionary which sets limits and default values. The values of the controls are available through the self.values dictionary as seen on line 21.

The first step to implementing a custom box is to figure out what the inputs and outputs are as seen on lines 16 - 17. The terminals on each box are tagged with a python 3 type annotation. The amitypes package implements support for numpy arrays as types Array1d, Array2d, Array3d, and Array which is a union of all the array types. These types are used to prevent users from making invalid connections between boxes.  For numbers, the standard python types "float" and "int" can be used.

A box must also implement a to_operation() method which takes inputs, outputs, and kwargs arguments. The to_operation method must return either a single node from ami.graph_nodes or a list of nodes. In this case we use a gn.Map node which applies a function called func to the In terminal (the argument arr of func) to return the Out terminal (return np.sum). 

The Map class takes the required keywords:

Various other types of graph nodes are available here. Additionally, there are many other examples of how to implement boxes within the AMI node library available here. 

After implementing a custom box, the box can be imported into AMI by clicking the Manage Library button on the lower left of the editor window. The box will appear at the bottom of the operations list. The path the box was loaded from will be saved when saving the graph and loaded again when reloading the graph. To display the output of MyProjection, drag and drop the WaveformViewer and connect the projection output to the viewer input.

Supported Types

• default python types: list, float, int, dict
• types in amitypes python package: Array1d, Array2d, Array3d
• A type "Any" can be used for more complex objects
• See type annotations described here:  https://docs.python.org/3/library/typing.html

Complexity For "Reusable Box" Pattern

• if you have a class or function that uses cython you have to create a wrapper class. Don't create cython class directly, pass arguments needed to create the cython instance to the wrapper, and the wrapper needs a __call__ method that instantiates the cython class.  Only create this instance on the first event (make the wrapper class None to start).  See HexAnode and DLDProc classes here: https://github.com/slac-lcls/ami/blob/master/ami/flowchart/library/Psalg.py. The reason this is necessary is that whatever is returned from the to_operation method must be picklable.