Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

For quick example suppose that we have this class defined in user module:

Code Block
none
none
bgColor#FFFFEE
titleBGColor#FFFFDD
titlemypackage/src/myana.py
borderStylesolid
# user analysis class
class myana(object):
    def __init__(self, name, lower, uppper, bins=100)
        self.lower = float(lower)
        self.upper = float(upper)
        self.bins = int(bins)
    ...

and this job configuration file:

Code Block
bgColor
none
none#FFFFEE
titleBGColor#FFFFDD
titlepyana.cfg
borderStylesolid
[pyana]
modules = mypackage.myana mypackage.myana:wide

[mypackage.myana]
lower = 0
upper = 100
name = default

[mypackage.myana:wide]
lower = 0
upper = 1000
bins = 1000
name = wide

With this the analysis job will instantiate two analysis objects with different parameters, equivalent to this pseudo-code:

Code Block
none
nonebgColor#FFFFEE
titleBGColor#FFFFDD
borderStylesolid
# import class myana
from mypackage.myana import myana

# create instances
instances = [ myana(lower = "0", upper = "100", name = "default"),
    myana(lower = "0", upper = "1000", bins = "1000", name = "wide") ]

...

Short

Long

Config File

Option type

Default

Description

-v

--verbose

verbose

integer

0

Command line options do not need any values but can be repeated multiple times, configuration file option accepts single integer number.

-c file

--config=file

 

path

pyana.cfg

Name of the configuration file.

<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="f927dfc127d06265-36132f3a-4bef4c5f-9398b4f6-6046e4385f8bbd7c69f74be3"><ac:plain-text-body><![CDATA[

-C name

--config-name=name

 

string

 

If non-empty string is given then configuration will be read from section [pyana.name] in addition to [pyana].

]]></ac:plain-text-body></ac:structured-macro>

-l file

--file-list=file

file-list

path

 

The list of input data files will be read form a given file which must contain one file name per line.

-n number

--num-events=number

num-events

integer

0

Maximum number of events to process, this counter will include damaged events too.

-j name

--job-name=name

job-name

string

 

Sets job name which is accessible to user code via environment method. Default name is based on the input file names.

-m name

--module=name

modules

string

 

User analysis module(s). Command line options can be repeated several times, configuration file option accepts space-separated list of names.

-p number

--num-cpu=number

num-cpu

integer

1

Number of processes to run, if greater than 1 then multi-processing mode will be used.

...

Here is an almost identical example from Initialization section above which illustrates the inheritance and overriding of the user options:

Code Block
bgColor
none
none#FFFFEE
titleBGColor#FFFFDD
titlepyana.cfg
borderStylesolid
[pyana]
modules = mypackage.myana mypackage.myana:wide

[mypackage.myana]
lower = 0
upper = 100
name = default

[mypackage.myana:wide]
; 'lower' option will be reused from [mypackage.myana] section
bins = 1000   ; this overrides default module value
; two options below will override [mypackage.myana] values
upper = 1000
name = wide

...

Here is a brief example of booking and filling of few histograms in user module:

Code Block
none
nonebgColor#FFFFEE
titleBGColor#FFFFDD
titlemypackage/src/myana.py
borderStylesolid
# user analysis class
class myana(object):
    def beginjob(self, event, env):
        # get histogram manager
        hmgr = env.hmgr()

        # book histograms, store references
        self.hist1 = hmgr.h1d('energy', 'Energy Distribution', 100, 0., 1.)
        self.hist2 = hmgr.h2d('energy vs something', 'Histo2 title', 100, 0., 1., 100, -20., 20.)

    def event(self, event, env):
        # fill histograms
        energy = ...
        something = ...
        self.hist1.Fill(energy)
        self.hist2.Fill(energy, something)

...

Here is very brief example of using SciPy for integration of experimental data:

Code Block
none
none
bgColor#FFFFEE
titleBGColor#FFFFDD
titlemypackage/src/myana.py
borderStylesolid
from scipy import integrate

class myana(object):

    def event(self, event, env):
        # Get Acqiris waveform object
        ddesc = evt.getAcqValue( "AmoITof", channel, env )
        wf = ddesc.waveform()
        ts = ddesc.timestamps()

        # integrate it using Simpson's rule
        integral = integrate.simps (wf, ts)

...

The framework can be run in single-process or multi-process mode, by default everything runs in singe-process mode. In multi-process mode analysis job spawns a number of processes all running on the same host. In that case framework is also responsible for distributing individual events to a single or multiple sub-processes and collecting and merging the results of the processing at the end of the job.

Framework handles few data types specially. For example EPICS data which is a part of the vent data does not appear in every L1Accept transition but every sub-process needs to have an access to current value of EPICS variable. So for EPICS the framework reads EPICS data from every event and accumulates current state in a separate structure. This structure is made available to all sub-processes as a part of the environment so the sub-processes need to access EPICS data through the environment and not reply on event data.

Every sub-process receives only a subset of all L1Accept transitions. Right now there is no control of the sequence and order of the events distributed to individual sub-processes. This fact could affect the type of the analysis that could be done on the data (e.g. event-to-event correlation may be hard to implement) and also the type of the result that analysis can produce in multi-processing mode. At the end of the analysis job the results of individual sub-processes may need to be merged together to produce a single result of the whole job. Frameworks currently knows how to merge two types of data:

  • data written to a file which has been open with env.mkfile() method, the files from sub-processes will be concatenated together in no specific order
  • histograms created by the histogram manager

For other types of the data it would be a user responsibility to store it in some location and then do manual merging after the job is finished.

Writing User Modules

Preferred way to run user analysis code is to create a separate package for each user and store all user modules in src directory in that package. If you plan to commit you code to repository then the package name must be unique and probably include your user name (or experiment name). To create an empty package run this command (it implies that analysis environment has been set):

No Format

% newpkg expname_ana
% mkdir expname_ana/src

Replace expname_ana with the true name of the package. This will create a directory expname_ana with a special SConscript file and doc/ subdirectory containing README and ChangeLog files.

There is a special template for user analysis modules and to create a module based on that template one can run the command:

No Format

% codegen -l pyana-module -o expname_ana/src expname_ana my_ana

Replace expname_ana with the name of the package you created above, and my_ana with the name of the module. The command will create a file my_ana.py in the directory expname_ana/src which will contain basic structure of the analysis module, you will only need to fill it with some code.

There are few simple examples of the user analysis modules located in the examples directory of pyana package. These can be used as a basis for writing your own modules. To see the examples extract pyana package:

No Format

% addpkg pyana

and browse through the examples directory for *.py and *.cfg files.