Page History
...
For quick example suppose that we have this class defined in user module:
Code Block | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||
# user analysis class class myana(object): def __init__(self, name, lower, uppper, bins=100) self.lower = float(lower) self.upper = float(upper) self.bins = int(bins) ... |
and this job configuration file:
Code Block | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||
[pyana] modules = mypackage.myana mypackage.myana:wide [mypackage.myana] lower = 0 upper = 100 name = default [mypackage.myana:wide] lower = 0 upper = 1000 bins = 1000 name = wide |
With this the analysis job will instantiate two analysis objects with different parameters, equivalent to this pseudo-code:
Code Block | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
# import class myana from mypackage.myana import myana # create instances instances = [ myana(lower = "0", upper = "100", name = "default"), myana(lower = "0", upper = "1000", bins = "1000", name = "wide") ] |
...
Short | Long | Config File | Option type | Default | Description | ||
---|---|---|---|---|---|---|---|
-v | --verbose | verbose | integer | 0 | Command line options do not need any values but can be repeated multiple times, configuration file option accepts single integer number. | ||
-c file | --config=file |
| path | pyana.cfg | Name of the configuration file. | ||
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="f927dfc127d06265-36132f3a-4bef4c5f-9398b4f6-6046e4385f8bbd7c69f74be3"><ac:plain-text-body><![CDATA[ | -C name | --config-name=name |
| string |
| If non-empty string is given then configuration will be read from section | ]]></ac:plain-text-body></ac:structured-macro> |
-l file | --file-list=file | file-list | path |
| The list of input data files will be read form a given file which must contain one file name per line. | ||
-n number | --num-events=number | num-events | integer | 0 | Maximum number of events to process, this counter will include damaged events too. | ||
-j name | --job-name=name | job-name | string |
| Sets job name which is accessible to user code via environment method. Default name is based on the input file names. | ||
-m name | --module=name | modules | string |
| User analysis module(s). Command line options can be repeated several times, configuration file option accepts space-separated list of names. | ||
-p number | --num-cpu=number | num-cpu | integer | 1 | Number of processes to run, if greater than 1 then multi-processing mode will be used. |
...
Here is an almost identical example from Initialization section above which illustrates the inheritance and overriding of the user options:
Code Block | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||
[pyana] modules = mypackage.myana mypackage.myana:wide [mypackage.myana] lower = 0 upper = 100 name = default [mypackage.myana:wide] ; 'lower' option will be reused from [mypackage.myana] section bins = 1000 ; this overrides default module value ; two options below will override [mypackage.myana] values upper = 1000 name = wide |
...
Here is a brief example of booking and filling of few histograms in user module:
Code Block | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||
# user analysis class class myana(object): def beginjob(self, event, env): # get histogram manager hmgr = env.hmgr() # book histograms, store references self.hist1 = hmgr.h1d('energy', 'Energy Distribution', 100, 0., 1.) self.hist2 = hmgr.h2d('energy vs something', 'Histo2 title', 100, 0., 1., 100, -20., 20.) def event(self, event, env): # fill histograms energy = ... something = ... self.hist1.Fill(energy) self.hist2.Fill(energy, something) |
...
Here is very brief example of using SciPy for integration of experimental data:
Code Block | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||
from scipy import integrate class myana(object): def event(self, event, env): # Get Acqiris waveform object ddesc = evt.getAcqValue( "AmoITof", channel, env ) wf = ddesc.waveform() ts = ddesc.timestamps() # integrate it using Simpson's rule integral = integrate.simps (wf, ts) |
...
The framework can be run in single-process or multi-process mode, by default everything runs in singe-process mode. In multi-process mode analysis job spawns a number of processes all running on the same host. In that case framework is also responsible for distributing individual events to a single or multiple sub-processes and collecting and merging the results of the processing at the end of the job.
Framework handles few data types specially. For example EPICS data which is a part of the vent data does not appear in every L1Accept transition but every sub-process needs to have an access to current value of EPICS variable. So for EPICS the framework reads EPICS data from every event and accumulates current state in a separate structure. This structure is made available to all sub-processes as a part of the environment so the sub-processes need to access EPICS data through the environment and not reply on event data.
Every sub-process receives only a subset of all L1Accept transitions. Right now there is no control of the sequence and order of the events distributed to individual sub-processes. This fact could affect the type of the analysis that could be done on the data (e.g. event-to-event correlation may be hard to implement) and also the type of the result that analysis can produce in multi-processing mode. At the end of the analysis job the results of individual sub-processes may need to be merged together to produce a single result of the whole job. Frameworks currently knows how to merge two types of data:
- data written to a file which has been open with
env.mkfile()
method, the files from sub-processes will be concatenated together in no specific order - histograms created by the histogram manager
For other types of the data it would be a user responsibility to store it in some location and then do manual merging after the job is finished.
Writing User Modules
Preferred way to run user analysis code is to create a separate package for each user and store all user modules in src
directory in that package. If you plan to commit you code to repository then the package name must be unique and probably include your user name (or experiment name). To create an empty package run this command (it implies that analysis environment has been set):
No Format |
---|
% newpkg expname_ana
% mkdir expname_ana/src
|
Replace expname_ana
with the true name of the package. This will create a directory expname_ana
with a special SConscript
file and doc/
subdirectory containing README
and ChangeLog
files.
There is a special template for user analysis modules and to create a module based on that template one can run the command:
No Format |
---|
% codegen -l pyana-module -o expname_ana/src expname_ana my_ana
|
Replace expname_ana
with the name of the package you created above, and my_ana
with the name of the module. The command will create a file my_ana.py
in the directory expname_ana/src
which will contain basic structure of the analysis module, you will only need to fill it with some code.
There are few simple examples of the user analysis modules located in the examples
directory of pyana
package. These can be used as a basis for writing your own modules. To see the examples extract pyana
package:
No Format |
---|
% addpkg pyana
|
and browse through the examples
directory for *.py
and *.cfg
files.