...

By default the core application options are read from {{\[pyana\]}} section of the configuration file. If the option {{\-C _name_}} or {{\-\-config-name=_name_}} is given on the command line then additional section {{\[pyana._name_]\}} is read and values in that section override values from {{\[pyana\]}} section.

...

...

[pyana]
modules = mypackage.myana mypackage.myana:wide

[mypackage.myana]
lower = 0
upper = 100
name = default

[mypackage.myana:wide]
; 'lower' option will be reused from [mypackage.myana] section
bins = 1000   ; this overrides default module value
; two options below will override [mypackage.myana] values
upper = 1000
name = wide

Analysis Tools

Initially there is only a small set of the analysis tools available to the user modules, but with time it will certainly grow. this section describes all the tools that we have today.

Writing Files

one of the simplest tasks that can be done is to write a subset of data to a file and then analyze those data with some external application such as Matlab. Writing data to files in Python is sufficiently simple once you know the format of the in-memory data and output data. If output data needs to be in ASCII format one can use Python print statement to format the data and write them to file. For binary data things could become more involved but there are multiple modules that deal with this problem such as struct and array Python modules.

One significant complication comes from the multi-processing capabilities of Pyana. With multi-processing enabled jobs runs in many processes with each process analyzing only a subset of the data set. At the end of the job the output files from all independent processes needs to be merged into a single file. Depending on the format of the output files merging can be either very easy, or very hard, or impossible. Pyana supports one simple merging mechanism for files when the files from all processes are copied into a single output file, very much like 'cat file1 ... fileN > file' command does. The order in which files are copied is not specified, so if the order is important some additional processing may be required. To enable Pyana merging mechanism one needs to use a special construct when opening output file from an analysis code. Instead of plain open(...) or file(..) functions one needs to use env.mkfile(...) method with the same arguments. In this call a temporary file will be created somewhere (most likely in /tmp directory) with a unique name. The function returns a regular Python file object which can be used with all standard tools. At the end of the job Pyana will collect the names of those temporary files and merge them together into one file with the same name as was given to env.mkfile(...) deleting all temporary files. this special method is safe to use even when running in a single-process mode in which case it is equivalent to regular open(...) method so there is no unnecessary copy involved.

When this simple merging procedure is not sufficient users need to do merging themselves (if at all possible). In that case the file in every process should be opened in a standard way using open(...) but the name of every file should be made unique. One simple way to generate a process-specific name is to use method env.jobNameSub() which returns a regular job name followed by dash and a process number ranging from 0 to a number of sub-processes. When this method is called in single-process job it returns regular job name. User can also construct arbitrary process-specific file name using the sub-process ID returned from env.subprocess() method.

...