Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

We have considered using ROOT to power psana, but there are a few issues with ROOT; for example, it tends to take over your application (eg by handling signals), any one library tends to bring in many other libraries non strictly required, it has a few global names which can collide with users symbols, its development is heavily driven by LHC oriented needs and it uses C++ as scripting language which may not be the best solution for many FEL users.

We've been considering adopting something other than ROOT for a while, but we haven't found any serious alternative. We claim that the key services our users would like are:

  • Ability to use both C++ and python (when performance allows).
  • Ability to plot histograms, xy plots, images, functions, etc.
  • Ability to fit complex data.

...

.

...

  • Ability to invoke high performance algorithms; these can range from basic algorithms, like FFTs, to more specialized operations, like peak finding.

There are many tools for plotting (see, for example, Mantid, or matplotlib), for saving and retrieving data (HDF5), for fitting (minuit2), for GUI development (QT), for basic algorithms (GSL) but no integrated solution.

We'd like to start the effort of creating an integrated ecosystem for data analysis for FEL facilities. Note that this effort does not want to replace whatever framework the facilities use to do analysis , eg psana(in our case psana). The final result would be a set of libraries that psana can invoke to implement the key operations indicated above.

This ecosystem would be based on existing libraries as much as possible and we would write new code only when needed. We could, for example, write a thin layer which allows to save histograms or other objects to HDF5.

Requirements

Core subset of ROOT Features

We claim that the key services our users would like are:

  • Ability to write both C++ and python modules.
  • Ability to persist basic objects (histograms, arrays, ntuple), ability to store and retrieve intermediate analysis results.
  • Ability to fit complex data.
  • Ability to invoke high performance algorithms; these can range from basic algorithms, like FFTs, to more specialized operations, like peak finding.
  • Ability to plot histograms (1D, 2D, profiles), xy plots (graphs), images, functions
  • Ability to interact with the plots (resize, region of interest, etc)

In addition:

  • Any function or class which can be used on the C++ side, can also be used on the python side and viceversa (similarly to Qt/PyQt)

Core Subset of ROOT Features and Possible Replacements

This table tries to identify possible replacements for the key features of ROOT that we'd like to provide.

Feature

ROOT Approach

Replacement

Feature

Comment

Interpreter

CINT, cling, PyRoot

Python

Input/Output

Easy to save/restore histograms, trees ; more complex to save arbitrary structures

Dedicated development effort? Based on HDF5?

Math libraries

Numerical algorithms, linear algebra, vectors, statistics, fitting

NumPy, SciPy, GSL, minuit. Different classes and functions for python and C/C++?

GUI

Widgets, signals/slots mechanism

Qt, PyQt

Histograms

1D, 2D, Profile

Dedicated development effort?

Trees

To store large quantities of same-class objects

Dedicated development effort?

2D graphics

Display 1D and 2D histograms, functions, graphs

3D graphics

Event display

Possible replacements

Feature

Comment

Interpreter

Python

Input/Output

Could need dedicated development effort, could be built on top of HDF5

Math libraries

NumPy, SciPy, GSL. Issue: different environments between python and C/C++?

GUI

QT, PyQt

Histograms

Could need dedicated development effort

Trees

Could need dedicated development effort

2D graphics

Matplotlib? How to display histograms and trees?

3D graphics

Event display

Possibly not needed for now