Versions Compared

Old Version 13

changes.mady.by.user Unknown User (chamont)

Saved on

compared with

New Version Current

changes.mady.by.user Unknown User (chamont)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 4.0

Highest Priority Items

  1. Support deletion of some files in the input data file list.
  2. The output data file list should include only the file whose at least one event has been kept

Under work

  1. Merge with CEL code.
  2. We need a complete redesign and implementation of errors management. At the same time, in the case of jobs which are empty for a reason or another, we should not consider it as an error, for the case it is the result of some partitioning of a big job to be parallelized at a higher level.

Documentation & Test suite

  1. Review all the inline documentation of DataServer package, which is mainly obsolete.
  2. The Discuss with ROOT team the best way to check the content of a produced root file (script scanFile) is not good enough. We need something checking deeper the content. Replace scanFile with RUN_Check ?? characterize a ROOT file (currently based on leaves, see the check step).
  3. We should integrate large-scale tests, and track the size of the parameter files (files end events lists) so to check they stay in good reasonnable limits.

Interface & Features

  1. be able to process beamtest input data files.
  2. provide soon feedback information to the web front-end (perhaps a "fake" run mode, which would be started by typing "skimmer try" ?)
  3. generically scan, display, and copy auxiliary objects found in ROOT files => do study hadd.
    The GlastDataTypes C++ class have harcoded data types and values. They should rather be read from a configuration file, under the controll of the user.
  4. That same configuration file, or a separate one, should contain parameters instead of the environnements environnement variables.
  5. For the communication with the Web front-end, we should favor a early detection of problems, and setup a standard way to feed back the front-end with such early information (which could include an evaluation of the job end). Perhaps a "fake" run mode, which would be started by typing "skimmer try" ?
  6. Enable tcuts through multiple data kinds.
  7. Idea : ensure that when a skimming is failing, the incomplete output data files is erased. Then, change the skimming step so that it will check that some output files already exists or not, and only do the ones which are lacking ? (a SK_FORCE_SKIP would enforce the skimming whatever the files, and perhaps would be "true" by default).

Bugs

  1. When one type a Control-C during the execution of the skimmer test suite, this let a ROOT process running indefinitively. We should find out at which level the Control-C is not transmistted.
  2. From time to time, the content of the produced ROOT files seems to change a little. Before we can investigate more this bug, we should wait for the test suite to be improved, and everything to be compiled, so that we can avoit avoid CINT problems.

Implementation

  1. I think we should plug to the FileListManager a list of PipelineConfig.
    When a FileListManager needs an information for a given task, it would ask to all
    instances of PipelineConfig, and see which one recognize this task and provide
    information about where to find the files. Such a structure should greatly help to
    manage the many kinds of pipelines under work.
  2. Study new class TEntryList
    The use of CINT interpreter is a great source of bugs. We should make
    anything we can compilable. . Replace TEventList with TEntryList ?
  3. To be explored : the use of ROOT MakeClass/MakeProject so to avoid loading libraries. First experiments are not very successfull, especially with mc/digi/recon files. But perhaps with tuples-like and FileHeader it could be of use.
  4. To be explored : hadd.
  5. The different datakinds are skimmed one by one. We could investigate if we can/should skim them in parallel. In the same topic, perhaps we should think if we must reduce the number of different ROOT macros which are (slowly) started one after the other. Perhaps we should find a way to decorrelate what we call the "steps" from the processes (currently, each step is necessarily a separated process).
  6. Study Riostream.h : it is defining a "using namespace std", so NOT TO BE USED IN HEADER FILES.
  7. Replace TEventList with TEntryList ?

Internal cleaning

Some files in DataServer/share are perhaps useless or needs to be re-activated :

...

  1. .

...