Warning

The skimming tool has been generalized and externalized. The last releases of source code, documentation and pending issues are now available at the new TRAC server. The tutorial below is still valuable, because the recent TSkim releases are still compatible with those old variables. Yet, it is strongly recomended to move to the new interface, described here, here and there. An upgrade of this tutorial is under work.

Date: 24 June 2008
Software by: David Chamont
Presentation by: Tom Glanzman
Update: 16 July 2008

Scope

This is intended to be a quick start guide for using the current skimmer version (v6r0p1) to extract merit/digi/recon events from real data as well as construct event lists. Event lists (text file with "Run EventId" pairs) may be used to drive the WIRED event display. FRED (i.e. Gleam) needs the explicit digi and recon files to read.

Two simple cases will be illustrated:

  1. Generate an event list and digi/recon event files after applying a TCut to the merit ntuple
  2. Generate digi/recon event files from a provided event list

Note that at some point the web interface to the skimmer will be able to handle these functions so this is but a stop-gap measure until the web interface is ready.

There is much more the skimmer can do but this guide does not (yet) cover any of that functionality.

Overview

  1. Log onto a SLAC Linux machine (e.g., noric)
  2. Identify files needed for skim operation, e.g., browse the data catalog
  3. Create a text file with the list of file names (e.g., filelist.txt)
  4. Edit the filelist.txt file as illustrated in the Examples below
  5. Locate yourself in a directory with sufficient space to hold the expected output
  6. Create/customize a script to run the skimmer (templates provided below)
  7. Run the skimmer

Examples. Code and files from the following examples are currently resident here (but they could disappear at some point):

/nfs/farm/g/glast/u18/skimmerTest/

Example 1 - Create an event list based on a merit TCut, then extract digi/recon events (subdir /treeCut)

Given this script and this filelist, the following files result:

$ ./treeCut.csh
[lots of output...]
$ ls -l
-rw-rw-r--    1 dragon   glast-pipeline      727 Jun 24 14:00 newfilelist.txt
-rw-rw-r--    1 dragon   glast-pipeline      881 Jun 24 14:00 libraryList.txt
-rw-rw-r--    1 dragon   glast-pipeline    15666 Jun 24 14:00 v14r4-BranchList.txt
-rw-rw-r--    1 dragon   glast-pipeline     1787 Jun 24 14:00 eventList.txt
-rw-r--r--    1 dragon   glast-pipeline   122406 Jun 24 14:00 L1PROC-skim_merit.root
-rw-r--r--    1 dragon   glast-pipeline   873903 Jun 24 14:00 L1PROC-skim_recon.root
-rw-r--r--    1 dragon   glast-pipeline   182663 Jun 24 14:00 L1PROC-skim_digi.root

These root files contain 119 events.

Example 2 - Extract digi/recon events from a provided event list (subdir /treeEvtList)

Given this script, this filelist and this eventlist, the following files result:

$ ./treeEvtList.csh
[lots of output...]
$ ls -l
-rw-rw-r--    1 dragon   glast-pipeline      727 Jun 24 14:04 newfilelist.txt
-rw-rw-r--    1 dragon   glast-pipeline      881 Jun 24 14:04 libraryList.txt
-rw-rw-r--    1 dragon   glast-pipeline    15666 Jun 24 14:04 v14r4-BranchList.txt
-rw-r--r--    1 dragon   glast-pipeline    51023 Jun 24 14:04 L1PROC-skim_merit.root
-rw-r--r--    1 dragon   glast-pipeline   194381 Jun 24 14:04 L1PROC-skim_recon.root
-rw-r--r--    1 dragon   glast-pipeline   123064 Jun 24 14:04 L1PROC-skim_digi.root

These root files contain 21 events.

Reference information

  • Skimmer User's Guide
  • The skimmer code is in CVS, package name = DataServer
  • At SLAC the skimmer code is located in this directory 
    /afs/slac/g/glast/ground/DataServer/
  • You might also be interested in the Data Access FAQ

Update 1

Various skimmer-related problems have been reported over the past few weeks. Below are some general hints to using the skimmer followed by descriptions of a couple of outstanding problems.

General hints

  • Define the following environment variables in your skimmer scripts

For noric machines:

setenv TMP /usr/work/<your-userid>
setenv TMPDIR /usr/work/<your-userid>

For batch machines:

setenv TMP /scratch/<your-userid>
setenv TMPDIR /scratch/<your-userid>

Note that if you are running multiple skimmer jobs on the same machine, you will be obligated to define unique $TMP and $TMPDIR spaces for each job.

  • Use skimmer v6r0p1 rather than v6r1
  • If problems, limit number of events processed per job to no more than about 100M events. If problems persist, reduce further.

Outstanding problems.

  • Skimming RECON and DIGI files is unreliable due to corrupt TTreeIndex structures. This problem arises when data in L1PROC is merged and the TTreeIndex not rebuilt. Solutions being devised now.
  • Skimming large numbers of MERIT events can cause skimmer to crash. One contributing factor may be ROOTs insistence of keeping the entire TTreeIndex in memory at all times when it is being used. There may also be some internal memory corruption if the TTreeIndex grows too large. Being investigated.
  • No labels