...

status: Complete
last update: 02 17 August 2011

This page is a record of the configuration for the P120 reprocessing project, event reclassification using Pass 7.3/7.4/7.6. This project involves reprocessing with Pass7 classification trees and (ultimately) new IRFs. The name "P120" derives from the word "processing" and the initial file version to be used for the output data products, e.g., r0123456789_v120_merit.root.

...

NOTE: One run, 242429468, of type TrigTest was declared 'good for science' but long after this task got started, so it has been intentionally omitted.

8/17/2011 update: Four orphaned runs are being reprocessed, including one TrigTest run and four nadir-pointed runs.

Bookkeeping

1. (This page): Define ingredients of reprocessing (processing code/configuration changes)
2. Processing History database: http://glast-ground.slac.stanford.edu/HistoryProcessing/HProcessingRuns.jsp?processingname=P120
   1. List of all reprocessings
   2. List of all data runs reprocessed
   3. Pointers to all input data files (-> dataCatalog)
   4. Pointers to associated task processes (-> Pipeline II status)
3. Data Catalog database: http://glast-ground.slac.stanford.edu/DataCatalog/folder.jsp
   1. Lists of and pointers to all output data files
   2. Meta data associated with each output data product

...

P120-MERIT

Anchor
P120-MERIT P120-MERIT

Status chronology

• 8/17/2011 - Begin reprocessing four orphan runs.
• 8/1/2011 - Begin and complete final backfill through run 333880535 (2011-08-01 08:35:33 UTC). For now, the three nadirOps runs are represented by dummy place-holder entries in the runFile.txt – their pipeline streams will fail.
• 7/29/2011 - Modified selection criteria for reprocessing run selection (findRunsRepro.py) to allow nadir-pointed data. This means adding ' || sIntent=="nadirOps"' to the dataCatalog selection string. See https://confluence.slac.stanford.edu/display/ISOC/Nadir+Obs+Test+-+26+July+2011 for a list of runs affected by the nadir-pointed test.
• 7/26/2011 - Recovered three missing runs (below).
• 7/21/2011 - Begin and (mostly) complete reprocessing block 13, through run 332930755 (2011-07-21 08:45:53 UTC), 528 new runs - special GRB request. Note three runs failed skim/merge and are being investigated:

  stream	run	UTC	Crash location
  16136	332054399	2011-07-11 05:19:57	Filtered Merit skim failure
  16151	332140182	2011-07-12 05:09:40	Electron Merit skim failure
  16156	332169056	2011-07-12 13:10:54	Filtered Merit skim failure

• 6/17/2011 - Begin and complete reprocessing block 11, through run 329923889 (2011-06-16 13:31:27 UTC), 889 new runs.
• 4/19/2011 - Begin and complete reprocessing block 10, through run 324849509 (2011-04-18 19:58:27 UTC), 52 new runs.
• 4/15/2011 - Begin and complete reprocessing block 9, through run 324551768 (2011-04-15 10:51:27 UTC)
• 4/13/2011 - Begin and complete reprocessing block 8, through run 324368491 (2011-04-13 06:21:29 UTC)
• 4/3/2011 - The three missing runs have been produced by Level 1. Runlist recreated and those runs rolled back. There are no missing runs at this point.
• 3/29/2011 - Due to some hidden I/O problems, two changes have been made to this task, neither of which should affect data content. Note that this change will take effect on runs after 321756673, or any runs rolled back after this date.

  Package	Old Version	New Version	Reason for change
  GPLtools	v1r15p1	GPLtools-02-00-00	Checks size of files before and after move between disk and xroot
  skimmer	07-07-00	08-01-00	detects failure to open input file

• 3/17/2011 - Last block complete (one instance of skimmer failure in mergeClumps)
• 3/15/2011 - Expand P120 to present (last run 321756673, 2011-03-14 00:51:11 UTC). Three runs continue to be dummied out. Once those runs have proper RECON files, their streams can be rolled back.

  run	task stream	# subStreams
  306353950	11624	10
  316611240	13431	8
  320850543	14171	7

• 2/4/2011 - Expand P120 to present (last run 318211122, 2011-01-31 23:58:40 UTC). This include three runs for which there are no Recon files which have, for the moment, been supplied with 'dummy' entries in the runFile.:

  306353950	'bad chunk' (known previously) - awaiting GR update
  313483912	missing 700s, being worked on
  316611240	'bad chunk' - awaiting GR update

• 1/12/2011 - Expand P120 coverage to include Crab ToO
  • Crab ToO 9/23/2010 15:50:50 to 9/27/2010 19:49:38, corresponding to MET r0306949696-r0307308940
  • reconfig through end of Sep 2010: 10916 -> 11841 runs, increase of 925 runs
  • No recon file for run 306353950 (being worked on, for the moment, place dummy file in runFile.txt)
  • First run in new block completed, awaiting checkered flag to continue...
• 8/29/2010 - Discovered three merge steps that silently failed (xroot file access). TASK complete.
• 8/28/2010 - processing formally complete (10916 runs), but some discrepancy in # of events
• 8/26/2010 - serious xroot problems. See initial distribution of files across xroot servers. From this report (courtesy Wilko) it is easy to see where problems are likely to arise - when the number of servers involved is small, e.g. two or three.
• 8/19/2010 - production continues at a crawl due to xroot server difficulties
• 8/16/2010 - resume full production, but at a slow trickle (max 350 simultaneous processClump jobs)
• 8/8/2010 - block 2 reprocessing complete. Many xroot server problems. (5 days to process 2084 runs)
• 8/3/2010 - begin block 2 reprocessing (through 255132033 MET), bringing the total runs reprocessed to 2721, about 5-1/2 months of data.
• 7/28/2010 - block 1 re-reprocessing complete
• 7/27/2010 - New GlastRelease (v17r35p10) containing new evtUtils, "to make the FT1EventClass bits compatible with the ScienceTools". Cleanup, including removing all files created last week during the first attempt.
• 7/21/2010 - block 1 reprocessing complete
• 7/20/2010 - agree upon 'pilot block' of runs (239557417 - 243220241), 637 runs. Begin...
• 7/19/2010 - submit first test run. success. await feedback

...