P107 Reprocessing
status: In Production Complete
last update: 2 5 November 2010
This reprocessing will create GCR (Galactic Cosmic Ray) root files which were inadvertantly not produced from a period of Nov 2009 through July 2010.
...
Task Location | /nfs/farm/g/glast/u38/Reprocess-tasks/P107-GCR |
Task Status | http://glast-ground.slac.stanford.edu/Pipeline-II/index.jsp |
GlastRelease | v15r47p12gr13 |
ROOT version | v5.26.00a-gl1 |
Input Data Selection | "standard" from https://confluence.slac.stanford.edu/display/SCIGRPS/LAT+Dataset+Definitions along with "&& (RunQuality != "Bad" || is_null ( RunQuality )" |
Input Run List | ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P107-GCR/config/runFile.txt |
jobOpts | ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P120-MERIT/config/reGCR.txt |
Output Data Products |
Processing log
- 11/5/2010 - Production complete
- 11/2/2010 - Validation passed, restart production.
- 11/1/2010 - process additional 79 test/validation runs (bring total # runs to 100). Use trickleStream, limiting number of simultaneous processClump jobs to 500. This put a significant strain on wain038, essentially 100% CPU utilization for ~1 hour, and ~350 MB/s I/O. Also affected were wains 037 and 039, but not to quite the same extent.
- 10/27/2010 - commission new form of task in which GCR files produced in clumps and then merged (by tskim); process first 21 test/validation runs.
- 10/20/2010 - discover two facts: first 20 test runs were limited to 100k events, and Gleam has huge memory leak and crashes after about 400k-600k events have been reprocessed. Report to Heather, et al.
- 10/19/2010 - Configure task and create first 20 test/validation runs. Successful.