status: Running
last update: 29 July 01 Aug 2013
This page is a record of the configuration and execution of the P300 reprocessing project, full reprocessing from DIGIs using Pass8 analysis code.
...
- 4/23/2013 - Begin setting up P300 task for Pass8 reprocessing.
- 5/10/2013 - Initial version of task running. See this page for infomation about the task creation, and this page for performance comparison with P202
- 7/2/2013 - after weeks of testing, final GR, jobOpts, etc. come together and production tentatively starts.
- block 1 is defined with this run range: 239557414 through 392842073 (2008-08-04 15:43:37 through 2013-06-13 18:47:50 UTC)
- Update task to use the P202 generation of FT2 files (adds columns LAMBDA, RA_SUN, DEC_SUN, as well as meaningful BTIs)
- First 10 trial jobs launched. Too many ACD INFO messages 'Missed Poca for ID 602 at...' so modify jobOpts to disable.
- xroot disk space at the start of Pass8 = 508 TB (just after commissioning of fermi-xrd11)
- 7/29/2013 - a new GR is prepared (20-09-01) which fixes some problems in ACD tracking issues affecting ~32 runs so far (out of 5283). trickleStream disabled, wait for pending jobs to start, rollback other failed jobs, switch to new GR, rollback of ACD-failed jobs...
- This new GR continues to seg fault, so revert back to GR 20-09-00 and restart trickleStream.
- 7/31/2013 - A new GR (20-09-02) with patch for ACD tracking installed after stream 5938. At the time, there were 33 failed processClumps jobs. All rolled back.
Configuration
Task Location | /nfs/farm/g/glast/u38/Reprocess-tasks/P300-ROOT |
Task Status | |
GlastRelease | 20-09-00 streams 0 - end (02 Jul 2013) |
Run Selection | based on a modified "standard" selection, see https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets |
s/c data | P202 FT2SECONDS which will eventually become a "standard" Public Release https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets |
Input Run List | ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P300-ROOT/config/runList.txt |
photonFilter | Not yet applicable, although this is defined by default: CTBParticleType==1 && ((FT1EventClass & 0x00003EFF)!=0) |
electronFilter | Not yet applicable, although this is defined by default: CTBParticleType==1 |
Code Variants | redhat5-i686-32bit-gcc41 (Optimized), note that rhel5-64 and rhel6-64 GR builds are not yet available |
jobOpts | ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P300-ROOT/config/Pass8Recon.txt |
Output Data Products | RECON, GCR, RELATION, MERIT, FILTEREDMERIT, ELECTRONMERIT (but note that FILTEREDMERIT and ELECTRONMERIT are empty or contain junk) |
...
This task generates all desired FITS data products.
NOTE: a temporary task called P300x-FITS has been created. It generates only EXTENDEDFT1 files and nothing else. There is no diffuse calculation performed. An untagged version of evtClassDefs is used which contains Matthew Wood's initial event classification and selections. This task does not yet existwill likely be a template for a future production P300-FITS task. All data produced by this task is "throw-away" and is not expected to survive beyond initial validation studies.
Status chronology
- 8/1/2013 - Initial P300x-FITS task created. (See warning above)
Configuration
DO NOT USE: NOT YET UPDATED
Task Location | /nfs/farm/g/glast/u38/Reprocess-tasks/P300-FITS |
Task Status | http://glast-ground.slac.stanford.edu/Pipeline-II/task.jsp?task=75031156 |
Input Data | MERIT (direct from P300-ROOT) |
spacecraft data | same as P300-ROOT |
Input Run List | ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P202-FITS/config/runList.txt |
evtClassDefs | 00-19-04 (6/23/2011) |
eventClassMap | EvtClassDefs_P7V6.xml |
ScienceTools | 09-27-01 (2/15/2012) 09-31-01 (3/5/2013) |
Code Variants | redhat5-i686-32bit-gcc41 (Optimized) |
Diffuse Model | based on contents of /afs/slac.stanford.edu/g/glast/ground/GLAST_EXT/diffuseModels/v2r0 /v3r0 |
Diffuse Response | 'source' using P7SOURCE_V6 P7REP_SOURCE_V10 IRF |
IRFs | P7REP_*_V10, contained within ScienceTools release |
Output Data Products |
...
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="096c0af35d0804b0-ea23f35b-42b74e06-b640a0eb-12b2d1c9eba16a73355a6a57"><ac:plain-text-body><![CDATA[ | Data Product | destination | data content [1] | event selection [1] | makeFT1 | gtselect | gtdiffrsp | gtmktime | ]]></ac:plain-text-body></ac:structured-macro> |
|---|---|---|---|---|---|---|---|---|---|
EXTENDEDFT1 | SLAC | FT1variables | ((FT1EventClass & 0x00003EFF)!=0) | | | | | ||
FT1 | FSSC+SLAC | FT1variables | 'source' and above | | | (inherited) | | ||
EXTENDEDLS1 | SLAC | LS1variables | ((FT1EventClass & 0x00003EFF)!=0) | | | | | ||
LS1 | FSSC+SLAC | LS1variables | 'transient' and above | | | (inherited) | | ||
ELECTRONFT1 | SLAC | FT1variables | CTBParticleType==1 | | | | |
...