Versions Compared

Old Version 47

changes.mady.by.user Tom Glanzman

Saved on

compared with

New Version 48

changes.mady.by.user Tom Glanzman

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • 4/23/2013 - Begin setting up P300 task for Pass8 reprocessing.
  • 5/10/2013 - Initial version of task running.  See this page for infomation about the task creation, and this page for performance comparison with P202
  • 7/2/2013 - after weeks of testing, final GR, jobOpts, etc. come together and production tentatively starts.
    • Block 1 is defined with this run range: 239557414 through 392842073 (2008-08-04 15:43:37 through 2013-06-13 18:47:50 UTC)
    • Update task to use the P202 generation of FT2 files (adds columns LAMBDA, RA_SUN, DEC_SUN, as well as meaningful BTIs)
    • First 10 trial jobs launched. Too many ACD INFO messages 'Missed Poca for ID 602 at...' so modify jobOpts to disable.
    • xroot disk space at the start of Pass8 = 508 TB (just after commissioning of fermi-xrd11)
  • 7/29/2013 - a new GR is prepared (20-09-01) which fixes some problems in ACD tracking issues affecting ~32 runs so far (out of 5283). trickleStream disabled, wait for pending jobs to start, rollback other failed jobs, switch to new GR, rollback of ACD-failed jobs...
    • This new GR continues to seg fault, so revert back to GR 20-09-00 and restart trickleStream.
  • 7/31/2013 - A new GR (20-09-02) with patch for ACD tracking installed after stream 5970. At the time, there were 33 failed processClumps jobs. All rolled back.
    • Last stream with GR 20-09-00 = 5970, run 274118943
    • First stream with GR 20-09-02 = 5971, run 274124672
    • Full list of rolled back substreams is here
  • 10/15/2013 - Update job options to use the new "L1current" flavor of calibrations/alignment rather than "p7repro".  First stream with new configuration: 20804.
  • 11/14/2013 - Block 1 complete except for routine cleanup

  • 11/21/2013 - Block 1 complete
    Problems encountered include:
    ** pipeline_summary file corruption causing loss of email or of it being ignored, finally jobs terminate via reaper (Tony notified)
    ** large blocks of failures due to dataCatalog not returning desired FT2 files (Brian notified, fix possible by moving query to scriptlet)
    ** jobs running out the clock due to anomalously slow xroot behavior (Wilko/Andy notified)
    ** problems with individual batch machines (Renata notified, and I also have the authority to remove batch machines from production)
    Summary of events processed in Block 1 from dataCatalog:

    NameTypeFilesEventsSizeCreated (UTC)
    ELECTRONMERITGroup2692901.4 GB09-May-2013 22:22:19
    FILTEREDMERITGroup2692901.4 GB09-May-2013 22:22:17
    GCRGroup2692958,653,424,7081.0 TB09-May-2013 22:22:18
    MERITGroup2692958,653,424,70898.3 TB09-May-2013 22:22:18
    RECONGroup2692958,653,424,708816.1 TB09-May-2013 22:22:20
    RELATIONGroup2692958,653,424,7087.7 TB09-May-2013 22:22:20

  • 4/30/2014 - Refactor P300-ROOT task to move all dataCatalog queries from batch jobs into scriptlets; update GPLtools (segregate INput and OUTput files); update git repository; etc..  Create task version 1.1 starting with stream 26929 to improve web app performance. Create JIRA for this task.  Submit first 50 streams of 'block 2'

  • 5/6/2014 - After big Oracle upgrade, regenerate/redefine 'block 2' to contain all runs through April 2014.  This block contains 31838 runs, in increase of 4909 runs, starting with 392848044 and ending with 420590482.  All future routine operational comments will be added tot he JIRA and not to this confluence page.

 

 

Configuration

Task Location

/nfs/farm/g/glast/u38/Reprocess-tasks/P300-ROOT

Task Version1.0 (streams 0-26928), 1.1 (streams 26929-end)

Task Status

http://glast-ground.slac.stanford.edu/Pipeline-II/index.jsp

GlastRelease

20-09-00 streams 0 - 5970 (02 Jul 2013)
20-09-02 streams 5971 - end (31 Jul 2013)

Run Selection

based on a modified "standard" selection, see https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets
(((sIntent=="nomSciOps" || sIntent=="nomSO_noSk_noCno_optGccc_allEna" || sIntent=="nomSciOps_diagEna" || (sIntent=="nomSciOps_Emin5MeV"&&RunMin>242070455) || nRun==242429468 ) && (RunQuality != "Bad" || is_null ( RunQuality ) ) ) || sIntent=="nadirOps" )

s/c data

P202 FT2SECONDS which will eventually become a "standard" Public Release https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets

Input Run List

ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P300-ROOT/config/runList.txt

photonFilter

Not yet applicable, although this is defined by default:  CTBParticleType==1 && ((FT1EventClass & 0x00003EFF)!=0)
pass7.6_Extended_cuts_L1 in evtClassDefs

NOTE: task P301-MERIT regenerates MERIT files from P300 MERIT files using TMineExt, updating the event classification data

electronFilter

Not yet applicable, although this is defined by default:  CTBParticleType==1

Code Variants

redhat5-i686-32bit-gcc41 (Optimized), note that rhel5-64 and rhel6-64 GR builds are not yet available

jobOpts

ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P300-ROOT/config/Pass8Recon.txt

calibrations & alignments

"p7repro" and "L1current" (the latter to conform with new Level 1 flavor, and to include new ACD calibration for 9 Sep 2012)

Output Data Products

RECON, GCR, RELATION, MERIT, FILTEREDMERIT, ELECTRONMERIT (but note that FILTEREDMERIT and ELECTRONMERIT are empty or contain junk)

...