Versions Compared

Old Version 37

changes.mady.by.user Tom Glanzman

Saved on

compared with

New Version 38

changes.mady.by.user Tom Glanzman

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • 4/23/2013 - Begin setting up P300 task for Pass8 reprocessing.
  • 5/10/2013 - Initial version of task running.  See this page for infomation about the task creation, and this page for performance comparison with P202
  • 7/2/2013 - after weeks of testing, final GR, jobOpts, etc. come together and production tentatively starts.
    • block 1 is defined with this run range: 239557414 through 392842073 (2008-08-04 15:43:37 through 2013-06-13 18:47:50 UTC)
    • Update task to use the P202 generation of FT2 files (adds columns LAMBDA, RA_SUN, DEC_SUN, as well as meaningful BTIs)
    • First 10 trial jobs launched. Too many ACD INFO messages 'Missed Poca for ID 602 at...' so modify jobOpts to disable.
    • xroot disk space at the start of Pass8 = 508 TB (just after commissioning of fermi-xrd11)
  • 7/29/2013 - a new GR is prepared (20-09-01) which fixes some problems in ACD tracking issues affecting ~32 runs so far (out of 5283). trickleStream disabled, wait for pending jobs to start, rollback other failed jobs, switch to new GR, rollback of ACD-failed jobs...
    • This new GR continues to seg fault, so revert back to GR 20-09-00 and restart trickleStream.
  • 7/31/2013 - A new GR (20-09-02) with patch for ACD tracking installed after stream 5970. At the time, there were 33 failed processClumps jobs. All rolled back.
    • Last stream with GR 20-09-00 = 5970, run 274118943
    • First stream with GR 20-09-02 = 5971, run 274124672
    • Full list of rolled back substreams is here
  • 10/15/2013 - Update job options to use the new "L1current" flavor of calibrations/alignment rather than "p7repro".  First stream with new configuration: 20804.
  • 11/14/2013 - block 1 complete except for routine cleanup

  • 11/21/2013 - block 1 complete

    NameTypeFilesEventsSizeCreated (UTC)Links
    ELECTRONMERITGroup2692901.4 GB09-May-2013 22:22:19Files
    FILTEREDMERITGroup2692901.4 GB09-May-2013 22:22:17Files
    GCRGroup2692958,653,424,7081.0 TB09-May-2013 22:22:18Files
    MERITGroup2692958,653,424,70898.3 TB09-May-2013 22:22:18Files
    RECONGroup2692958,653,424,708816.1 TB09-May-2013 22:22:20Files
    RELATIONGroup2692958,653,424,7087.7 TB09-May-2013 22:22:20Files

Configuration

Task Location

/nfs/farm/g/glast/u38/Reprocess-tasks/P300-ROOT

Task Status

http://glast-ground.slac.stanford.edu/Pipeline-II/index.jsp

GlastRelease

20-09-00 streams 0 - 5970 (02 Jul 2013)
20-09-02 streams 5971 - end (31 Jul 2013)

Run Selection

based on a modified "standard" selection, see https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets
(((sIntent=="nomSciOps" || sIntent=="nomSO_noSk_noCno_optGccc_allEna" || sIntent=="nomSciOps_diagEna" || (sIntent=="nomSciOps_Emin5MeV"&&RunMin>242070455) || nRun==242429468 ) && (RunQuality != "Bad" || is_null ( RunQuality ) ) ) || sIntent=="nadirOps" )

s/c data

P202 FT2SECONDS which will eventually become a "standard" Public Release https://confluence.slac.stanford.edu/display/SCIGRPS/Official+LAT+Datasets

Input Run List

ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P300-ROOT/config/runList.txt

photonFilter

Not yet applicable, although this is defined by default:  CTBParticleType==1 && ((FT1EventClass & 0x00003EFF)!=0)
pass7.6_Extended_cuts_L1 in evtClassDefs

electronFilter

Not yet applicable, although this is defined by default:  CTBParticleType==1

Code Variants

redhat5-i686-32bit-gcc41 (Optimized), note that rhel5-64 and rhel6-64 GR builds are not yet available

jobOpts

ftp://ftp-glast.slac.stanford.edu/glast.u38/Reprocess-tasks/P300-ROOT/config/Pass8Recon.txt

calibrations & alignments

"p7repro" and "L1current" (the latter to conform with new Level 1 flavor, and to include new ACD calibration for 9 Sep 2012)

Output Data Products

RECON, GCR, RELATION, MERIT, FILTEREDMERIT, ELECTRONMERIT (but note that FILTEREDMERIT and ELECTRONMERIT are empty or contain junk)

...

  • 8/1/2013 - Initial P300x-FITS task created. 10 trial runs processed. (See caveats above.)

  • 8/7/2013 - First year of data run through the task, amounting to 5537 runs and 11790391606 events:

    First run

    239557417

    2008-08-04 15:43:37

    Last run

    271850279

    2009-08-13 09:57:59

  • 8/28/2013 - Second year of data ready and processed -> extendedFT1. Note that 36 troublesome runs omitted from 1st year data were included in this sample. There should be no missing runs in this sample.

    First run

    239557417

    2008-04-2008 15:43:37

    Last run

    302647722

    2010-08-04 20:48:40

    10,976 runs, 2,677,747,829 events, 234.7 GB

  • 9/25/2013 - Third year of data ready.

    First run

    239557417

    2008-04-2008 15:43:37

    Last run

    334184989

    2011-08-04 21:09:47

    This block contains 16,516 runs and 36,056,385,847 events, or an increase over year 2 of 5,540 runs.

  • 11/21/2013 - Prepare for block 1 backfill.

Configuration

CAUTION: The following data describes the experimental P300x-FITS task, not the production task

...