Page History

Officially started on Dec 22, 2015.

HPS-JAVA Snapshot:  3.5-20151218.205540-15

DST-maker Release:  0.10

Run Spreadsheet:   Google Docs

Detector:  HPS-EngRun2015-Nominal-v3-4-fieldmap

Batch Farm Scripts:

• /u/group/hps/production/data/EngRun2015/pass4
• github

Outputs:

Note:  DSTs had some bugs in pass4.  Correct pass4 DSTs were regenerated here:  /mss/hallb/hps/engrun2015/pass4dst 

• Disk:  /work/hallb/hps/data/engrun2015/pass4

  • All DQ(M) and logs

  • Unskimmed Recon:

    • LCIO:  Canonical
    Canonical
    • and Calibration runs
    ' full LCIOs and DSTsSkimmed LCIOs and DSTs
    • only
    • DST:  All

  • Skimmed Recon:

    • LCIO:  moller, pulser, v0
    • DST:  All (fee, moller, pulser, v0, s0, p0)
• Tape:  /mss/hallb/hps/engrun2015/pass4
  • Everything
• Skims:  FEE, Moller, Pulser, V0, S0, P0

...

Output Directory Structure:

Image Modified

Result:

3.5K EVIO files processed in about 5 days on the batch farm (in competition with upass4) with a "failure" rate of only 1% (almost entirely due to timeouts, limit was set at 30 hours).  Failures resubmitted on 1/1/2016 with a much larger time limit (time limit is the parameter with largest effect on batch farm priority7 of 30 jobs timed out again, appears to be stuck in some Minuit stuff in dqm).

30% of the EVIO files resulted in empty (filesize=0) v0 and moller skims (here's a list of their run/file numbers).  80% 80% of those empties were from 1.5 mm, which is about 50% of 1.5 mm.  Note that files of zero size cannot be written to tape, so you will not see them in /mss.  Zero EVIO files resulted , and that no jobs resulted in empty pulser skims.

Here's a histogram of wall time per EVIO file (the 800 empty 1.5 mm v0/moller files skims surely correspond to the peak at 9.5 hours, although I did not confirm that), where the bump at 30 hours is timeout failures:

Disk Usage:

/work/hallb/hps/data/engrun2015/pass4

2.5 TB:

Image Added