Page History

status: In development Complete
last update: 27 January 21 July 2015

This page is a record of the configuration and execution of the P302 reprocessing project, generation FITS files from the P301 MERIT files.

...

• 1/27/2015 - Completed block 1, through 30 Sep 2014

• 2/17/2015 - Rerun stream 29158, Run 405333211, after Warren revises the FT2 file – which adds back the final 10 minutes of this run.  (Jean Ballet found this problem.)

• 2/18/2015 - Completed block 2, through 31 Jan 2015

• 3/9/2015 - Create P302-FITS v 2.0 task.  The only difference is changing evtClassDefs from 01--01--01 to 01--01–04 which updates event selection to eliminate sweep and periodic trigger events.  (This was discovered by Don Horner at FSSC as duplicate (sweep) events in files transferred there a couple weeks ago.)
    Begin processing entire mission once again...

•  3/11/2015 Reprocessing complete.  New Datacatalog summary:

  Name	Files	Events	Size	Created (UTC)
  EXTENDEDFT1	36020	14,613,584,327	1.3 TB	21-Jan-2015 20:32:16
  EXTENDEDLS1	36020	14,613,504,070	2.7 TB	21-Jan-2015 20:32:16
  FT1	36020	740,187,922	68.6 GB	21-Jan-2015 20:32:16
  LS1	36020	1,956,117,915	376.5 GB	21-Jan-2015 20:32:16

  Problem with xFT1 and xLS1 files: #evts should be identical.  After studying logs, decide there must be a silent failure mode to gtmktime.  Rollback and now the statistics look like this:

  Name	Files	Events	Size	Created (UTC)
  EXTENDEDFT1	36020	14,613,589,234	1.3 TB	21-Jan-2015 20:32:16
  EXTENDEDLS1	36020	14,613,589,234	2.7 TB	21-Jan-2015 20:32:16
  FT1	36020	740,188,083	68.6 GB	21-Jan-2015 20:32:16
  LS1	36020	1,956,127,293	376.5 GB	21-Jan-2015 20:32:16

• 04/18/2015 - Begin block 2, through 7 Apr 2015

• 04/19/2015 - Block 2 complete.  Summary from dataCatalog:

  Name	Files	Events	Size	Created (UTC)
  EXTENDEDFT1	37023	15,032,071,005	1.3 TB	21-Jan-2015 20:32:16
  EXTENDEDLS1	37023	15,032,071,005	2.8 TB	21-Jan-2015 20:32:16
  FT1	37023	763,368,036	70.7 GB	21-Jan-2015 20:32:16
  LS1	37023	2,015,775,627	388.0 GB	21-Jan-2015 20:32:16

• 06/11/2015 - Block 3, 7-Apr through 3-Jun-2015 complete (MERIT statistics from P301-MERIT task)

  Name	Files	Events	Size	Created (UTC)
  MERIT	37888	82,148,825,606	139.7 TB	28-Mar-2014 22:40:39
  EXTENDEDFT1	37888	15,381,147,951	1.4 TB	21-Jan-2015 20:32:16
  EXTENDEDLS1	37888	15,381,147,951	2.9 TB	21-Jan-2015 20:32:16
  FT1	37888	779,586,312	72.2 GB	21-Jan-2015 20:32:16
  LS1	37888	2,059,797,732	396.

•  

...

• 4 GB	21-Jan-2015 20:32:16

• 6/27/2015 - Block 4, through 24 June 2015 FINAL block for Pass8!!
• 8/5/2015 - Block 5, an emergency reprocess of 40 runs in the period 7/27-7/30/2015 which suffered bad CAL calibrations.

Configuration

...

_FT1variables_LS1variables

Data Product	destination	data content [1]	event selection [1]	makeFT1	gtselect	gtdiffrsp	gtmktime
FT1	FSSC
Notes
EXTENDEDFT1	SLAC	pass8	.2_Extended_cuts_L1	pass8.2_Transient_cuts_L1
LS1FT1	FSSC							pass8_FT1variables	evclass=128					produced from extendedFT1
EXTENDEDLS1	SLAC	pass8	.2_Extended_cuts_L1	pass8.2_Transient_cuts_L1
LS1	FSSC	pass8_LS1variables	evclass=65544					produced from extendedLS1

[1] /afs/slac/g/glast/ground/releases/volume04/evtClassDefs/01-01-0004/data

Note on 'Code Variant': The SLAC batch farm contains a mixture of architectures , both hardware (Intel/AMD 64-bit) and software (RHEL5-64, gcc v4.1, etc.). At this time, GlastRelease builds only on RHEL5-32 (RHEL6-64 are built but not yet validated), while ScienceTools builds for RHEL5-64, RHEL6-64.

...