Versions Compared

Old Version 16

changes.mady.by.user Tom Glanzman

Saved on

compared with

New Version 17

changes.mady.by.user Tom Glanzman

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Block

First run
MET

First run
UTC

Last run
MET

Last run
UTC

#Runs

#Merit Evts

DataCatalog Source

Note

1

239557414

2008-08-04 15:43:34

242047683

2008-09-02 11:28:02

431

909,050,672

/Data/Test/Flight/Repro/ReproTest7

reprocessed for alignment fix

1

242053458

2008-09-02 13:04:17

307579060

2010-09-30 22:37:38

11413

24,906,606,023

/Data/Flight/Level1/LPA/

Standard Level 1 output (ignore StdIntent)

1 Subtotal

 

 

 

 

11844

25,815,656,695

 

 

2

307585048

2010-10-01 00:17:26

311108362

2010-11-10 18:59:20

617

1,368,042,493

/Data/Flight/Level1/LPA/

Standard Level 1 output

 

 

 

 

 

 

 

/Data/Flight/Level1/LPA/

Standard Level 1 output
changed to correct gtdiffrsp parameters

Grand Total

 

 

 

 

12461

27,183,699,188

 

 

...

This task is roughly equivalent to the P105-FT1 task but with modernization.

Status chronology

  • 11/12/2010 - discover that gtdiffrsp params were incorrect (evtclass parm ignored, so all events got all diff rsp calc). Next (last) backfill block will be corrected.
  • 11/11/2010 - gear up for reprocessing data block2
  • 10/18/2010 - 146 mergeClumps jobs ran out of CPU time. Changed to xxl batch queue and rolled back.
  • 10/14/2010 - Validation complete, begin production for data block1
  • 10/1/2010 - Begin task construction

...

Data Product

makeFT1

gtdiffrsp

gtmktime

gtltcube

FT1

true

true for
evclass==3,4 (incorrect)

evclsmin==3,4 (correct)

true

false

LS1

true

false

true

false

...

Note on diffuse response calculation: gtdiffrsp is called two times in succession. The first time with For the bulk of this processing, the incorrect parameter was used for gtdiffrsp (IRF P6_V3_DIFFUSE and evclass==3, followed by and IRF P6_V3_dataclean and evclass==4). After block2 this was corrected to IRF P6_V3_DIFFUSE and evclsmin==3, and IRF P6_V3_DATACLEAN and evclsmin==4. The resulting FT1 file has four columns of diffuse response, two columns (files have five columns for diffuse response of which two pairs are filled in for DATACLEAN, one pair for DIFFUSE and zero for all other events. Each pair contains galactic and extragalactic response) for each of the two IRFs.

Timing

  • The most time-consuming step in P116-FT1 is the 'mergeClump' step wherein the diffuse response is calculated (twice). The CPU time required for this step, after ~2300 jobs have completed appears in the following plot:

    Thus, depending on the run and the speed of the batch machine (bimodal distribution for fell vs hequ), the jobs are finishing up after anywhere from 2 hours to 11 hours, with the most probable being about 5 hours.

...