Request to deploy L1Proc 5.7

I see that 4 runs have been processed. Where can we find the FT1 files ? Has someone checked that they are identical to the ones reprocessed by Nicola ?

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/electronFT1/gll_el_p302_r0557032605_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedFT1/gll_xp_p302_r0557032605_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedLS1/gll_xe_p302_r0557032605_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p302_r0557032605_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2/gll_pt_p202_r0557032605_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2Seconds/gll_pt1s_p203_r0557032605_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ls1/gll_ev_p302_r0557032605_v000.fit

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/electronFT1/gll_el_p302_r0556844466_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedFT1/gll_xp_p302_r0556844466_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedLS1/gll_xe_p302_r0556844466_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p302_r0556844466_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2/gll_pt_p202_r0556844466_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2Seconds/gll_pt1s_p203_r0556844466_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ls1/gll_ev_p302_r0556844466_v000.fit

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/electronFT1/gll_el_p305_r0500005777_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedFT1/gll_xp_p305_r0500005777_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedLS1/gll_xe_p305_r0500005777_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p305_r0500005777_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2/gll_pt_p305_r0500005777_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2Seconds/gll_pt1s_p305_r0500005777_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ls1/gll_ev_p305_r0500005777_v000.fit

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/electronFT1/gll_el_p305_r0400004554_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedFT1/gll_xp_p305_r0400004554_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedLS1/gll_xe_p305_r0400004554_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p305_r0400004554_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2/gll_pt_p305_r0400004554_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2Seconds/gll_pt1s_p305_r0400004554_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ls1/gll_ev_p305_r0400004554_v000.fit

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/merit/r0557032605_v000_merit.root

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/merit/r0556844466_v000_merit.root

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/merit/r0500005777_v000_merit.root

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/merit/r0400004554_v000_merit.root

I reran 557032605 from the start. Hope these are better.

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/electronMerit/r0557032605_v001_electronMerit.root

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedFT1/gll_xp_p305_r0557032605_v001.fit

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedLS1/gll_xe_p305_r0557032605_v001.fit

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p305_r0557032605_v001.fit

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2/gll_pt_p305_r0557032605_v001.fit

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2Seconds/gll_pt1s_p305_r0557032605_v001.fit

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ls1/gll_ev_p305_r0557032605_v001.fit

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/merit/r0557032605_v001_merit.root

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p305_r0561647741_v003.fit

root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p305_r0526392606_v003.fit

If I fdiff the gll_ph_p305_r0561647741_v003.fit file from the V5.7 pipeline against the gll_ph_p305_r0561647741_v305.fit file from the reprocessing, I see the header differences in the times Nicola mentions but also differences in the LIVETIME column start at row 6732.

Here are some of the rows just before and after the difference:

gll_ph_p305_r0561647741_v305.fit
        TIME                    EVENT_ID    RUN_ID      LIVETIME
        s                                               s
   6730   5.616498884017138E+08     4116530   561647741   3.024368094205856E+02
   6731   5.616498885720376E+08     4116850   561647741   3.026071331501007E+02
   6732   5.616498895114102E+08     4118642   561647741   1.795598268508911E-01
   6733   5.616498897924322E+08     4119154   561647741   4.605817794799805E-01

gll_ph_p305_r0561647741_v003.fit
        TIME                    EVENT_ID    RUN_ID      LIVETIME
        s                                               s
   6730   5.616498884017138E+08     4116530   561647741   3.024368094205856E+02
   6731   5.616498885720376E+08     4116850   561647741   3.026071331501007E+02
   6732   5.616498895114102E+08     4118642   561647741   3.035465058088303E+02
   6733   5.616498897924322E+08     4119154   561647741   3.038275277614594E+02

If I check against the p302 file, it has the same LIVETIME for the same events as the reprocessed file (I removed one event that isn't in the p305 files):

 gll_ph_p302_r0561647741_v001.fit
 TIME                    EVENT_ID    RUN_ID      LIVETIME
      1   5.616498884017138E+08     4116530   561647741   3.024368094205856E+02
      2   5.616498885720376E+08     4116850   561647741   3.026071331501007E+02
      4   5.616498895114102E+08     4118642   561647741   1.795598268508911E-01
      5   5.616498897924322E+08     4119154   561647741   4.605817794799805E-01

Giacomo says the LIVETIME column doesn't matter in the FT1 files, but I'd still like to know why it changed in V5.7 pipeline.

Here's the full output from fdiff for the run.

diff_r0561647741.txt

So, this is what I have found out.

1. For the difference in the LIVETIME column: the reprocessing reprocesses one run at the time, without splitting the file in Chunks, while the L1 split the run in chunks. It is natural that the LIVETIME column gets reset at the beginning of each chunks, therefore is normal that the behavior is different. The LIVETIME column is anyway useless, obsolete and should not even be in the FT1 file. I have no idea why is there in the first place, so the difference should not be a problem
2. For the HEADER of the file. This is more subtle and I need to trace down the differences, but I have checked the GTI and these are correct and identical (and corresponds, as they should, with the first and last entry of the FT2 file). If FSSC overwrite these entries, I think that this should not be a concern either.

From the FSSC perspective, the time values in the run file headers aren't particularly important.  The run files get split apart into various HTM pixels when ingested by the data server. I think the photon weekly files do just copy the start/stop values for a week from the earliest and latest run files for the week but a small difference shouldn't be an issue.  It looks like the TSTOP value is always set to be a second or so larger than the time of the last photon anyway. 

At Wednesday's meeting, I thought someone said that the header keywords did matter. Did I misunderstand?