Summary
We request to put in production several new tags (listed below).
Reason for change
Pass 8.3 aka P305.
Handle building components from github.
Test Procedure
We have processed several full LPA runs:
And reprocessed ft2 files for the same runs:
Rollback procedure
We can easily switch back to previous version of L1Proc .
CCB JIRA
- SSC-428Getting issue details... STATUS
Details
https://confluence.slac.stanford.edu/display/SCIGRPS/Release+of+P305+data+and+updated+FT2+files
Complete set of tags for L1Proc 5.7
GlastRelease: GlastRelease-20-10-04-gr02
ScienceTools: ScienceTools-11-05-01
svac/L1Pipeline: L1Pipeline-05-07-00
calibTkrUtil: calibTkrUtil-04-00-00
fitsGen: fitsGen-07-01-03
dataMonitoring/AlarmsCfg: AlarmsCfg-07-00-12 (Why is this here? It has its own CCB process. And as long as I'm going off in this direction, shouldn't calibrations have some kind of control?)
dataMonitoring/Common: Common-07-00-01
dataMonitoring/DigiReconCalMeritCfg: DigiReconCalMeritCfg-02-01-08
dataMonitoring/FastMon: FastMon-05-03-05
dataMonitoring/FastMonCfg: FastMonCfg-02-02-03
datMonitoring/IGRF: IGRF-03-01-04
svac/Monitor: Monitor-03-14-03
svac/TestReport: TestReport-12-03-02
svac/findGaps: findGaps-02-03-00
users/richard/pipelineDatasets: pipelineDatasets-01-00-00
ft2Util: ft2Util-02-04-00
evtClassDefs: evtClassDefs-01-01-05
GPLtools: GPLtools-02-00-00-wf03
ROOT: ROOT 5.34.03-gr01
12 Comments
Philippe Bruel
I see that 4 runs have been processed. Where can we find the FT1 files ? Has someone checked that they are identical to the ones reprocessed by Nicola ?
simone maldera
Hi Philippe,
I posted some partial check here:
Checks on L1proc-dev FT1 files
Warren Focke AUTHOR
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/electronFT1/gll_el_p302_r0557032605_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedFT1/gll_xp_p302_r0557032605_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedLS1/gll_xe_p302_r0557032605_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p302_r0557032605_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2/gll_pt_p202_r0557032605_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2Seconds/gll_pt1s_p203_r0557032605_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ls1/gll_ev_p302_r0557032605_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/electronFT1/gll_el_p302_r0556844466_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedFT1/gll_xp_p302_r0556844466_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedLS1/gll_xe_p302_r0556844466_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p302_r0556844466_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2/gll_pt_p202_r0556844466_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2Seconds/gll_pt1s_p203_r0556844466_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ls1/gll_ev_p302_r0556844466_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/electronFT1/gll_el_p305_r0500005777_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedFT1/gll_xp_p305_r0500005777_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedLS1/gll_xe_p305_r0500005777_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p305_r0500005777_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2/gll_pt_p305_r0500005777_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2Seconds/gll_pt1s_p305_r0500005777_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ls1/gll_ev_p305_r0500005777_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/electronFT1/gll_el_p305_r0400004554_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedFT1/gll_xp_p305_r0400004554_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedLS1/gll_xe_p305_r0400004554_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p305_r0400004554_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2/gll_pt_p305_r0400004554_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2Seconds/gll_pt1s_p305_r0400004554_v000.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ls1/gll_ev_p305_r0400004554_v000.fit
Warren Focke AUTHOR
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/merit/r0557032605_v000_merit.root
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/merit/r0556844466_v000_merit.root
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/merit/r0500005777_v000_merit.root
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/merit/r0400004554_v000_merit.root
Warren Focke AUTHOR
I reran 557032605 from the start. Hope these are better.
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/electronMerit/r0557032605_v001_electronMerit.root
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedFT1/gll_xp_p305_r0557032605_v001.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/extendedLS1/gll_xe_p305_r0557032605_v001.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p305_r0557032605_v001.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2/gll_pt_p305_r0557032605_v001.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft2Seconds/gll_pt1s_p305_r0557032605_v001.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ls1/gll_ev_p305_r0557032605_v001.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/merit/r0557032605_v001_merit.root
Warren Focke AUTHOR
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p305_r0561647741_v003.fit
root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p305_r0526392606_v003.fit
Nicola Omodei
I checked this the two runs above (I only show the result for the first but the same holds for the second run: root://glast-test-rdr.slac.stanford.edu//glast/Data/Flight/Level1/LPA/dev/5.7/ft1/gll_ph_p305_r0561647741_v003.fit) comparing the one reprocessed (P305) and the one in the old (v5.6) L1 pipeline and one in the new (v5.7):
I have found the following differences that I don't understand:
Left: L1 v5.6, center L1 v5.7 and right REP 305
In the headers of the 2 extensions:
There is:
1 second of difference in the HEADER file entries between the L1 and the reprocessed: DATE-OBS (but no difference in the MET entry TSTART)
2 seconds in the HEADER file entries: DATE-END (but 1 second in the TEND)
1 second of difference in the TELAPSE between the L1 files and the reprocessed files.
The GTI themselves (second extension in the Ft1 file) are identical. The events are identical between the reprocessed P305 and the L15.7 (obviously are different than the L1 v5.6).
Don Horner
If I fdiff the gll_ph_p305_r0561647741_v003.fit file from the V5.7 pipeline against the gll_ph_p305_r0561647741_v305.fit file from the reprocessing, I see the header differences in the times Nicola mentions but also differences in the LIVETIME column start at row 6732.
Here are some of the rows just before and after the difference:
If I check against the p302 file, it has the same LIVETIME for the same events as the reprocessed file (I removed one event that isn't in the p305 files):
Giacomo says the LIVETIME column doesn't matter in the FT1 files, but I'd still like to know why it changed in V5.7 pipeline.
Here's the full output from fdiff for the run.
diff_r0561647741.txt
Nicola Omodei
So, this is what I have found out.
Don Horner
From the FSSC perspective, the time values in the run file headers aren't particularly important. The run files get split apart into various HTM pixels when ingested by the data server. I think the photon weekly files do just copy the start/stop values for a week from the earliest and latest run files for the week but a small difference shouldn't be an issue. It looks like the TSTOP value is always set to be a second or so larger than the time of the last photon anyway.
Nicola Omodei
Right. As long as the GTI are correct, I think this shouldn't be any issue.
Warren Focke AUTHOR
At Wednesday's meeting, I thought someone said that the header keywords did matter. Did I misunderstand?