Page History

...

1. Define the name of the columns
2. Update File Format Document & Interface Control Document.
3. Implement the new columns in fitsGen, which means adding them to the template FT2 file: https://github.com/fermi-lat/fitsGen/blob/master/data/ft2.tpl  Note: If things have been implemented as they should, no change to the actual code should be necessary.
4. Change the ft2 code. to get the velocities from the magic7 and save them in the new columns
   1. Create pipeline tasks to reprocess the FT2 data
   2. Create pipeline tasks to properly apply the Bad Time interval data.
5. Test everything end-to-end with a sample of the new ft2 files. (by this I mean the pulsar software)
   1. Use the code to reprocess 1-2 weeks of data for testing.
      
      1. we reprocessed a two-week chunk of data from runs 545208791 to 546407418.  This included several files with BTI data in the database
      2. more files were reprocessed with the P310 task(s). This includes a variety of cases for BTIs and a selection of different DATA_QUALs.
   2. Check to make sure the velocities are being copied correctly from the magic7 packets and that they make sense based on the change in coordinates - Don/Tom.
   3. Ingest files at FSSC - Don/Alex
   4. Verify that the BTI data is correct - ME/Tom/Simone
   5. Test with Pulsar Tools - Dave Smith Smith 
   6. Non-pulsar tests - Joe Eggen.
   7. Check with the C& A group. Don brought it up at the 6/24 C&A group meeting.  No No one raised any issues (or volunteered to do any testing).
   8. Check with other mission elements. Don checked with Michelle Hui at the GIOC that the change won't affect them.
6. Get CCB approval to make the change operational.
   1. I've never done this before so I don't know the process here either - T.S. -> MEM: I'll prepare the CCB request once validation is completed 
7. Release a new GlastRelease
   
   1. What is the process for doing this? MEM will ask Heather
   2. Do we need a completely new GlastRelease or just the updated L1PROC codebase? MEM: Yes, we need a new version of GR because of the updated fitsGen. But we only need this for the new version of L1Proc, not for the reprocessing.
      
8. 1. Paul Ray checked the files against the velocities computed by PINT and found good agreement.
      
9. Fix all the runs that need repiping (Runs to be rePiped and reprocessed): still in progress.
   
10. Obtain a new GlastRelease:
    
    1. JIRA issue for the new GR: 

       Jira
       server SLAC National Accelerator Laboratory serverId 1b8dc293-975d-3f2d-b988-18fd9aec1546 key LPATE-198

       
       
    2. We also need to update the spacecraft's geoposition (

       Jira
       server SLAC National Accelerator Laboratory serverId 1b8dc293-975d-3f2d-b988-18fd9aec1546 key GRINF-76

       ). A new version of the astro package is included in the new GR.
11. Remake the P310 tasks
12. Re-reprocess a selection of files
13. Verify that the geo lattitude and longitude are properly created (Toby will do this).
    
    1. Repeat the checks in item 5. above (spot-checks are probably fine).
    2. Verify that FT1 files produced with the new GR are correct/unchanged from the previous version
14. Reprocess the FT2 files Reprocess the FT2 files and ingest them in the data catalog. (Usually done in two steps. Initial reprocess & backfill):
    
    1. Pick an end date/time for the reprocessing window, once we all agree that we are ready to go. 
    2. Create a list of runs for reprocessing and BTI flagging from the datacatalog. Validate against the list at the FSSC server.
    3. Install the reprocessing tasks in the PROD pipeline.
    4. The reprocessing step will be putting them into /Data/Flight/Reprocess/P310, how do we move them to /Data/Flight/Level1/LPA (where the current version is) when we are ready to go? MEM: we absolutely don't move the files from one place to the other. They will stay in the reprocessing folder. The only place where they are merged is at the FSSC photon database. TS: That makes things easier   Good to know.
    5. Tom estimates that the reprocessing may take ~1 week.
       
    6. .
    7. FSSC will pick them up from xrootd for ingestion directly.
    8. The reprocessing will take ~1 month. Started 2020-03-04.
       
15. Transfer the files to the FSSC and ingest them into the FSSC's data server.
    1. Have Wilko has setup a proxy server so Don can copy the files directly from xroot.
    2. Don copies files and does basic checks and archives them.
    3. Alex starts the process to ingest them into the data server database.  He estimates the ingest may take ~1 week.
    4. This may be able to take place in parallel with the reprocessing by transferring batches as they are reprocessed. 
    5. .
16. Create a new version of the L1Proc, rePipeL1 and Flag FT2 tasks.
    1. Update the International Geomagnetic Reference Field (IGRF).  See Updating the IGRF-13 implementation in the astro package.
    2. Test them meaningfully on DEV.
       1. including check that the FT1 files haven't changed.(see Checking FT1 files in the context of the FT2 reprocessing and Bad interpolation of FT2 information during processing).
    3.  Get CCB approval to make the change operational.
17. Coordinate a switchover date between ISOC and FSSC.
    1.  Create the backfill 
       1.  Reprocess 2020 runs with the new IGRF model. 
    2. Switch to the new L1 task(s).  Note: backfill and switch must be coordinated and timed together.
    3. Switch the data server to use the new files.  The FSSC has an internal wiki page on the steps for this.
     Switch L1 to use the new GlastRelease.  Note: backfill and switch must be coordinated and timed together.
    

...

DataCatalog Commands:

datacat find --sort nRun --filter 'nRun>=239557414 && nRun<=585049107' --group FT2 /Data/Flight/Level1/LPA > P310_FT2_bulk.txt

To be updated with a newer upper range run when the cutoff date is chosen.

Checking FT1 files in the context of the FT2 reprocessing