Versions Compared

Old Version 6

changes.mady.by.user Maria Elena Monzani

Saved on

compared with

New Version 7

changes.mady.by.user Tom Stephens

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. (tick) Define the name of the columns
  2. (tick) Implement the new columns in fitsGen, which means adding them to the template FT2 file: https://github.com/fermi-lat/fitsGen/blob/master/data/ft2.tpl  Note: If things have been implemented as they should, no change to the actual code should be necessary.
  3. (tick) Change the ft2 code. to get the velocities from the magic7 and save them in the new columns
    1. (tick) Create pipeline tasks to reprocess the FT2 data
    2. (tick) Create pipeline tasks to properly apply the Bad Time interval data.
  4. Test everything end-to-end with a sample of the new ft2 files. (by this I mean the pulsar software)
    1. (tick) Use the code to reprocess 1-2 weeks of data for testing. Q: Is there any particular time period that would be useful or would any period be fine?
      1. we reprocessed a two-week chunk of data from runs 545208791 to 546407418.  This included several files with BTI data in the database
    2. Ingest files at FSSC - Don/Alex
    3. Verify that the BTI data is correct - Tom
    4. Test with Pulsar Tools - Dave Smith
    5. Any other testsWho should be doing this testing?
  5. Get CCB approval to make the change operational
    1. I've never done this before so I don't know the process here either - T.S. -> MEM: I'll prepare the CCB request once validation is completed 
  6. Release a new GlastRelease
    1. What is the process for doing this? MEM will ask Heather
    2. Do we need a completely new GlastRelease or just the updated L1PROC codebase? MEM: Yes, we need a new version of GR because of the updated fitsGen. But we only need this for the new version of L1Proc, not for the reprocessing.
  7. Reprocess the FT2 files and ingest them in the data catalog. (Usually done in two steps. Initial reprocess & backfill)
    1. Install the reprocessing tasks in the PROD pipeline
    2. The reprocessing step will be putting them into /Data/Flight/Reprocess/P310, how do we move them to /Data/Flight/Level1/LPA (where the current version is) when we are ready to go? MEM: we absolutely don't move the files from one place to the other. They will stay in the reprocessing folder. The only place where they are merged is at the FSSC photon database.
  8. Instruct L1 to use the new GlastRelease  Note: 6 backfill & 7 must be coordinated and timed together.
  9. Remove old FT2 from public server and re-ingest new files. MEM: removal doesn't actually happen. The files are transferred to the FSSC and ingested into a new DB. Once that's done and L1 is moved to the new version, it's just a softlink to be switched from the old to the new DB. Important: this step actually happens before 8. We need an almost complete DB before making the L1 switch. Then there is a short (1-2 days) backfill after the L1 switch happens.
  10. Update File Format Document & Interface Control Document

...