Page History

...

2) Change the config_fk (mootkey) in MOOD's Acq_summary in the MOOD/FMX MySQL database.

• To access the database, you need to login to fermilnx02 (in 2023) and issue the command mysql -h glastMOOD -P8208 -u <username> -p. You will need to get a password from the secret/fpd/fermi/

  isoc 

  isoc vault.  Richard may need to put in a ticket for you to get permission. In 2023, we used Steve Tether's account (tether) since the "magician" account is overpowered for what we need (it's basically the super-user account).

• For SQL commands, see https://www-glast.stanford.edu/protected/mail/opsprob/10898.html. Before you have to activate the db with use mood;.

3) In some receipts (e.g. in older emails) is written that a repipe is neccessary.  Michael doesn't recall repipes for the 2017 key mismatch. At least no helpful repipes.  In 2023 for sure no repipes were performed. Don't repipe unless really neccessary!

4) Rollback the affected deliveries

• If they are in a Failed state, you can just do a standard rollback.

• If findChunks is stuck, then bkill the process and wait for the reaper to get it. Most of them should then auto rollback without intervention.

• Most likely someone has created run locks manually to halt runs from being processed with wrong moot keys and successively failing. Don't forget to remove these run locks.

History

• The MOOT Key Mismatch problem occurred on 2023 July 18. See the Fermi news item for 2023 July 20 at https://fermi.gsfc.nasa.gov/ssc/library/news/
  • 15 LPA runs from the LAT had to have their MOOT keys manually fixed in Oracle and MySQL before they could be processed correctly through L1:
    • 711402686, 711405073, 711411046, 711416992, 711422925, 711428850, 711434773, 711440537, 711446088, 711451782, 711457476, 711463169, 711468863, 711474557, 711478257
• The MOOT Key Mismatch problem occurred on 2017 November 30. See: https://www-glast.stanford.edu/protected/mail/opsprob/10893.html

...