Page History

...

• we are trying to collect information about upcoming Fermi computing outages (disks, oracle, network) to improve planning
• when planning an outage, please send an email to datalist and write the description here (including requested duration and preferred timeframe)
• we will try to combine outages as much as possible, in order to maximize uptime for time-critical services (FASTCopy, pipeline, etc.)
• once the plan is finalized, don't forget to send a message to glast-outage and the collaboration (if applicable)

Dec 2019 Power Outage (Fermi)

Dec 2017 Power Outrage (Fermi)

Upcoming outage requests

• Outage of mysql-node03 to move to HA rack.

Feb 03, 2014 - Oracle and OS patching (ghost vulnerability patches)

• Outage of FASTCopy starting at 9:00am, reboot of FASTCopy machines
• Oracle OS reboot and patching starting at 10:00am
• Reboot Fermi linux xrootd servers and fermilnx machines

Feb 11, 2014 - Oracle and OS patching; also retirement of various glastlnx machines

• 10am  - duration is likely several hours
• This outage affects all NFS servers (wains), including user disk as well as xroot servers.
• Expect interruptions in all Fermi services as they are moved from old glastlnx -> new fermilnx machines

Dec 11, 2013 - Oracle server battery replacement

• 10am - glast-oracle03 to have battery replaced in storage array.  Expected outage duration: 30m

Dec 4, 2013 - OS Patching and re-IP'ing

• 10am - all Fermi wain-class servers will be rebooted for OS patching.  
• 10am - glast-oracle03/04 will be rebooted for OS patching.
• At the same time, 16 wains will have new IP addresses assigned in anticipation of retiring old network switches and reconfiguring the network in January 2014.
• Two Three wains will be physically relocated to consolidate rack space

...