Confluence will be unusable 23-July-2024 at 06:00 due to a Crowd upgrade.
All times are PDT. Red entries are active. Most recent entry first.
Nodes | Services | Start Time | Expected End Time | Actual End Time | Reason | Comments | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
psananeh All | All | Sat, Mar 19th, 7am | Sat, Mar 19th, 7pm | Mon, Mar 21st, 10am | NEH power cut | Completed | |||||||||
All | All | Wed, Mar 23rd, 10am | Wed, Mar 23rd, 3pm |
| NEH power cut | This power cut was NOT planned | |||||||||
psana | Science data access | Thu, Mar 24, 11am | Thu, Mar 24, 5pm |
| Lustre testing | Completed | |||||||||
All | All | Sat, Mar 26th, 7am | Sat, Mar 26th, 7pm | Mon, Mar 28th, 1pm | NEH power cut | Completed | |||||||||
psana | Science data access | Thu, Mar 31st, 11am | Thu, Mar 31st, 5pm |
| Enabling HA for Lustre system | Completed | |||||||||
psana | Science data access | Fri, Apr 1st, 6pm | Mon, Apr 4th, 10am |
| NEH cooling outage | Completed | |||||||||
psana | Science data access | Thu, Apr 28, 2pm | Thu, Apr 28, 6pm | Thu, Apr 28, 3pm | Lustre maintenance . | Completed | |||||||||
lclsq | NEH storage and processing | Mon, Dec 19, 7am | Thu, Dec 22, 4pm |
| Chilled water outage |
| |||||||||
psana batch nodes | All Science data is currently unavailable. Psananeh psanafeh is up for Matlab use, but no access to data on Lustre file system. | Saturday, Oct 1, 6am |
|
| Lustre file system remains down after the unplanned power outage on Saturday. | The system administrators are working to bring them back. |
|
| |||||||
| All LCLS computing services | Monday Nov 14, 7am |
|
| Electrical work at NEH server room and FEH. | pslogin is up. NFS server, LDAP, DNS, pswww are up. |
|
| |||||||
psana, | All Science data, All user home directories, all DAQ cache nodes. All online services. | Wed, Sep 28, 10am | Wed, Sep 28, 6pm | Wed, Sep 28, 6pm | Upgrade of Lustre hardware. |
|
|
| |||||||
psana | Science data access | Tue, Sep 20, 11.15am |
| Tue, Sep 20, 6.15pm | NEH power outage | B950 and several other buildings experienced short power glitch but the lustre file servers did not survive the interruption and is still being brought up. |
|
| All | All | Fri, Apr 29, 6.30pm | Sun, May 1st, 11pm | Sun, May 1st, 9pm | NEH power outage | Completed |
psana | Science data access | Thu, May 5th, 1pm Jun 2, 1pm | Thu, Jun 2, 5pm |
| Lustre failover testing. |
| |||||||||
NEH online nodes | NEH DAQ, outside ssh access |
| Thu, May 5th25th, 5pm |
| Server room upgrade, ana02 memory upgrade Lustre maintenance | Completed | |||||||||
psana | Science data access | Thu, May 12, 1pm | Thu, May 12th, 6pm | Thu, May 12th, 6.30pm | Lustre maintenance | Completed. Upgraded memory on psanaoss101-104, and replaced 10Gbit cards with 1 port SMCs. 717W power supplies are in place on psanaoss103-104 now. | |||||||||
psana | Science data access | Thu, May 5th, 1pm | NEH online nodes | NEH DAQ, outside ssh access | Thu, May 25th5th, noon 5pm | Thu, May 25th, 7pm |
| 5th, 5pm | Lustre maintenance | Completed | |||||
All | All | Fri, Apr 29, 6.30pm | Sun, May 1st, 11pm | Sun, May 1st, 9pm | NEH power outage Server room upgrade, ana02 memory upgrade | Completed | |||||||||
psana | Science data access | Thu, Apr 28, 2pm | Thu, Jun 2Apr 28, 1pm 6pm | Thu, Jun 2, 5pm |
| Lustre failover testing. | Apr 28, 3pm | Lustre maintenance | Completed | ||||||
psana | Science data access | Tue Fri, Sep 20, 11.15am |
| Tue, Sep 20, 6.15pm | NEH power outage | B950 and several other buildings experienced short power glitch but the lustre file servers did not survive the interruption and is still being brought up. | |||||||||
psana, | All Science data, All user home directories, all DAQ cache nodes. All online services. | Wed, Sep 28, 10am | Wed, Sep 28, 6pm | Wed, Sep 28, 6pm | Upgrade of Lustre hardware. |
|
|
| |||||||
psana batch nodes | All Science data is currently unavailable. Psananeh psanafeh is up for Matlab use, but no access to data on Lustre file system. | Saturday, Oct 1, 6am |
|
| Lustre file system remains down after the unplanned power outage on Saturday. | The system administrators are working to bring them back. |
|
| |||||||
Apr 1st, 6pm | Mon, Apr 4th, 10am |
| NEH cooling outage | Completed | |||||||||||
psana | Science data access | Thu, Mar 31st, 11am | Thu, Mar 31st, 5pm |
| Enabling HA for Lustre system | Completed | |||||||||
All | All | Sat, Mar 26th, 7am | Sat, Mar 26th, 7pm | Mon, Mar 28th, 1pm | NEH power cut | Completed | |||||||||
psana | Science data access | Thu, Mar 24, 11am | Thu, Mar 24, 5pm |
| Lustre testing | Completed | |||||||||
All | All | Wed, Mar 23rd, 10am | Wed, Mar 23rd, 3pm |
| NEH power cut | This power cut was NOT planned | |||||||||
All | All | Sat, Mar 19th, 7am | Sat, Mar 19th, 7pm | Mon, Mar 21st, 10am | NEH power cut | Completed | All LCLS computing services | Monday Nov 14, 7am |
|
| Electrical work at NEH server room and FEH. | pslogin is up. NFS server, LDAP, DNS, pswww are up.
|
|