Figuring out what we need to get through. None of the pages mapping services to servers are up to date.
In theory, everything we need except batch (and xroot?) is already in HA.
Note: the HA rack content is (or was) here: https://docs.google.com/spreadsheet/ccc?key=0AveNVqKMtmVEdDRmNTA1UVE1bDQ1YmV6ZWY0anBla2c#gid=0
The following table of servers must remain powered up and operational for Fermi Level 1 to succeed.
- Confirm current H.A. rack occupants (thanks Shirley!): https://portal.slac.stanford.edu/info/ITHelp/KB%20Assets/HA-Servers.xlsx
- Confirm the VM-master for a given VM. Use the 'node' command, e.g., $ node -whereis fermilnx-v12
- Confirm the tomcat <-> service associations. Table here: http://glast-ground.slac.stanford.edu/ServerMonitoring/exp/Fermi/applications.jsp?d-6677469-s=2&d-6677469-o=2
- Confirm that the tomcat-VM associations in this table.
- Question for Warren: do we not need the DataCatalog (tomcat08)?
| server | VM/service | function | HA rack |
|---|---|---|---|
| fermi-gpfs02 | xrootd server | ||
| fermilnx05-vmm | fermilnx-v02 | xrootd redirector | |
| fermilnx07-vmm | fermilnx-v12 | xrootd redirector | |
| wain031 (or equivalent) | NFS storage | yes | |
| fermilnx01 | LAT config, fastcopy and real-time telemetry | yes | |
| fermilnx02 | LAT config, fastcopy and real-time telemetry | yes | |
| fermilnx05-vmm | fermilnx-v03 | archiver | |
| fermi-oracle01 | oracle primary | yes | |
| fermi-oracle02 | oracle secondary | yes | |
| mysql05/06 | mysql-node03 | calibration, etc. DB | yes |
| hequNNN - hequNNN+24 | batch hosts | ||
| fermilnx03-vmm | fermilnx-v07/tomcat01 | Commons | yes |
| fermilnx09-vmm | fermilnx-v16/tomcat06 | rm2 | |
| fermilnx09-vmm | fermilnx-v17/tomcat09 | Pipeline-II | |
| fermilnx09-vmm | fermilnx-v18/tomcat10 | FCWebView, ISOCLogging, MPWebView TelemetryMonitor, TelemetryTableWebUI | |
| fermilnx07-vmm | fermilnx-v10/tomcat11 | DataProcessing | |
| fermilnx07-vmm | fermilnx-v11/tomcat12 | TelemetryTrending |
The services for L1:
oracle
- pipeline
- data catalog
- group manager
mysql
- calibrations
tomcats
- pipeline
- data catalog
- data processing
isoc servers
xroot
- fermi-gpfs02 (xrootd server)
- fermilnx-v02 (redirector)
- fermilnx-v12 (redirector)
nfs
- Pretty much everything that's currently on wain031
LSF
- ~25 hosts should let us keep up
Here's what ISOC tasks need:
FASTCopy chain
--------------
wain031
fermilnx01
fermilnx02
trscron
fermilnx-v03 (Archiver)
Whatever the pipeline server runs on.
xroot servers
astore system
Web servers
-----------
tomcat01 Commons
tomcat06 rm2
tomcat09 Pipeline-II
tomcat10 FCWebView, ISOCLogging, MPWebView
TelemetryMonitor, TelemetryTableWebUI
tomcat11 DataProcessing
tomcat12 TelemetryTrending
Notes:
9/29/2017 - (TG and WK) added list of xrootd servers needed to bridge the gap in December; Started table of server names