Overview

We have decided to set up virtual machines on the Fermilnx boxes. This will enable us to map existing glastlnx boxes onto virtual machines, to first approximation keeping the existing break down of applications/ (virtual) servers.

Proposed mapping

See also: Applications on glastlnx machines and fermilnx migration, Servers and Aliases, Web Server Applications, Tom's migration spreadsheet, Migration Check List.

Existing

New

Aliases

Status (date)

Comment

glastlnx03

fermilnx-v07 (HA)

glast-xrootd01
glast-tomcat01

Oct 14: Tomcat Migration Complete
Oct 14: updated crontabs for tomcat scripts. Turned off tomcat server on glastlnx03.

 

glastlnx04

fermilnx-v08 (HA)

glast-rdr
glast-tomcat02
glast-jobcontrol01

Oct 14: Tomcat Migration Complete.
There are no web applications on this server. 
Oct 14: updated contrabs for tomcat scripts. Turned off tomcat server on glastlnx04

Oct:14 The server is NOT running. It has not been completely installed.

This node runs job control daemons for several accounts, at least glast, glastraw,glastmc

glastlnx05

fermilnx-v12

glast-rdr
glast-tomcat03

Oct 14: Migration Complete
This is the dev server.

Oct 14: crontab migrated.

 

glastlnx07

fermilnx-v14

centaurusa 
glast-java
svn
glast-heprep01

 

This machine requires user login. This machine is used as a Fermi CVS server, and a subversion server for a variety of user groups. svn functionality should move elsewhere.

glastlnx08

fermilnx-v19

glast-tomcat04

Oct 9: This is where the Elog runs. We have to be careful when moving it. The elog entries are currently on /nfs/farm/g/glast/u25/tomcat/glast/writable/glastlnx08/webapps.
Should they be moved?

This is used as a test SRS web server -- this should move somewhere else. Decorator -- could move elsewhere? Xroot proxy?

glastlnx09

 

 

 

This is now used by EXO as a database server. A replacement machine is (about to be) ordered.
This machine is used as a CVS server for lcsim and a few other project. Should be shutdown and projects moved elsewhere.

glastlnx10

fermilnx-v05

glast-tomcat08

 

 

glastlnx12

fermilnx-v13 
(Dedicated NIC)

glast-tomcat05

Merged Prod Pipeline (08/29/13)

 

glastlnx13

fermilnx-v16

glast-tomcat06

Migrated Dev Pipeline (07/19/13)
Migrated Dev Crawler (07/19/13)
Migrated Astroserver  (07/19/13)

 

glastlnx16

fermilnx-v17

glast-tomcat09

 

 

glastlnx17

fermilnx-v18

glast-tomcat10

Oct 14: Web Migration complete. Crontabs updated. Turned off tomcat server on glastlnx17.

 

glastlnx18

fermilnx-v10

glast-tomcat11

Oct 14: Migration Complete.
Oct 14: Updated tomcat crontab.
Turned off tomcat server on glastlnx18

There is also a SRS tomcat server in this machine -- it should migrate somewhere else

glastlnx19

fermilnx-v11

glast-tomcat12

Oct 14: Migration Complete.
Oct 14: Updated tomcat crontab. Turned off tomcat server on glastlnx19

There is also an EXO tomcat server on this machine -- it should migrate somewhere else

glastlnx20

fermilnx-v03

glast-test-rdr

Migrated (05/14/13)

Xrootd test redirector

glastlnx21

fermilnx-v04

 

Migrated Datacatalog Crawler (07/19/13)

Production datacatalog crawler

glastlnx22

fermilnx-v06

glast-xrd-xfer
glast-test-rdr


Migrated (05/14/13)

Xroot proxy server

glastlnx23

fermilnx-v15

 

 

 

Next Steps

We should identify a set relatively standalone services to move to fermilnx-v* machines. Some suggestions:

  • Test tomcat server
  • Data catalog crawler
  • Astro server loaders

As we move services we should also check which version of Java they are using, and where possible switch to Java 7.

We will need to move aliases to point to new machines, in a phased way. It would probably also be good to introduce new aliases for some services which do not currently have them.

Goals

We need to identify critical applications and estimate resource usage for these applications and spread them across the servers accordingly. Since we are using fewer servers, we are increasing the number of services per machine so we want to limit effects incurred from possibly misbehaving applications. I'm going to try to rate the applications on a scale of 1-5 in these 3 metrics. The first metric is the priority of the application during a normal power scenario, assuming SLAC is not in a power outage scenario. The second metric is the estimated CPU usage and memory usage of the application. The third metric is the possibility an application will misbehave and abuse resources for an extended amount of time.

Application

Priority/Criticality

CPU/Memory Requirements (average/peak)

Resource abuse potential

Pipeline (Production)

5

2/4

3

Pipeline Mail client

5

1/2

1

Pipeline Jobcontrol clients

5

1/2

2

Datacatalog server (crawler)

3

2/2

2

Astroserver (loader)

2

2/5

3

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Resource management

Ideally we could put all applications on a few machines without worrying about interference from other applications. In practice this isn't practical. It's been suggested to use virtual machines and split up services among the machines.

1 Comment

  1. Stephan Zimmer

    Hi,
    which of these fermilnx boxes will be pipeline-critical? I need the IP addresses of these to request an update to Lyons firewall in order to ensure continued operations.