You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

18 Feb 2015

Intro

Fermi is looking to purchase two storage building blocks with ~640 TB each year with 1/2 to accommodate new data, and the other 1/2 to allow for aged equipment retirement/repurposing.  The following proposal resulted from a meeting held 2/18/2015 with Shirley, Yemi, Lance, Wilko, Richard, and Tom.

Fermi currently supports an xroot service consisting of 33 Sun Thumper/Thor class servers plus 12 Dell R610/620 class servers.  Fermi also supports an NFS service consisting of four Sun Thor class servers.  One additional Thumper class server is used by LSST.  The Sun servers range in age from 4 to 8 years.  The oldest Dell systems are three years old.

 

2015 Program

  • First experience with GPFS.  Use new servers, fermi-gpfs01 and fermi-gpfs02 exclusively as xroot resources running GPFS
  • Retire six wains which have the highest failure rate: 053, 054, 055, 056, 069, 071 (??)
  • Modernize and improve NFS system, with the special aim of increasing and improving service for the User and Group spaces currently on wain025
  • Update the LSST storage server, wain006
  • Donate left-over wains to Steffen Luitz for LZ use at IR-2

Proposed configuration changes for Fermi xroot and NFS servers

  • New servers: fermi-gpfs01 and fermi-gpfs02
    • Dual-connect storage between these two machines
    • Internet connectivity (2 x 10 Gbps per host?)
    • Install GPFS
    • Install xrootd
    • Balance data across xroot cluster
  • New NFS/GPFS service on former fermi-xrd01 and fermi-xrd02 (and wainXXX, wainYYY)
    • fermi-xrd01 and fermi-xrd02 
      • Drain xroot data (~180 TB)
      • Swap 2x R610 with SCS-owned R720 machines
      • Rename (fermi-gpfs03 and fermi-gpfs04?)
      • Internet connectivity (1 x 10 Gbps per host?)
      • Decide upon storage configuration
        • Number of spindles for Users/Groups (from wain025)
        • Number of spindles for production partitions (from wain026 and wain032)
        • Number of "spare" spindles for future expansion
      • Install GPFS (total capacity will be ~160 TB)
      • Migrate wain025, wain026, wain032 to new system
    • Drain xroot data from wainXXX and YYY (~64 TB)
      • Rename hosts (fermi-cnfs01, fermi-cnfs02?)
      • Install GPFS and CNFS software
      • Configure in such a way that wain025 partitions are handled in such a way so that service does not negatively impact other partitions
  • Upgraded NFS service for LSST
    • Repurpose wain025 (??) to replace wain006
  • Drain xroot from other wains that are to be retired

Timeline

  • 7/1/2014 - new servers arrive
  • 7/30/2014 - storage arrays arrive
  • 9/18/2014 - cables located, beginning of GPFS testing
  • 1/13/2015 - xroot in production (readonly), and data migration begins

 

Ref: NFS and Xroot Disk Allocations

  • No labels