The Dell MD3460 controller array can support up to 120 drives with an MD3060e expansion tray attached. Single drive capacities have been steadily increasing, but transfer speeds have remained fairly constant. As a result, RAID6 rebuild times can exceed 24hrs when using 4TB drives. This is the main reason for choosing Dynamic Disk Pools over RAID6 - (DDP) provide lower rebuild times are possible since the RAID compared to RAID6 because the parity data is spread across all drives in the pool. For the latest Fermi storage configuration, we are benchmarking eight 15-disk DDP volumes on each 120-drive MD3460+MD3060e storage config. Both arrays are dual-attached to a failover pair of servers (fermi-gpfs01,2). In a failover scenario, one machine could serve all sixteen DDP volumes.
The sgpdd-survey benchmark performs parallel I/O using raw unformatted LUNs. The tests spawn an increasing number of threads (thr) that write and read to an increasing number of concurrent regions (crg). As the tests progress, I/O becomes more random. The transfer operations use 1MB records/chunks (rsz). This is a local test that does not require any networking or formatted filesystem. The purpose is to understand the performance limits of the storage hardware before we create filesystems (GPFS, Lustre, etc.). The benchmark results show the MD3460 is capable of sustaining throughput well above 2GB/sec. In order to avoid networking bottlenecks, we may use 4x10Gb ethernet link aggregation instead of the usual 2x10Gb. We also have a limited number of 40Gb ports on the SLAC network.
One 15-disk LUN
Max write ~1GB/sec , Max read ~1GB/sec
...