HPSS Prod2 Setup

hpss prod2 is a new HPSS setup that employs 1TB T10K data cartridges. The prod2 setup has

• four tape drives
• two data movers each
  • 1Gb Ethernet per mover
  • Fibre Channel connection to tape drives
  • About a total of 5TB disk cache.
    The data movers use 1GbE to connect to the network and Fibre Channel to connect to the tape drives.

Transferring a single

About 100GB of single files were transfered from one data server to HPSS. The transfer rate is about 55MB/s for larger files (> 100MB/s). For files below 100 MB rate measurement was accurate as the time resolution was only 1s.

Transfer 8TB of data (test 20090909)

About 8TB of data was transfered from eight servers (~1TB/server). After about 10h transfers started to fail because the disks on the movers were filled. At that time 6.5 TB were transfered.

The average transfer rate from disk to tape was about 78MB/s. Only one tape drive at a time was allowed for writing
and therefore files cold not be copied fas enough to tape and the cache disks filled up.

Transfer 8TB of data (test 20091007)

Another test transferring about 8TiB of data was done. This time three out of the four HPSS tape drivers were allowed for writing. The 8TiB of data were spread over eight data servers and the migration to HPSS was running in parallel on all eight servers. Below are some stats for the transfers:

While the transfers were running the network IO and (for some time) the disk IO on the two HPSS movers were recorded.
The HPSS instance has two disk arrays that are accessed by both movers and show up as devices sdc and sdd.

The disk IO was not monitored during the first 7:30 hours therefore no data are shown for this time in the disk IO plots.

Reading files from HPSS

Reading a single file, tape needs to be mounted

Single files were read from HPSS. The next table shows the time it took to transfer a file from HPSS to a server.
To transfers were done. For the first one the tape was not mounted so this transfer includes the overhead due to
mounting and positioning the tape. For the second transfer the file was already in the HPSS disk cache.

Reading a few files from the same tape

Four files were read from the same tape. All file were between 1.1GB to 1.5 GB large. For the first file (most likely) the tape had to be remounted. The file was read twice the first time from disk and the second time from the disk cache. The figure below shows the effective transfer rates for the transfers and the time difference between reading from tape and disk (first-second). The position of the files on tape is also shown.

Reading sequentially all files from a tape

All 377 files were read from the tape H0000700, except the first file as this one was split across two tapes. One client was used to read the file sequentially. The checksum and size of the transfered files matched the ones obtained from the xrootd cluster.
The lower figure below shows the elapsed time for each transfer versus the file size. The blue line shows the relation between elapsed time and file size to be:

elap = 1.5s * size[MB] / 45[MB/s]

The 1.5 as well as the 45 approximate values.
The upper figure shows the elapsed time subtracted by the expected transfer time. The expected transfer time was assumed to be the file size divided by 45MB/s.