Table of Contents | ||||||
---|---|---|---|---|---|---|
|
hpss prod2 is a new HPSS setup that employs 1TB T10K data cartridges. The prod2 setup has
...
While the transfers were running the network IO and (for some time) the disk IO on the two HPSS movers were recorded.
The HPSS instance has two disk arrays that are accessed by both movers and show up as devices sdc and sdd.
The disk IO was not monitored during the first 7:30 hours therefore no data are shown for this time in the disk IO plots.
Single files were read from HPSS. The next table shows the time it took to transfer a file from HPSS to a server.
To transfers were done. For the first one the tape was not mounted so this transfer includes the overhead due to
mounting and positioning the tape. For the second transfer the file was already in the HPSS disk cache.
| first access | second access | delta |
---|---|---|---|
file size | elapsed [s] | elapsed [s] | [s] |
31680 | 115 | 2 | 113 |
555840 | 107 | 2 | 105 |
1167944991 | 122 | 13 | 109 |
28304089310 | 600 | 307 | 293 |
23764091017 | 488 | 257 | 231 |
Four files were read from the same tape. All file were between 1.1GB to 1.5 GB large. For the first file (most likely) the tape had to be remounted. The file was read twice the first time from disk and the second time from the disk cache. The figure below shows the effective transfer rates for the transfers and the time difference between reading from tape and disk (first-second). The position of the files on tape is also shown.
All 377 files were read from the tape H0000700, except the first file as this one was split across two tapes. One client was used to read the file sequentially. The checksum and size of the transfered files matched the ones obtained from the xrootd cluster.
The lower figure below shows the elapsed time for each transfer versus the file size. The blue line shows the relation between elapsed time and file size to be:
elap = 1.5s * size[MB] / 45[MB/s]
The 1.5 as well as the 45 approximate values.
The upper figure shows the elapsed time subtracted by the expected transfer time. The expected transfer time was assumed to be the file size divided by 45MB/s.