Page History
...
- clean up existing space
- explore replacing disks in existing fileservers with 2 TB ones
- find a new vendor(s)
- change storage model
Clean up
We have 1.05 PB of space in xrootd now. Some 640 TB of that is taken up by L1, and 90% of that space is occupied by recon, svac and cal tuples.
...
Total Space | 1043 TB | 35 servers (wain006 removed) |
xrootd overhead | 21 TB | (disk 'buffer' space) |
Available Space | 1022 TB |
|
...
Current usage:
...
Space used | 966.778 TB | 95% |
Space free | 55.580 TB | 5% |
...
Consumption rate:
...
Level 1 | 783 GB/day | (averaged over period since 4 Aug 2008) |
Therefore, with current holdings and usage, there is sufficient space for 71 days of Level 1 data (running out ~17 Feb 2011)
...
Usage distribution:
...
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="494489c94af4c86b-cc270097-4cdd49f5-9d74b63c-1cffc85b3b20d5a0afea0128"><ac:plain-text-body><![CDATA[ | path | size [TB] | #files | Notes | ]]></ac:plain-text-body></ac:structured-macro> |
|---|---|---|---|---|---|
/glast/Data | 785 | 807914 |
| ||
/glast/Data/Flight | 767 | 792024 |
| ||
/glast/Data/Flight/Level1/ | 713 | 535635 | 670 TB registered in dataCat | ||
/glast/mc | 82 | 11343707 |
| ||
/glast/mc/ServiceChallenge/ | 56 | 6649775 |
| ||
/glast/Scratch | 51 | 358511 | ~50 TB recovery possible | ||
/glast/Data/Flight/Reprocess/ | 50 | 226757 | ~20 TB recovery possible | ||
/glast/level0 | 13 | 2020329 |
| ||
/glast/bt | 4 | 760384 |
| ||
/glast/test | 2 | 2852 |
| ||
/glast/scratch | 1 | 51412 |
| ||
/glast/mvmd5 | 0 | 738 |
| ||
/glast/admin | 0 | 108 |
| ||
/glast/ASP | 0 | 19072 |
|
...
Detail of Reprocess directories:
...
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="3351dbf6772e5c0f-15e967b7-4c0b4a1f-a4d694f2-3d725cf6525927cd68b8c152"><ac:plain-text-body><![CDATA[ | path | size [GB] | #files | Notes | ]]></ac:plain-text-body></ac:structured-macro> |
|---|---|---|---|---|---|
/glast/Data/Flight/Reprocess/P120 | 22734 | 65557 | removal candidate | ||
/glast/Data/Flight/Reprocess/P110 | 14541 | 49165 | removal candidate | ||
/glast/Data/Flight/Reprocess/P106-LEO | 8690 | 1025 |
| ||
/glast/Data/Flight/Reprocess/P90 | 3319 | 2004 | removal candidate | ||
/glast/Data/Flight/Reprocess/CREDataReprocessing | 631 | 22331 |
| ||
/glast/Data/Flight/Reprocess/P115-LEO | 533 | 1440 |
| ||
/glast/Data/Flight/Reprocess/P110-LEO | 524 | 1393 | removal candidate | ||
/glast/Data/Flight/Reprocess/P120-LEO | 503 | 627 | removal candidate | ||
/glast/Data/Flight/Reprocess/P105 | 264 | 33465 |
| ||
/glast/Data/Flight/Reprocess/P107 | 212 | 4523 |
| ||
/glast/Data/Flight/Reprocess/P116 | 109 | 25122 |
| ||
/glast/Data/Flight/Reprocess/Pass7-repro | 22 | 16 | removal candidate | ||
/glast/Data/Flight/Reprocess/P100 | 8 | 20089 | removal candidate |
...
Detail of top-level Monte Carlo directory:
...
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="22f2a3563f865c76-51f3fd50-4d8849c0-99b4b32d-9bd67de53263bc6727b6eed8"><ac:plain-text-body><![CDATA[ | path | size [GB] | #files | Notes | ]]></ac:plain-text-body></ac:structured-macro> |
|---|---|---|---|---|---|
/glast/mc/ServiceChallenge | 57458 | 6649775 |
| ||
/glast/mc/OpsSim | 24691 | 4494890 |
| ||
/glast/mc/DC2 | 1818 | 136847 |
| ||
/glast/mc/OctoberTest | 41 | 29895 |
| ||
/glast/mc/XrootTest | 33 | 18723 | removal candidate | ||
/glast/mc/Test | 14 | 13403 | removal candidate |
Detail of Monte Carlo 'ServiceChallenge' directories
...
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="b671384a-85e6-476a-bee9-74a7bb02764c"><ac:plain-text-body><![CDATA[ | path | size [GB] | #files | Notes | ]]></ac:plain-text-body></ac:structured-macro> |
|---|---|---|---|---|---|
/glast/mc/ServiceChallenge/allHI-Fe-GR-v15r53p3-Lyon | 6040 | 199881 | |||
/glast/mc/ServiceChallenge/backgnd-proton-GR-v15r6p1 | 4318 | 46676 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r0-2 | 3054 | 561645 | |||
/glast/mc/ServiceChallenge/allGamma-GR-v13r9p3-Lyon | 2676 | 90000 | |||
/glast/mc/ServiceChallenge/Interleave55d-GR-v11r17 | 2230 | 47520 | |||
/glast/mc/ServiceChallenge/Interleave55d-GR-v11r12 | 2187 | 39600 | |||
/glast/mc/ServiceChallenge/backgndSC-GR-v11r2 | 2158 | 108000 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r0 | 1555 | 258517 | |||
/glast/mc/ServiceChallenge/allGamma-GR-v15r0 | 1447 | 60020 | |||
/glast/mc/ServiceChallenge/allGamma-GR-v13r9p12 | 1337 | 50001 | |||
/glast/mc/ServiceChallenge/Interleave55d-GR-v11r12-r-P5 | 1267 | 7920 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r40-Limbo2 | 1039 | 50505 | |||
/glast/mc/ServiceChallenge/allGamma18GeV-GR-v13r9p3-Lyon | 990 | 90000 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r40-Limbo | 964 | 46980 | |||
/glast/mc/ServiceChallenge/GRBgrid-GR-v13r9p22 | 959 | 12000 | |||
/glast/mc/ServiceChallenge/GRBgrid-GR-v11r17 | 869 | 12000 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r9p16 | 753 | 130006 | |||
/glast/mc/ServiceChallenge/GRBgrid-GR-v13r9p4-Lyon | 737 | 12000 | |||
/glast/mc/ServiceChallenge/backgnd-electron-GR-v15r6p1 | 582 | 5600 | |||
/glast/mc/ServiceChallenge/backgnd-positron-GR-v15r6p1 | 572 | 5600 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r0-HE | 435 | 48282 | |||
/glast/mc/ServiceChallenge/skyPointed-GR-v11r17 | 346 | 21600 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r0-Day | 345 | 18729 | |||
/glast/mc/ServiceChallenge/backgnd-GR-HEAD1_1171_2_1-Day | 345 | 18720 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v14r11-Day | 344 | 18728 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r5-Day | 343 | 17221 | |||
/glast/mc/ServiceChallenge/backgnd-GR-HEAD1_1171_4_2-Day | 340 | 18720 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r35-Day | 339 | 18720 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r33-Day | 339 | 18720 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r3-Day | 339 | 18720 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r29-Day | 339 | 18720 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r21-Day | 339 | 18720 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r13-Day | 339 | 18720 | |||
/glast/mc/ServiceChallenge/backgnd-GR-HEAD1_1171_4_4-Day | 339 | 18720 | |||
/glast/mc/ServiceChallenge/backgnd-GR-HEAD-1085-4-2-Day | 339 | 18720 | |||
/glast/mc/ServiceChallenge/backgnd-GR-HEAD1-1148-Day | 338 | 18677 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r51p1-Lyon | 337 | 17292 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v14r8-Day | 323 | 18720 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r5p8-Day | 323 | 15840 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r5p6-Day | 323 | 15840 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v14r2-Day | 316 | 18723 | |||
/glast/mc/ServiceChallenge/opssim2-P6repro-GR-v13r9p19 | 313 | 3446 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v14r7-Day | 310 | 18723 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v14r6-Day | 310 | 18720 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v14r4-Day | 310 | 18720 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r9p12-Day | 310 | 17285 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r11p1-Day | 308 | 18720 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r7-Day | 307 | 15840 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r9-Day | 306 | 15840 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r8-Day | 306 | 15840 | |||
/glast/mc/ServiceChallenge/backgnd-GR-HEAD1_1165 | 302 | 15842 | |||
/glast/mc/ServiceChallenge/GRBgrid-GR-v13r9p20 | 300 | 3606 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r1p1-Day | 295 | 14400 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r11p1-Day-22b | 286 | 15840 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r11p1-Day-22 | 286 | 15840 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r11p1-Day-21 | 286 | 15840 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r39p1-HE-350 | 284 | 780029 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r5p8-Day-Lyon | 283 | 12947 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v12r4-Day | 276 | 14400 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v11r17-Earth10 | 270 | 727248 | |||
/glast/mc/ServiceChallenge/bkg-GR-v13r9p3-Day-OBF2 | 268 | 12960 | |||
/glast/mc/ServiceChallenge/bkg-GR-v13r9p3-Day-OBF1 | 268 | 12960 | |||
/glast/mc/ServiceChallenge/backgnd-neutron-GR-v15r6p1 | 265 | 2800 | |||
/glast/mc/ServiceChallenge/allGamma-GR-v15r29-Lyon | 245 | 5263 | |||
/glast/mc/ServiceChallenge/backgndSC-GR-v11r5-Day-Bertini | 238 | 7200 | |||
/glast/mc/ServiceChallenge/backgndSC-GR-v11-Day-Bertini | 238 | 7200 | |||
/glast/mc/ServiceChallenge/backgndSC-GR-v11r11-Day | 233 | 7200 | |||
/glast/mc/ServiceChallenge/backgndSC-GR-v11-Day-oldGeom | 232 | 7200 | |||
/glast/mc/ServiceChallenge/allGamma-GR-v15r0-LowE | 221 | 60003 | |||
/glast/mc/ServiceChallenge/backgnd-alpha-GR-v15r6p1 | 213 | 1869 | |||
/glast/mc/ServiceChallenge/allGamma-GR-v13r9p12-LowE | 196 | 50001 | |||
/glast/mc/ServiceChallenge/opssim2-P6repro-GR-v15r6p1 | 177 | 1733 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r39p1-HE-100 | 164 | 120010 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v13r5p5-Day | 152 | 15840 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v11r17-Day-CRtest | 144 | 6302 | |||
/glast/mc/ServiceChallenge/allGamma-GR-v15r7-LPM | 142 | 6000 | |||
/glast/mc/ServiceChallenge/backgnd-GR-v15r38-Nadir | 133 | 37171 | |||
/glast/mc/ServiceChallenge/backgnd-gamma-GR-v15r6p1 | 128 | 1869 | |||
/glast/mc/ServiceChallenge/allGamma-GR-v11r14 | 121 | 1000 | |||
/glast/mc/ServiceChallenge/AG-GR-v17r35p1-IRFS72-allE-L | 117 | 8011 | |||
/glast/mc/ServiceChallenge/allGamma-GR-v15r50p9-OVERLAY | 113 | 4800 | |||
/glast/mc/ServiceChallenge/allGamma-GR-v15r50p9-LOWNOISE | 110 | 4800 |
is quite lengthy and appears in full here.
...
Breakdown of space usage by Level 1 data products (from dataCatalog):
...
Filetype: | TotSize/#Runs = | Avg file size | (%) |
|---|---|---|---|
RECON: | 399257.600/13532 = | 29.505 GiB | (58.2%) |
CAL: | 100147.200/13536 = | 7.399 GiB | (14.6%) |
SVAC: | 71270.400/13535 = | 5.266 GiB | (10.4%) |
DIGI: | 61542.400/13537 = | 4.546 GiB | ( 9.0%) |
FASTMONTUPLE: | 28262.400/13537 = | 2.088 GiB | ( 4.1%) |
MERIT: | 22528.000/13537 = | 1.664 GiB | ( 3.3%) |
GCR: | 676.800/13537 = | 51.196 MiB | ( 0.1%) |
FASTMONTREND: | 432.800/13537 = | 32.739 MiB | ( 0.1%) |
DIGITREND: | 337.100/13537 = | 25.500 MiB | ( 0.0%) |
FILTEREDMERIT: | 324.900/ 7306 = | 45.538 MiB | ( 0.0%) |
MAGIC7HP: | 244.700/13315 = | 18.819 MiB | ( 0.0%) |
LS1: | 183.700/13537 = | 13.896 MiB | ( 0.0%) |
CALHIST: | 181.400/13536 = | 13.723 MiB | ( 0.0%) |
RECONTREND: | 172.700/13537 = | 13.064 MiB | ( 0.0%) |
TKRANALYSIS: | 172.400/13536 = | 13.042 MiB | ( 0.0%) |
LS3: | 156.800/13537 = | 11.861 MiB | ( 0.0%) |
RECONHIST: | 124.400/13536 = | 9.411 MiB | ( 0.0%) |
MAGIC7L1: | 55.000/13315 = | 4.230 MiB | ( 0.0%) |
FASTMONHIST: | 53.400/13535 = | 4.040 MiB | ( 0.0%) |
LS1BADGTI: | 48.200/ 7306 = | 6.756 MiB | ( 0.0%) |
CALTREND: | 45.500/13537 = | 3.442 MiB | ( 0.0%) |
FT1: | 39.100/13537 = | 2.958 MiB | ( 0.0%) |
DIGIHIST: | 32.600/13537 = | 2.466 MiB | ( 0.0%) |
etc... |
|
|
|
Replacing Existing Disks
New Vendors
Two obvious candidates are in use at the Lab now: DDN by LCLS, and Dell el cheapo by ATLAS.
...
These are $160/TB, but low density.
Change Storage Model
The idea is to use HPSS as a storage layer to transparently retrieve xrootd files on demand. Wilko thinks the system can push 30-50 TB/day from tape. This is comparable to the rate needed for a merit reprocessing and so is thought not to be a major impediment. In this model, we would remove the big files from disk after making 2 tape copies. They would be retrieved back into a disk buffer when needed. So we would have a relatively fixed disk buffer, and a growing tape presence.