Page History
...
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="7df9a0dd955c80a8-f8110b71-4a6f4f37-87158dff-3b450c55769f1ed758e77cc0"><ac:plain-text-body><![CDATA[ | path | size [TB] | #files | Notes | ]]></ac:plain-text-body></ac:structured-macro> |
|---|---|---|---|---|---|
/glast/Data | 785 | 807914 |
| ||
/glast/Data/Flight | 767 | 792024 |
| ||
/glast/Data/Flight/Level1/ | 713 | 535635 | 670 TB registered in dataCat | ||
/glast/mc | 82 | 11343707 |
| ||
/glast/mc/ServiceChallenge/ | 56 | 6649775 |
| ||
/glast/Scratch | 51 | 358511 | ~50 TB recovery possible | ||
/glast/Data/Flight/Reprocess/ | 50 | 226757 | ~20 TB recovery possible | ||
/glast/level0 | 13 | 2020329 |
| ||
/glast/bt | 4 | 760384 |
| ||
/glast/test | 2 | 2852 |
| ||
/glast/scratch | 1 | 51412 |
| ||
/glast/mvmd5 | 0 | 738 |
| ||
/glast/admin | 0 | 108 |
| ||
/glast/ASP | 0 | 19072 |
|
...
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="1ef810caac032e9d-f73fbf38-40434d77-943e9904-921c30d2a29828acb39c089b"><ac:plain-text-body><![CDATA[ | path | size [GB] | #files | Notes | ]]></ac:plain-text-body></ac:structured-macro> |
|---|---|---|---|---|---|
/glast/Data/Flight/Reprocess/P120 | 22734 | 65557 | removal candidate | ||
/glast/Data/Flight/Reprocess/P110 | 14541 | 49165 | removal candidate | ||
/glast/Data/Flight/Reprocess/P106-LEO | 8690 | 1025 |
| ||
/glast/Data/Flight/Reprocess/P90 | 3319 | 2004 | removal candidate | ||
/glast/Data/Flight/Reprocess/CREDataReprocessing | 631 | 22331 |
| ||
/glast/Data/Flight/Reprocess/P115-LEO | 533 | 1440 |
| ||
/glast/Data/Flight/Reprocess/P110-LEO | 524 | 1393 | removal candidate | ||
/glast/Data/Flight/Reprocess/P120-LEO | 503 | 627 | removal candidate | ||
/glast/Data/Flight/Reprocess/P105 | 264 | 33465 |
| ||
/glast/Data/Flight/Reprocess/P107 | 212 | 4523 |
| ||
/glast/Data/Flight/Reprocess/P116 | 109 | 25122 |
| ||
/glast/Data/Flight/Reprocess/Pass7-repro | 22 | 16 | removal candidate | ||
/glast/Data/Flight/Reprocess/P100 | 8 | 20089 | removal candidate |
...
<ac:structured-macro ac:name="unmigrated-wiki-markup" ac:schema-version="1" ac:macro-id="9fcff2dbac68cbb5-5e303679-48384e57-9fe09fb6-684fa3961e06efd8b6d46479"><ac:plain-text-body><![CDATA[ | path | size [GB] | #files | Notes | ]]></ac:plain-text-body></ac:structured-macro> |
|---|---|---|---|---|---|
/glast/mc/ServiceChallenge | 57458 | 6649775 |
| ||
/glast/mc/OpsSim | 24691 | 4494890 |
| ||
/glast/mc/DC2 | 1818 | 136847 |
| ||
/glast/mc/OctoberTest | 41 | 29895 |
| ||
/glast/mc/XrootTest | 33 | 18723 | removal candidate | ||
/glast/mc/Test | 14 | 13403 | removal candidate |
...
Below are the steps that were outlined above. I put some rough time estimates for the different steps.
- Testing large scale staging. (2 weeks)
- Deploy a new xrootd version (the current version supports only an old depreciated interface to HPSS) (1 week)
- configure the xroot to stage files from HPSS to disk if they are missing (1 day)
- Setup monitoring of the staging (partly comes with xrootd) (2 days)
- Setup (I guess Nagios) to send out warnings if the xrootd disk space will be filled up
- Develop code to do allow purging files using file lists or regEx (1 week)
- Setup new dual copy file family (2 days)
- Re-migrate files to have dual copies (1 day work, 2-3 weeks for migration)
- Tools to sort files by tape and prestage them in order to increase the HPSS throughput.