By default you get a fairly small AFS quota on your home directory. You can use the AFS Quota Self-service Form to increase your total AFS quota up to 10GB. If you run into questions, please contact the unix-admin
Dedicated ATLAS NFS spaces are available. NFS are good for log file, batch job input and output, and scratch. These NFS spaces are not backed up. So code are recommended to remain on AFS space.
These spaces are not automatically cleaned up.
Xrootd spaces
There are two high performance xrootd spaces - The Tier 2 space and the Tier 3 space. These space are primarily for storing ROOT files and ATLAS datasets. They are not good for large number of small files such as logs, codes, and small text files.
The Tier 2 space is for Tier 2 only so users should not write to it. However, SLAC ATLAS users are encouraged to use R2D2 to transfer official ATLAS datasets to the WT2 (the SLAC Tier 2) spaces (space token: USERDISK, GROUPDISK, LOCALGROUPDISK, SCRATCHDISK). Once they are at the Tier 2 storage, they are accessible (readonly of course) from all SLAC interactive nodes and batch nodes.
SLAC ATLAS group owns a proof cluster. It is actually a proof cluster, a batch cluster and a xrootd storage cluster.
To read input data in batch jobs, we recommend either directly read the ROOT files using the xrootd protocol, or by copying files (ROOT file and non-ROOT file) via "xrdcp" to batch nodes' /scratch space.
For output files, write the output to batch node's /scratch space first. At the end of your batch job, use "cp" to copy them back to the NFS space or "xrdcp" to copy them to the Tier 3 xrootd space.
To read an xrootd files in Athena using the xrootd protocol:
filelist = [] filelist += ["root://atl-xrdr//atlas/xrootd/atlasuserdisk/rucio/data15_13TeV/07/53/DAOD_MUON0.06783725._000004.pool.root.1"] filelist += ["root://atl-xrdr//atlas/xrootd/atlasuserdisk/rucio/data15_13TeV/4f/ca/DAOD_MUON0.06783725._000001.pool.root.1"] ... athenaCommonFlags.PoolESDInput=filelist |
The Tier 2 space supports FAX global logical file name (gLFN), so the above two files can be simplified as:
filelist = [] filelist += ["root://atl-xrdr//atlas/rucio/data15_13TeV:DAOD_MUON0.06783725._000004.pool.root.1"] filelist += ["root://atl-xrdr//atlas/rucio/data15_13TeV:DAOD_MUON0.06783725._000001.pool.root.1"] ... athenaCommonFlags.PoolESDInput=filelist |
To read an xrootd file in ROOT:
root[0] TXNetFile f("root://atl-xrdr//atlas/xrootd/usr/g/gowdy/myFile.root"); |
To copy files in and out of xrootd:
xrdcp myFile.root root://atl-xrdr//atlas/xrootd/usr/g/gowdy/ xrdcp root://atlprf01:11094//atlas/local/myFile.root . |
Filesystem type access only available on atlint0[1-4].
$ ssh atlint01.slac.stanford.edu $ df -h /atlas /xrootd/atlas Filesystem Size Used Avail Use% Mounted on xrootdfs 741T 682T 60T 93% /atlas xrootdfs 2.8P 1.9P 893T 69% /xrootd/atlas $ cat /atlas/dq2/site-size $ rucio download ... |
atlint03 has a relatively old version of xrootd. this version of xrootd has slow memory leak issues when used via the filesystem type access. So avoid using atlint03 for large number of filesystem type access.