Page History

...

NFS filesystems

Dedicated ATLAS NFS space is spaces are available. NFS are good for log file, batch job input and output, and scratch. These NFS spaces are not back backed up. So code are recommended to remain on AFS space.

...

Xrootd spaces

There are two high performance xrootd spaces - The Tier 2 space and the Tier 3 space. These space are primarily for storing ROOT files , any type (POOL, ntuples, whatever). xrootd offers high performance access to these types of and ATLAS datasets. They are not good for large number of small files such as logs, codes, and small text files.

Tier 2 space

The Tier 2 space is for Tier 2 only so users should not write to it. However, SLAC ATLAS users are encouraged to use R2D2 to transfer official ATLAS datasets to the WT2 (the SLAC Tier 2) spaces (space token: USERDISK, GROUPDISK, LOCALGROUPDISK, SCRATCHDISK). Once they are at the Tier 2 storage, they are accessible (readonly of course) from all SLAC interactive nodes and batch nodes.

• On atlint0[1-4] these space this space is mounted at /xrootd/atlas/
• In your batch jobs, you can access them via the xrootd protocol (explained below). Files in xrootd have a URL-type format. The accessing URL is root://atl-xrdr//atlas/xrootd/... (change: from /xrootd/atlas/XXX above to /atlas/xroot/XXX)

Tier 3 space

SLAC ATLAS groups group owns a proof cluster. It is actually a proof cluster, a batch cluster and a xrootd storage cluster. The  

•  On atlint0[1-4] the xrootd storage cluster (the Tier 3 space) is

...

• mounted at /atlas

...

• .
• You can use R2D2 to transfer official ATLAS datasets to the space under directory /atlas/dq2. The Tier 3's name in R2D2 is SLAC-ATLAS-T3_GRIDFTP.
• Files under /atlas/proof and /atlas/output are used by the proof jobs.
• Filles Space under /atlas/local are for users to read and write.
• In your batch jobs, you can access them via the xrootd protocol (explained below). The accessing URL is root://atlprf01:11094//atlas/...

...

To read input data in batch jobs, we recommend either directly read the ROOT files using the xrootd protocol, or by copying files (ROOT file and non-ROOT file) via "xrdcp" to batch nodes' /scratch space.

...

filelist = []
filelist += ["root://atl-xrdr//atlas/xrootd/atlasuserdisk/data09rucio/data15_cos13TeV/DPD_CALOCOMM/r733_p37/data09_cos.00121416.physics_L1Calo.merge.DPD_CALOCOMM.r733_p37_tid073973/DPD_CALOCOMM.073973._00004607/53/DAOD_MUON0.06783725._000004.pool.root.1"]
filelist += ["root://atl-xrdr//atlas/xrootd/atlasuserdisk/rucio/data15_13TeV/4f/ca/DAOD_MUON0.06783725._000001.pool.root.1"]
...
athenaCommonFlags.PoolESDInput=filelist

The Tier 2 space supports FAX global logical file name (gLFN), so the above two files can be simplified as:

filelist = []
filelist += ["root://atl-xrdr//atlas/xrootdrucio/atlasuserdisk/data09_cos/DPD_CALOCOMM/r733_p37/data09_cos.00121416.physics_L1Calo.merge.DPD_CALOCOMM.r733_p37_tid073973/DPD_CALOCOMM.073973._000016data15_13TeV:DAOD_MUON0.06783725._000004.pool.root.1"]
filelist += ["root://atl-xrdr//atlas/rucio/data15_13TeV:DAOD_MUON0.06783725._000001.pool.root.1"]
...
athenaCommonFlags.PoolESDInput=filelist

To read an xrootd file in ROOT:

root[0] TXNetFile f("root://atl-xrdr//atlas/xrootd/usr/g/gowdy/myFile.root");

...

xrdcp myFile.root root://atl-xrdr//atlas/xrootd/usr/g/gowdy/

xrdcp root://atlprf01:11094//atlas/local/myFile.root .

...

$ ssh atlint01.slac.stanford.edu
$ df -h /atlas /xrootd/atlas
Filesystem      Size  Used Avail Use% Mounted on
xrootdfs        741T  682T   60T  93% /atlas
xrootdfs        2.8P  1.9P  893T  69% /xrootd/atlas
$ cat /atlas/dq2/site-size

$ rucio download ...

atlint03 has a relatively old version of xrootd. this version of xrootd has slow memory leak issues when used via the filesystem type access. So avoid using atlint03 for large number of filesystem type access.