...
There are two high performance xrootd spaces - The Tier 2 space and the Tier 3 space. These space are primarily for storing ROOT files and ATLAS datasets. They are not good for large number of small files such as logs, codes, and small text files.
On ATLAS interactive nodes atlint0[1-4] (and only on them), one can see these spaces as mounted filesystems:
No Format |
---|
$ ssh atlint01.slac.stanford.edu
$ df -h /atlas /xrootd/atlas
Filesystem Size Used Avail Use% Mounted on
xrootdfs 741T 682T 60T 93% /atlas
xrootdfs 2.8P 1.9P 893T 69% /xrootd/atlas
$ ls -l /atlas
total 15
drwxrwxrwx 1 daemon daemon 140 Oct 7 11:43 dq2/
drwxrwxrwx 1 daemon daemon 6 Oct 25 2012 group/
drwxrwxrwx 1 daemon daemon 4096 Oct 7 12:27 local/
drwxrwxrwx 1 daemon daemon 4096 Dec 23 2014 output/
drwxrwxrwx 1 daemon daemon 4096 Aug 4 11:28 proof/
drwxrwxrwx 1 daemon daemon 37 Jan 23 2012 test/
$ cat /atlas/dq2/site-size
{"sizes": {"total": 813980054454272, "available": 65063350558720}, "time" :"2015-11-18T19:30:01"}
$ cd /atlas/local
$ rucio download ... |
/xrootd/atlas is the Tier 2 space. /atlas is the Tier 3 space. Avoid using atlint03 for large number of filesystem access.
Tier 2 space
The Tier 2 space is for Tier 2 only so users should not write to it. However, SLAC ATLAS users are encouraged to use R2D2 to transfer official ATLAS datasets to the WT2 (the SLAC Tier 2) spaces (space token: USERDISK, GROUPDISK, LOCALGROUPDISK, SCRATCHDISK). Once they are at the Tier 2 storage, they are accessible (readonly of course) from all SLAC interactive nodes and batch nodes.
...
SLAC ATLAS group owns a proof cluster. It is actually a proof cluster, a batch cluster and a xrootd storage cluster.
- On atlint0[1-4] the xrootd storage cluster (the Tier 3 space) is mounted at /atlas .
- You can use R2D2 to transfer official ATLAS datasets to the space under directory /atlas/dq2. The Tier 3's name in R2D2 is SLAC-ATLAS-T3_GRIDFTP.
- Files under /atlas/proof and /atlas/output are used by the proof jobs.
- Space under /atlas/local are for users to read and write.
- In your batch jobs, you can access them via the xrootd protocol (explained below). The accessing URL is root://atlprf01:11094//atlas/...
...
Code Block |
---|
xrdcp myFile.root root://atlprf01:11094//atlas/local/myFile.root
xrdcp root://atlprf01:11094//atlas/local/myFile.root myFile.root
|
Filesystem type access. This is only available on atlint0[1-4].
No Format |
---|
$ ssh atlint01.slac.stanford.edu
$ df -h /atlas /xrootd/atlas
Filesystem Size Used Avail Use% Mounted on
xrootdfs 741T 682T 60T 93% /atlas
xrootdfs 2.8P 1.9P 893T 69% /xrootd/atlas
$ cat /atlas/dq2/site-size
$ rucio download ... |
...