Data analysis during shifts
During a shift, the DAQ writes data to the NAS in S20. You must be on facet-srv20 to access the data! The data can be found here:
[fphysics@facet-srv20 /nas/nas-li20-pm00 ]$ pwd /nas/nas-li20-pm00 [fphysics@facet-srv20 /nas/nas-li20-pm00 ]$ ll -hrt total 24K drwxrwxr-x 3 fphysics facet 4.0K Jul 23 16:03 E300 drwxrwxr-x 3 fphysics facet 4.0K Jul 23 16:53 E326 drwxrwxr-x 3 fphysics facet 4.0K Aug 11 16:09 E305 drwxrwxr-x 3 fphysics facet 4.0K Aug 11 17:18 E320 drwxrwxr-x 3 fphysics facet 4.0K Aug 12 16:13 E327 drwxrwxr-x 4 fphysics facet 4.0K Aug 13 13:36 TEST
Data retrieval after shifts
Everyday at 10 AM, the data is transferred from the NAS to SLAC Central Computing where it is redundantly stored (no data loss). Both FACET and FACET-II data is located here:
sgess@rhel6-64c nas-li20-pm00 $ pwd /nfs/slac/g/facetdata/nas/nas-li20-pm00 sgess@rhel6-64c nas-li20-pm00 $ ll -hrt total 11M drwxrwxr-x 4 fdata ar 512 Dec 7 2015 THz drwxrwxr-x 3 fdata ar 512 Dec 7 2015 T506 drwxrwxr-x 3 fdata ar 512 Dec 7 2015 E214 drwxrwxr-x 4 fdata ar 512 Dec 10 2015 T504 drwxrwxr-x 4 fdata ar 512 Jan 12 2016 E210 drwxrwxr-x 5 fdata ar 512 Feb 6 2016 E217 drwxrwxr-x 4 fdata ar 512 Feb 13 2016 E212 drwxrwxr-x 4 fdata ar 512 Feb 13 2016 E209 drwxrwxr-x 5 fdata ar 512 Feb 14 2016 E201 drwxrwxr-x 3 fdata ar 512 Feb 14 2016 E204 drwxrwxr-x 5 fdata ar 512 Feb 14 2016 E224 drwxrwxr-x 5 fdata ar 512 Feb 14 2016 E200 drwxrwxr-x 3 fdata ar 512 Feb 18 2016 E215 drwxrwxr-x 4 fdata ar 512 Mar 24 2016 E225 drwxrwxrwx 3 fdata ar 512 Jul 23 16:03 E300 drwxrwxrwx 3 fdata ar 512 Aug 11 16:09 E305 drwxrwxr-x 5 fdata ar 512 Aug 13 13:36 TEST drwxrwxrw- 2 fdata ar 512 Aug 14 15:04 E320 drwxrwxrw- 2 fdata ar 512 Aug 14 15:04 E326 drwxrwxrwx 3 fdata ar 512 Aug 15 12:13 E327
Transferring data (to local machine)
The best way to transfer data is to use the rsync command which is better than scp if the transfer gets interrupted.
To get the data, from your local machine do (for example):
rsync -av USERNAME@centos7.slac.stanford.edu:/nfs/slac/g/facetdata/nas/nas-li20-pm00/E301 .
Recently (2024) SLAC implemented stricter network policies, and centos7 is accessible only from slac internal network. If you want to transfer data to your computer outside slac network, you need to use a jump host (or proxy):
rsync -av -e 'ssh -J USERNAME@rocky9.slac.stanford.edu' USERNAME@centos7.slac.stanford.edu:/nfs/slac/g/facetdata/nas/nas-li20-pm00/E301 .
The above command will download THE ENTIRETY OF THE E301 FOLDER! You will want to go further to figure out which dataset you actually need/want. To do so:
ssh USERNAME@centos7.slac.stanford.edu ls /nfs/slac/g/facetdata/nas/nas-li20-pm00
The above will show all subfolders in nas-li20-pm00/ (such as E301).
To look inside E301:
ls /nfs/slac/g/facetdata/nas/nas-li20-pm00/E301
And so on. If I want the data from the 08091 run on 5/14/2024 of E301, I would do the following, which will download all of E301_08091/ into the directory the commandline is currently in.
rsync -av USERNAME@centos7.slac.stanford.edu:/nfs/slac/g/facetdata/nas/nas-li20-pm00/E301/2024/20240514/E301_08091 .