This page documents some SLAC specific examples of using the GRID for dataset production and retrieval.
The centrally maintained ATLAS page for starting on the GRID is the place to start
DQ2 is the ATLAS data management system. There is significant documentation on it's general prinicples, usage, and troubleshooting here
In order to being using the DQ2 tools at SLAC one simply needs to source the following script
source /afs/slac.stanford.edu/g/atlas/etc/hepix/GridSetup.sh |
This script should be automatically run for you if you are using the standard ATLAS setup, described here:SLAC ATLAS Computing Environment
You then need to get a GRID ticket, which will last 12 hours or so:
voms-proxy-init -voms atlas |
To add your files to a personal dataset (which can also be used by others):
dq2-put --long-surls -s /xrootd/atlas/usr/f/fizisist/test user.DavidWilkinsMiller.misal1_csc11.005009.J0_pythia_jetjet.pile1sf01.AOD.v13003003.verylow.TEST9 |
The dataset name you use should conform to "user09.DN.name.datatype.version"
where DN is your identifier extracted from your certificate, and can be computed from:
python /afs/cern.ch/atlas/offline/external/GRID/ddm/Operations/utils/dn.py |
To list the files in a dataset (note, you can use wildcards...):
dq2-ls -f user.DavidWilkinsMiller.misal1_csc11.005009.J0_pythia_jetjet.pile1sf01.AOD.*.verylow.TEST9 |
To get the dataset:
cd /tmp dq2-get user.DavidWilkinsMiller.misal1_csc11.005009.J0_pythia_jetjet.pile1sf01.AOD.v13003003.verylow.TEST9 ls -l user.DavidWilkinsMiller.misal1_csc11.005009.J0_pythia_jetjet.pile1sf01.AOD.v13003003.verylow.TEST9/ |
A commonly used set of tools for distributed analysis is PANDA
In order to being using these tools at SLAC, one simply needs to source 1 script and set 1 environment variable
source /afs/slac/g/atlas/packages/panda-client/etc/panda/panda_setup.sh export PATHENA_GRID_SETUP_SH="/afs/slac/package/vdt/wlcg-client/setup.sh" |