Useful links
- SDF guide and documentation, particularly on using Jupyter notebooks interactively or through web interface.
- Rachel H's documentation on training GNNs with SALT on SDF.
- SDF web browser interface, where you can launch Jupyter jobs on SLURM
- Fork of training-dataset-dumper git repository
Presentations and useful meetings
- See all B-jet calibration meetings on Indico
- Framework experience (Prajita, July 6)
- Plans (Prajita, July 13)
SDF preliminaries
Starting Jupyter sessions via SDF web interface
- SDF web interface > My Interactive Sessions > Services > Jupyter (starts a server via SLURM)
- Choose Custom Singularity Containers > enter the following custom commands:
export SINGULARITY_IMAGE_PATH=/sdf/group/atlas/g/XbbXccTrainingData/singularity/vertexing.sif
function jupyter() { singularity exec --nv -B /sdf,/gpfs,/scratch,/lscratch ${SINGULARITY_IMAGE_PATH} jupyter $@; }
Producing H5 samples
We are using a custom fork of dataset-dumper, developed for producing h5 files for NN training based on FTAG derivations. The custom fork is modified to store the truth jet pT via AntiKt4TruthDressedWZJets container.
- Add documentation about training dataset dumper fork
The current set of available Ntuples is available on:
/gpfs/slac/atlas/fs1/d/pbhattar/BjetRegression/Input_Ftag_Ntuples ├── Rel22_ttbar_AllHadronic ├── Rel22_ttbar_DiLep └── Rel22_ttbar_SingleLep
Analyzing H5 samples
Notebooks
- Chunking h5 files: /sdf/home/b/bbullard/bjes/analysis/ChunkH5.ipynb
Miscellaneous tips
You can grant read/write access for SDF directories to ATLAS group members via
chgrp atlas-a <your_directory> chmod g+r <your_directory>