...
In order to run on batch, we will use this modulefile system and submit jobs by creating a text file (eg myscript.sh) like
| Code Block |
|---|
#!/bin/bash -l #BSUB -P jupyter #BSUB -J my_batch_job_name #BSUB -q slacgpu #BSUB -n 1 #BSUB -R "span[hosts=1]" #BSUB -W 72:00 #BSUB -B # setup env source /etc/profile.d/modules.sh export MODULEPATH=/usr/share/Modules/modulefiles:/opt/modulefiles:/afs/slac/package/singularity/modulefiles module purge module load PrgEnv-gcc module slac-ml/20190606.1 # run the notebook, executing all cells cd MY_DATA_DIRECTORY jupyter nbconvert --to notebook --inplace --execute mynotebook.ipynb |
You can then submit the job to batch via
| Code Block |
|---|
bsub < myscript.sh |
and you can monitor the job with
| Code Block |
|---|
bjob -l {jobid} |
We recommend either using nbconvert or papermill to provide parameter access to notebooks.