Impact-T + Bmad Simulation Code:

Uses the Impact_Bmad code from Eric's code

Notes:

• if your Jupyter sessions doesn't start make sure there is enough room in your home directory. The soft limit is hit at 25 M 1000 byte blocks, so you'll want to keep your 'Used' column under 25000000. Run the below command to check after you are in your home dir.

 df --block-size=1000 . 

• you can also edit the custom apptainer script to copy the notebook files to another path such as /sdf/group/facet/<your-name> (if you have permissions to)
• The code runs in parallel based on how many threads are allocated when the jupyter session is made. If you need more threads than what OnDemand provides you can launch the container with the command line instructions.

Via Open OnDemand (S3DF):

(no port forwarding needed)

1. login to open ondemand at https://s3df.slac.stanford.edu/ondemand
2. click on the top menu bar and choose interactive apps → Jupyter
3. 
4. Select facet2 for Jupyter Image
5. Select impact_bmad_sim for  Image
6. Click use JupyterLab instead of Jupyter Notebook
7. For run on cluster type select: Batch; s3df; account: FACET; Partition Milano
8. Select number of hours and cores 
9. Configure rest of settings - *to use the FACET account you need to be added via coact; you can try using the beamphysics account if FACET doesnt work.
10. 
11. Launch and connect
12. The container will copy the necessary notebooks to $HOME/impact_bmad_container_notebooks . Click on this directory via the Jupyter UI and run the Impact / Bmad Sim. 

If you want to customize the path where the notebook files are copied to from the container then:

1. instead of step 4 above select custom
2. select apptainer image
3. paste in the code below and change NOTEBOOK_ROOT

4. (also available here)

   # Set the environment variable for the Apptainer image path
   export APPTAINER_IMAGE_PATH=/sdf/group/facet/sanjeev/containers/impact-bmad_latest.sif
   export NOTEBOOK_ROOT=$HOME/impact_bmad_container_notebooks
   mkdir -p $HOME/impact_bmad_container_notebooks
   
   # Define the jupyter function to use Apptainer for executing Jupyter with necessary bindings and running mkdir and cp commands
   function jupyter() {
   apptainer exec -B /usr,/sdf,/fs,/sdf/scratch,/lscratch ${APPTAINER_IMAGE_PATH} bash -c "
           mkdir -p ${NOTEBOOK_ROOT} &&
           cp -rn /opt/notebooks/* ${NOTEBOOK_ROOT}/";
     	apptainer exec -B /usr,/sdf,/fs,/sdf/scratch,/lscratch ${APPTAINER_IMAGE_PATH}  jupyter $@;
   }

5. continue from step 6

Via Command Line (S3DF)

• Connect to s3df with port forwarding (juptyer notebooks will run on port 5555 and be forwarded to your local machine)

 ssh -L 5555:localhost:5555 <SLAC UNIX USERNAME>@s3dflogin.slac.stanford.edu

• connect to interactive compute cluster

 ssh -L 5555:localhost:5555 iana

• Run this script which copies jupyter notebook files out of the container and starts a session 
• tbd

Via Command Line (not S3DF)

??