Versions Compared

Old Version 44

changes.mady.by.user Wilko Kroeger

Saved on

compared with

New Version 45

changes.mady.by.user Wilko Kroeger

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • a partition and slurm account should be specified when submitting jobs. The slurm account would be lcls:<experiment-name> e.g. lcls:xpp123456. The account  is used for keeping track of resource usage per experiment

    Code Block
    % sbatch -p milano --account lcls:xpp1234 ........
  • In the S3DF by default memory is limited to 4GB/core. Usually that is not an issue as processing jobs use many core (e.g. a job with 64 cores would request 256GB memory)
  • the memory limit will be enforced and your job will fail with an OUT_OF_MEMORY status
  • memory can be increased using the--memsbatch option (e.g.: --mem 16G, default unit is megabytes)
  • Default total run time is 1 day, the --time option allows to increase/decrease it.

  • Number of cores

    Warning

    Some cores of a milano batch node are exclusively used for file IO (WekaFS). Therefore although a milano node has 128 core only 125 120 can be used (might change in the future).
    submitting a task with  --nodes 1 --ntasks-per-node=128 would fail with:  Requested node configuration is not available

  • Environment Varibales: sbatch option can also be set via environment variables which is useful if a program is executed that calls sbatch and doesn't allow to set options on the command line e.g.:

    Code Block
    languagebash
    % SLURM_ACCOUNT=lcls:experiment  executable-to-run [args]
or
% export SLURM_ACCOUNT=lcls:experiment
% executable-to-run [args]

    The environment variables are SBATCH_MEM_PER_NODE (--mem), SLURM_ACCOUNT(--account) and SBATCH_TIMELIMIT (--time). The order arguments are selected is: command line, environment and withing sbatch script. 

...