Currently we use a single queue for script jobs and job submission.

  • Probably this is not the right approach because once an item is queued it must be executed.
  • Perhaps better to have seperate queue for each farm, plus scripts.

We need to be able to submit to multiple job sites

  • Need to indicate which locations a task can run
  • Presumably once a stream starts running all subsequent jobs in that stream must run at the same site?
    • Not necessarily, maybe only substreams need to run at same site
      • Could be possible that each job could specify which farms to run on

XML changes

<task farm="all|SLAC|SLAC,IN2P3">
<task farmPolicy="Stream|Job">
<job  farm="all|SLAC|SLAC,IN2P3">

Need to add FARM table
FARM
NAME
MAXJOBS
MAXJOBSPERTASK

We need to be able to limit the number of jobs

  • tasks with highest priority should be given preference
    • can task priority be used to order things in the execution queue?
      • Yes, looks like we can use a "PriorityBlockingQueue" in place of our current ArrayBlockingQueue.
  • No labels