Versions Compared

Old Version 4

changes.mady.by.user Joris Guillouf

Saved on

compared with

New Version 5

changes.mady.by.user Joris Guillouf

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Focus with ScienceTools is just ease of distribution

  • Would it be useful to distribute the tools in VMs? Containers? Both?

    Info Joris : I found this VM : Virtual Machine version 3


  • Are there external dependencies (like xroot-d) that would cause problems with virtualization if backend changes?

  • We need automated build system for ST: Release manager vs. manual builds 

  • GR uses xrootd ST does not (Eric)
  • Use of virtualization is for convenience - which is most useful thing to do? (Richard)

...

  • Needs someone that he could show the pipeline code and train to do heavy lifting when it comes to kicking the pipeline
  • Docker containers for something like the batch system may cause some problems, since
  • For something like the L1 pipeline, a number of images would need to be launched simultaneously
  •  Would size of the software cause problems with deployment?
  • We would need a system where you restrict loading images to the batch farm to prevent collisions/problems
  • There is probably a precedent for this, however, Matt has no experience deploying on this scale 
  • File size of ~1 GB is best, a few is manageable for production. 
  • IT dept supportive of docker@SLAC. There is 1 machine with RHEL7
  • Lyon is a much larger computing center - likely they will upgrade to Docker first

    • Now full support for Docker at Lyon (Fred)

      Info

      Joris : Lyon wants to use Singularity because they have security issues with UGE + Docker.


Infrastructure:

  • Last purchase went into dev cluster
    • many nodes @RHEL6, upgrade to RHEL7 and doing docker with this
    • Still figuring out NFS/AFS sorted out with RHEL7. GPFS? 
  • It's good to come up with a plan because of security implications if NFS underneath. 
    • Use right docker (UID issues w/security)
  • SLAC will give us a few nodes for testing docker. Fall back way to install on user machines. (Brian)
    • AFS on RHEL6 docker
    • read files if world readable. 
    • NFS is hardest. 
  • Timeline for RHEL7, 12mo? 2018? (Matt)
    • RHEL7 support is dodgy. 
    • Configuration stuff is hard part

...

  • GPL_TOOLS (staging and logging)
  • REPRO common tools
  • REPRO task scripts
  • GlastRelease
  • ScienceTools
  • GLAST_EXT software (e.g., python, root)
  • Ftools (KIPAC installation)
  • ROOT skimmer
  • FITS skimmer (possibly unnecessary?)
  • evtClassDefs
  • calibration and alignment files
  • diffuse models
  • xroot tools
  • xroot /glast/Scratch space
  • /scratch on local batch machines
  • data catalog query (FT2 file and current version of FITS files)
  • mySQL DB (calibration and alignment)
  • Fermi astrotools (could probably eliminate)

3. Questions