Versions Compared

Old Version 4

changes.mady.by.user Johan BREGEON

Saved on

compared with

New Version 5

changes.mady.by.user Unknown User (clavalley)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Wiki Markup
This page compiles the developement approach for the implementation of an interface between the \[Pipeline2 system\|https://confluence.slac.stanford.edu/pages/editpage.action?pageId=33053\] and european \[EGEE (Enabling Grids for E-sciencE)\|http://www.eu-egee.org/\] grid.  Pipeline2 system is now able to launch jobs either in SLAC or CC IN2P3 batch farms.   This project is then aimed to extend this distributed system to EGEE grid ressources. 

According to Pisa Team (Michael Kuss and Franz can write! and Johan too., Johan Bregeon, Francesco Longo) work, the use of EGEE workers is a good solution to produce huge amounts of MonteCarlo simulation data.   On this purpose, they have created EGEE GLAST Virtual Organization which is now operational and through which GLAST software has been installed in some EGEE sites.  An important amount of data has been produced using this system, independently of Pipeline2 interfaces and data management procedures.  

The developpment of the link between Pipeline2 interfaces and procedures will then allow to use EGEE ressources to answer to MonteCarlo simulations Pipeline2 requests. 

Some preliminary ideas have been given in order to organize this work:

  • Keep taskConfig.xml file as parameterization basis, new job type: EGEE (in addition to SLAC and IN2P3)
  • Keep the Structured file System @ CC IN2P3
  • Keep JobControlService Deamon at CC IN2P3
  • use EGEE UI environement @ Lyon  (generic certificate & proxy manual or automatic renewal, if possible) to launch Job.
  • Completely separate job launching and data retrieval

Some implementation steps have been suggested as well:

  • org-glast-pipeline-server: the files attached to the Job instance should be different for EGEE.   Then  this package should be able to distinguish between an EGEE job and a SLAC/IN2P3 farms one, according to  taskConfig.xml file "type" variable value.
  • A new package contained in org-glast-jobcontrol jar file needs to be developped:  EGEEJobControlService (equivalent to LSFJobControlService and BQSJobControlService),  including appropriate submit(), getStatus() and cancel() methods.
  • Job outputs can remain in EGEE Storage Elements, providing they are referenced in catalog databases, and that they can be easily downloaded by users