The beginings of a pipeline II project has been created.

Getting the code

The package currently lives in the Glast-Java CVS. You can access it from:

:ext:<userid>@glast-java.slac.stanford.edu:/cvs/java

in module org-glast-pipeline-server. This can also be viewed using ViewCVS.

Building and Running the Project

You can build the project from the command line using maven 1.0.2, as follows:

cd org-glast-pipeline-server
maven
java -jar release/org-glast-pipeline-server-0.1.jar

or to run with JMX enabled:

java -Dcom.sun.management.jmxremote -jar release/org-glast-pipeline-server-0.1.jar

Using Netbeans IDE to open the project

You can open maven projects directly in NetBeans once you have installed the mevenide plugin. There are detailed instructions on installing mevenide here.

Pipeline II cruise-control project

Cruise control has been set up to build this project. The results can be viewed here.

Pipeline II Jira project

There is now a pipeline II Jira project. A summary of open issues is whown below.

Key Summary Assignee Status
Refresh

Where Next

Goal should be to create a standalone application which can read an XML configuration file (like this one) and then run the various stages of the pipeline. To begin with I think we should not worry about web interface or batch interface (just have a dummy "job" which starts, waits a few seconds and the finishes).

So we need:

  • Finish off more of the core database design
  • Add a thin SQL layer for talking to the database. Initially this can have hardwired SQL statements, later it may switch to using stored procedures. Should base this on Dan's existing stored procedures, and this java interace.
  • Implement the XML importer. This can be ported from the existing pipeline web interface.
  • Implementation of "environment variables" and simple script runner.

I have added some initial thoughts on Running tasks in Pipeline II.

Notes

MBeans

We might want to look at using at MBeans for monitoring/simple control of the application. Here are some possibly useful links:

Executing scripts

It turns out that Java 1.5 has got built-in support for creating and running thread pools for executing tasks from a queue. See:

  • No labels