Some short term tasks for the next generation data pipeline:
- Implement batch submission interface (also needed ASAP for Data Server)
- (Based on Navids batch submission system)
- Write extended schema for next generation pipeline import/export
- Data Catalog
- Poll user community for data catalog requirements, using CS11 meta-data catalog as example
- Investigate XPath support in Oracle
- Develop XML schema for import/export from data catalog
- Implement Data Catalog + web front-end
- Conditions/Resource checker
- Probably just extend Navid's disk space etc checker to also put data into database
- Investigate use of Rules engine (JESS and similar)
- Create simple pipeline mockup using JESS.