Some short term tasks for the next generation data pipeline:

  • Implement batch submission interface (also needed ASAP for Data Server)
    • (Based on Navids batch submission system)
  • Write extended schema for next generation pipeline import/export
  • Data Catalog
    • Poll user community for data catalog requirements, using CS11 meta-data catalog as example
    • Investigate XPath support in Oracle
    • Develop XML schema for import/export from data catalog
    • Implement Data Catalog + web front-end
  • Conditions/Resource checker
    • Probably just extend Navid's disk space etc checker to also put data into database
  • Investigate use of Rules engine (JESS and similar)
    • Create simple pipeline mockup using JESS.
  • No labels