News from May 27, 2008

  2008/05/27
CCB Request to release new Pipeline II, Data Catalog
Last changed: May 28, 2008 13:12 by Tony Johnson
Labels: sassoccb

Reason for change

This new release incorporates changes required to support multiple "versions" of files as required by Level 1 processing. Currently L1 uses an arbitrary naming scheme which is not understood by the data catalog or tools like the data skimmer. This results in multiple copies of the same event being skimmed. This release fixes this problem, and also simplified the book keeping required by L1Proc.

The database will be converted to the new structure. Based on the time taken to convert the DEV database, and the number of datasets in PROD, we estimate that it will take about 8 hours to do the conversion on PROD. Database views have been created which will mean tools which care only about the most recent version of a file, and which do not update the database will not see any change/

Test Procedure

These changes have been tested on a DEV version of the server. We have tested ASP, MC and L1 tasks using the new release.

Rollback procedure

This update requires changes to the data catalog database. They can be rolled back if a problem is found immediately, but rolling back after a substantial number of datasets have been registered in the new schema will be difficult.

Related JIRA

SSC-51@JIRA

Details

DataCatalog 2.0

Pipeline 1.2

This release also incorporates the performance improvements made in release 1.1.1

and minor changes to the job control daemon documented separately.

Data Crawler 1.3

Posted at 27 May @ 4:04 PM by Tony Johnson | 0 Comments

May 2008
Sun Mon Tue Wed Thu Fri Sat
        1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31
             

May 30, 2008
May 17, 2008