This pl/sql mod is to fix handling of foreign datasets. The pipeline was retrieving all runs from the foreign task, not just the one run needed at a time. This was shown to work in test on Dan's demo foreign task pipeline (and subsequently in prod on our reprocessing task).
Here's the diff between new and old. ('--' and everything after is a pl/sql comment)
The new query uses the runname to restrict the selected dataset list.
Dan
glast01:dflath> diff pdbprocedures.sql ~glastdpf/pdb/pdbprocedures.sql 949c949,950
< and T.Task_PK = DS.Task_FK;
—
> and
> T.Task_PK = DS.Task_FK;
2293c2294
< where TaskProcess_PK = TaskProcess_FK_in;
—
> where TaskProcess_pk = TaskProcess_FK_in;
2296,2314c2297,2310
< select T.TaskName, DS.DatasetName, DSI.FilePath, DSI.FileName, R.Run_PK, R.RunName --, DSI.DSInstance_PK
< from DSInstance DSI, Task T, Dataset DS, Run R
< where DSI.Run_FK in
< (select Run_PK from Run where RunName =
< (select RunName
< from Run
< where Run_PK = Run_FK_in
< )
< and Task_FK != Task_FK_in
< )
< and DSI.Dataset_FK in
< (select Dataset_FK
< from TP_DS
< where TaskProcess_FK = TaskProcess_FK_in
< and RW = 'R' – note: with task_fk != task_fk_in cut, above, this is probably redundant
< )
< and DSI.Dataset_FK = DS.Dataset_PK
< and DS.Task_FK = T.Task_PK
< and R.Run_PK = DSI.Run_FK;
—
> – select DS.DATASETNAME, DSI.FILEPATH, DSI.FILENAME, T.BaseFilePath, T.TaskName, R.Run_PK, R.RunName;
>
> select T.TaskName, DS.DATASETNAME, DSI.FILEPATH, DSI.FILENAME, R.Run_PK, R.RunName
> from Dataset DS, DSInstance DSI, Task T, Run R
> where DS.Dataset_PK in
> (select Dataset_FK from TP_DS
> where TaskProcess_FK = TaskProcess_FK_in
> and RW = 'R')
> and DSI.Run_FK = R.Run_PK
> and R.Run_PK = Run_FK_IN
> and DSI.Dataset_FK = DS.Dataset_PK
> – and T.Task_PK = DS.Task_FK
> and T.Task_PK = R.Task_FK
> and T.Task_PK != Task_FK_in;
SVAC pipeline tasks/scripts v3r3p2, implementing the following changes:
----------------------------------------------------------------
Change some oracle-related environment variables, do both cleanup and kludgification of the methods used to set them. Changes were necessary due to changes in SLAC's default Oracle environment. Changes adress JIRA
PIT-18 .
Changes to partially adress JIRA SVAC-73 (don't interpret bad DB queries as valid but empty data).
Final part of the fix for SVAC-60 (geometry issues w/ 10, 12, 14 towers).
Clean up some cruft left over from when configReport used the schema.
Reduce recon chunk size for VdG runs.
configReport-v3r3p2/ConfigTables was modified (in the GINO DB) to change the LSF queue from short to medium.
Code Versions
Engineering Model (sim/recon)v5r0608p6
System Tests for this version
System Tests Result
System tests:
anders
FRED version
Pipeline tag
v1r0p2
GRITS tag (web browsing and task configuration)
glast-ground v0r3p7
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)
online/svac (task defs, scripts):
pipeline tasks:
online: v2r3p0
svac pipeline code and tasks:
code/tasks v3r3p2 **changed**
pipelineDatasets v0r3
ISOC code and tasks:
v0r5p0
Apps that run in pipeline:
eLog: v2r2p7
ConfigTables: v3r1p5
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p20(SVAC tuple)
Approval:
For the record:
2005-10-20
Run 135004697 was deleted from tasks svacTuple-v3r3p2 reconReport-v3r3p2 recon-v3r3p2, and recon recreated. The run initially failed recon ianppropriately due to NFS/LSF issues. On rollback, the first process in the task failed due to the presence of files left over from the first run, but this failure was not detected, and downstream tasks were launched inappropriately.
2005-10-18
Run 141000346 was deleted from all tasks where it existed
(configReport-v3r3p1 digiReport-v3r3p1 digitization-v3r3p1 online-v2r3p1
updateELogDB-v3r3p1) and a new GINO run of online-v2r3p1 created for that data-taking run. This was necessary because of a (reproducible) failure to export the run from the DMZ. The data were copied by hand for the new injection.
2005-10-17
Run 134001212 had spent soem 8h finalizing in digitization-v3r3p1.
Deleted and resubmitted succesfully.
Strange thing about the logfile: The LSF output was at the beginning of the file, not at the end as it usual is.
2005-10-14
- Run 140003114 was stuck in updateELogDB.
- Run 398001916 had been a week in preparing for digiReportUrl in digiReport-v3r3p1
- Run 141000315 was stuck in digiRootFile in digitization-v3r3p1.
Deleted and resubmitted.
2005-10-12
In order to clean up after 2005/10/11's pipeline problems:
Runs 140002363 and 134001171 of configReport-v3r3p1 were deleted from GINO's database and resubmitted.
157 runs were deleted from all SVAC tasks where they were present, and restarted. Runs were:
399002655 398001964 398001987 141000329 141000333 141000335 141000334 141000330 141000332 399002687 399002683 399002684 399002686 399002685
399002681 399002682 399002679 399002680 399002674 399002677 399002678
399002672 399002675 399002676 399002673 399002671 399002668 399002669 399002670 399002666 399002667 399002665 399002661 399002660 399002663
399002662 399002656 399002658 399002657 399002654 399002651 399002653
399002652 399002647 399002650 399002646 399002649 399002648 399002644
399002645 399002643 399002642 399002641 399002638 399002632 141000328
399002639 399002633 399002636 399002637 399002635 134001192 134001206
399002631 134001208 134001200 134001205 134001203 134001194 134001201
134001199 134001195 134001196 134001198 398002030 398002027 398002028
398002029 134001184 134001183 134001188 134001189 134001180 134001182
399002628 399002630 134001181 399002627 399002624 399002626 399002625
398002026 134001177 134001178 398002025 134001175 398002023 398002019
398002022 398002021 398002020 398002014 398002016 398002017 398002015
398002011 398002013 398002012 398002009 398002010 398002008 134001167
398002007 398002006 398002002 398002003 398002005 398002001 398001999 398002000 398001998 398001992 398001995 398001996 398001997 398001993
398001994 398001991 398001989 398001990 398001988 398001981 398001982
398001986 398001984 398001983 398001978 398001980 398001977 399002621
399002623 399002622 398001973 398001975 398001974 398001972 398001976
398001971 398001966 398001968 398001969 398001963 398001970 398001965
398001967 399002619 399002620
LSF was upgraded from version 5 to 6. Some changes to the log file format engendered some mods to the LSFoutput.pm module in the pipeline.