Pipeline front end changes
PFE v0r3p7
From Matt:
A part of the released version of the Pipeline Front End (v0r3p4) broke after a Microsoft server patch to glast05. The problem surfaced due to the HTTP session being destroyed and re-created on each HTTP request. I'm not sure which patch was responsible, as several are applied each week. A user first noticed this issue, which is documented in JIRA as http://jira.slac.stanford.edu/browse/PFE-49.
The part of PFE that broke was remembering the user's selected pipeline server, being one of "prod", "dev" or "test". Since the user's session was always being destroyed, the default value of "prod" was always being used, making it impossible for users to upload pipeline configurations to the "dev" and "test" servers.
The problem did not appear on my development computer, which runs a different version of Windows and IIS than glast05. My development computer uses Windows XP SP2 with IIS 5, while glast05 uses Windows Server 2003 with IIS 6. I found that a newer version of my ColdFusion-based Pipeline Front End "fixed" the problem on glast05. Since it was important to fix this problem ASAP, and since initial attempts to understand the "session being destroyed" problem were fruitless, I pushed a new version of the Pipeline Front End to glast05, v0r3p7.
I recommend to the CCB that GLAST use version v0r3p7 as the production Pipeline Front End, which is the version that I pushed to glast05 45 minutes ago. If this is not acceptable, then I can restore the previous version (v0r3p4), which would still have the "session destroyed" bug. All version of PFE are tagged in CVS, making it trivial to switch between versions...
This has been verified by Warren.
Code Versions
Engineering Model (sim/recon) v4r060302p23
System Tests for this version
System Tests Result
FRED version
Pipeline tag
v1r0p2
GRITS tag (web browsing and task configuration)
glast-ground v0r3p7 *** changed ***
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)
online/svac (task defs, scripts):
pipeline tasks:
online: v2r1p2
svac pipeline code and tasks:
code/tasks v3r1p17
ISOC code and tasks:
v0r5p0
Apps that run in pipeline:
eLog: v2r2p6
ConfigTables: v3r1p4
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p17 (SVAC tuple)
Approved: 31 May 2005 unanimous
svac changes
SVAC tasks at v3r1p17.
TEM ids were appended to the module serial numbers in the eLog DB in eLog v2r2p6. The script that builds recon jobOption files was not modified to strip the TEM id off again before writing the serial number to to jo file. It has been fixed to do so now. Recon has been tested with this change.
See http://jira.slac.stanford.edu/browse/SVAC-56 .
This is fixed in svacPipeline v3r1p17.
Code Versions
Engineering Model (sim/recon) v4r060302p23
System Tests for this version
System Tests Result
FRED version
Pipeline tag
v1r0p2
GRITS tag (web browsing and task configuration)
glast-ground v0r3p4
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)
online/svac (task defs, scripts):
pipeline tasks:
online: v2r1p2
svac pipeline code and tasks:
code/tasks v3r1p17 **changed**
ISOC code and tasks:
v0r5p0
Apps that run in pipeline:
eLog: v2r2p6
ConfigTables: v3r1p4
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p17 (SVAC tuple)
Approved: 25 May 2005 Emergency: Eduardo & Richard.
svac changes
SVAC tasks at v3r1p16.
recon jobOptions updated to stop CAL modules from asking for TKR calib constants.
eLog at v2r2p6 to address some sort of issue with tower/bay naming: change code to include TEM id in tkr_ser_no and cal_ser_no.
JIRA issue(http://jira.slac.stanford.edu/browse/ELB-69)
ConfigTables at v3r1p4 to address JIRAs http://jira.slac.stanford.edu/browse/SVAC-54 and http://jira.slac.stanford.edu/browse/SVAC-55 .
Code Versions
Engineering Model (sim/recon) v4r060302p23
System Tests for this version
System Tests Result
FRED version
Pipeline tag
v1r0p2
GRITS tag (web browsing and task configuration)
glast-ground v0r3p4
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)
online/svac (task defs, scripts):
pipeline tasks:
online: v2r1p2
svac pipeline code and tasks:
code/tasks v3r1p16 **changed**
ISOC code and tasks:
v0r5p0
Apps that run in pipeline:
eLog: v2r2p6 **changed**
ConfigTables: v3r1p4 **changed**
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p17 (SVAC tuple)
Approved: 24 May 2005 Bill, Eduardo & Richard. Steve unavailable on short notice, but approves by phone.
draft action
svac changes
eLog at v2r2p5
AdditionalInputFiles and suiteRunList column in the elogReport table are changed from varchar2 to CLOB so that they can hold more than 4000 characters of data. update.py is modfified accordingly. (JIRA issue: http://jira.slac.stanford.edu/browse/SVAC-53)
SVAC workset at v3r1p14.
Changes to fix http://jira.slac.stanford.edu/browse/SVAC-6 (the job was only halfway done before, as noted in comments on that iisue).
Some code cleanup and fixes to scripts to start reprocessing runs.
Code Versions
Engineering Model (sim/recon) v4r060302p23
System Tests for this version
System Tests Result
FRED version
Pipeline tag
v1r0p2
GRITS tag (web browsing and task configuration)
glast-ground v0r3p4
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)
online/svac (task defs, scripts):
pipeline tasks:
online: v2r1p2
svac pipeline code and tasks:
code/tasks v3r1p14 **changed**
ISOC code and tasks:
v0r5p0
Apps that run in pipeline:
eLog: v2r2p5 **changed**
ConfigTables: v3r1p3
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p17 (SVAC tuple)
Approved: unanimous 13 May 2005
The following runs:
135001832 135001834 135001838 135001840 135001844 135001846
of the following tasks:
recon-v3r1p12 reconReport-v3r1p12 svacTuple-v3r1p12
were deleted from GINO's DB, and recon-v3r1p12 (which should launch the other two) resubmitted.
This was necessary to take advantage of new CAL calibrations.
Warren Focke
I updated GINO's DB to set the LSF queue for online-v2r1p1/ldf2fits to medium (was short). This was to address JIRA issue PIT-5 .
Warren Focke
> The following runs of online-v2r1p1 were deleted from GINO's DB
AGAIN
> and resubmitted
AGAIN, because the original, corrupt FITS files did not get overwritten the first time I did it, so this time I had to delete the FITS files, too.
>
> 135001500
> 135001501
> 135001503
> 135001504
> 135001721
> 135001722
>
> Warren Focke
> 2005/05/04
>
>
The following runs of online-v2r1p1 were deleted from GINO's DB and resubmitted to address JIRA issue PIT-5 .
135001381
135001500
135001501
135001503
135001504
135001506
135001507
135001718
135001719
135001721
135001722
Warren Focke
2005/05/04
The following runs were deleted from GINO's DB:
135001500 135001501 135001503 135001504 135001461 135001466 135001467
for tasks:
configReport-v3r1p10 digiReport-v3r1p10 digitization-v3r1p10 online-v2r1p1 recon-v3r1p10 reconReport-v3r1p10 svacTuple-v3r1p10 updateELogDB-v3r1p10 (some run/tasks combinations in that table are actually empty)
and online-v2r1p1 was resubmitted for all of them. This was necessary because the ldf.fits file had not been correctly picked up the first time they were run thru.
EM changes
EngineeringModel v4r060302p23
calibGenCAL v3r6p4
Zach Fewtrell
Most significant is the ENABLE_GRP ciFit jobOption which allows the user to disable smoothing and grouping in intNonlin splines, this is necessary for charge injection data which is taken with the incorrect calib_gain setting. Sasha's previous patch had the same result, but this solution is more elegant. Tested that both tags produce the same output. There was a minor xml formatting issue in ciFit which needed to be fixed for multi-tower use. Basically the instrument dimensions (nCol & nRow) in the xml header info were incorrect. Minor updates include improved progress print outs & better comments in the
ciFit smoothing code.
RootAnalysis v8r3
Heather Kelly
Added examples for GEM and Diagnostic processing.
TkrRecon v10r5p8
Tracy Usher and Leon Rochester
The new default values for maxGaps and maxConsecutiveGaps are set to (2,1) which should be appropriate for I&T and not adversely affect simulation studies and includes fix for first-hit energy which was being incorrectly set in one-track events. Addresses JIRA http://jira.slac.stanford.edu/browse/TKR-14 and http://jira.slac.stanford.edu/browse/TKR-15
LatIntegration v2r26
Anders Borgland
Minor fix to VDG JobOptions files.
RootAnalysis v8r2
Heather Kelly
Fixes crash when reading new recon ROOT files, there was an error in how the TkrRecon data was being processed.
Addresses JIRA http://jira.slac.stanford.edu/browse/ROOT-9
svac changes
SVAC scripts & tasks are at v3r1p11. This includes some changes to
scripts:
- modify queryElogReportTable.pl to exit with code nonzero if the queried
info is not in the database (JIRA issue: http://jira.slac.stanford.edu/browse/SVAC-46) - modify recon.pl to use VDG.txt in LatIntegration job option directory to
reconstruct VDG photons (JIRA issue: http://jira.slac.stanford.edu/browse/SVAC-44) - use eLogFeeder v2r2p4 to exit with exited code 1 if sql statement is not
successfully executed
TestReport v3r2p7
The title of the Gem delta event time histogram says: "Time interval between adj acent event in millsecond with a cut of 1 ms." It should be: "Time interval betw een adjacent events in milliseconds." There is no cut of 1 ms.
http://jira.slac.stanford.edu/browse/Jira-SVAC-14
EngineeringModelRoot v1r3p18
- awb Removed 'EventSequence' as it now will be identical to EventID.
- awb Removed 'GoodEvent'. Users should require 'EventFlags==0' to have
events without any errors. - awb Renamed mergeRoot_debug to mergeRoot_mc. The 'bug' it fixed is no
longer in Root V4. However, merging mc.root files does not work with
mergeRoot for some reason. The memory usage goes through the roof
and it essentially comes to a stand still. - awb Changed default file size to 25 GB in mergeRoot, mergeRoot_mc and
mergeNtuple. - awb Updated the documentation.
- wbf Added EvtTicks.
EngineeringModelRoot is v1r3p18. The only change from p17 is fixing a bug in the evtTicks variable that was introduced in that release.
online
Anders and Warren found a minor bug in the online pipeline. The LDF file choosing mechanism was flawed and sometimes grabbed the errXXXXXXXXX.ldf file. This was run through ldf2digi which discarded all the error events. As this file is an error skim, all events in the file were discarded.
I've fixed the code, and I'd like to run it through your CCB. The CVS tag is onlinePipeline V02-01-02. There is no change to the pipeline xml.
http://jira.slac.stanford.edu/browse/PIT-4
Code Versions
Engineering Model (sim/recon) v4r060302p23 **changed**
System Tests for this version
System Tests Result
FRED version
Pipeline tag
v1r0p2
GRITS tag (web browsing and task configuration)
glast-ground v0r3p4
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)
online/svac (task defs, scripts):
pipeline tasks:
online: v2r1p2 **changed**
svac pipeline code and tasks:
code/tasks v3r1p12 **changed**
ISOC code and tasks:
v0r5p0
Apps that run in pipeline:
eLog: v2r2p4
ConfigTables: v3r1p3
TestReport: v3r2p7 (digi & recon reports)**changed**
EngineeringModelRoot: v1r3p17 (SVAC tuple)
Approved: Eduardo, Richard, Bill 4 May 2005