From Anders:
We have another request to the CCB. Because of large LDF file sizes some runs time out in the batch queue for the online task (in ldf2fits). More details in JIRA http://jira.slac.stanford.edu/browse/OSYS-27
. We have a new tag than changes the queue from short to long: online v2r2p0
This tag also has a fix for logic bug in the wrapper script (still JIRA OSYS-27). The check to see if '$rc' has been defined has been moved out so that we only use it in case it actually has been defined.
We are planning to switch to a new pipeline tomorrow for the EngineeringModelRoot fix and would like to do this change at the same time.
From Jim Panetta:
There's a new online pipeline tag containing the following fixes:
1) Check of value of $rc in wrapper fails with !defined error
Reorder if statement, checking for defined($rc)
2) Move running of ldf2fits job to long queue from short queue.
Several long ldf2fits jobs have failed for mysterious reasons,
LSF not saying anything about the failure. Investigation showed
that the slac normalized CPU time for these failed jobs was
almost exactly 20 minutes, indicating that LSF probably terminated
them.
This tag is V02-02-00, or v2r2p0, depending on naming conventions.
Anders or Warren will install the new tag by:
a) cvs update -r V02-02-00 in the onlinePipeline area
b) uploading the online-v2r2p0.xml file from the above area
From Warren:
The change to the wrappers (16 of mine, 1 of Jim's) consists of replacing:
---------------------------------------------------
my $rc = $ex->execute();
if ($rc == 0) {
#terminated successfully:
exit(0);
} elsif ( defined($rc) ) {
#your app failed, interpret return code
#and then exit non-zero
#(do some stuff here if you want)
exit($rc);
} else {
---------------------------------------------------
with:
---------------------------------------------------
my $rc = $ex->execute();
if ( defined($rc) ) {
if ( $rc == 0 )
else
} else {
---------------------------------------------------
SVAC tasks will be v3r1p23.
Code Versions
Engineering Model (sim/recon) v5r0608p2
System Tests for this version
System Tests Result
FRED version
Pipeline tag
v1r0p2
GRITS tag (web browsing and task configuration)
glast-ground v0r3p7
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)
online/svac (task defs, scripts):
pipeline tasks:
online: v2r2p0 **changed**
svac pipeline code and tasks:
code/tasks v3r1p23 **changed**
ISOC code and tasks:
v0r5p0
Apps that run in pipeline:
eLog: v2r2p6
ConfigTables: v3r1p4
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p19(SVAC tuple)
Changes made in the PROD server today: report from Dan Flath -
(1) Added table Archive:
(a) Columns; Archive_PK, ArchiveStatus_FK
(b) Constraints; PK_Archive (Pri Key),
FK_ArchiveStatus_Archive (Foreign Key Archive.ArchiveStatus_FK -> ArchiveStatus.ArchiveStatus_PK)
(c) Indexes; Archive_ArchiveStatus_idx (on Archive.ArchiveStatus_FK)
(d) Sequences; Archive_Seq (for Primary Key values)
(2) Removed from Run table:
(a) Columns; ArchiveFileName, ArchiveStatus_FK
(b) Constraints; FK_ArchStat_Run (Foreign Key Run.ArchiveStatus_FK -> ArchiveStatus.ArchiveStatus_PK)
(2) Added to Run table:
(a) Columns; Archive_FK
(b) Constraints; FK_Archive_Run (Foreign Key Run.Archive_FK -> Archive.Archive_PK)
(c) Indexes; Run_Archive_idx (on Run.Archive_FK)
The short story is that these changes will reduce data duplication and keep the schema highly normalized (the current state of things). All archive info in the Run table is now in an ancilliary table and linked by a foreign key relationship.
I'll have to update the stored procedures and archive code. I'll update again later when that's done.
Thanks,
Dan
svac changes
Dario discovered a bug in the filling of an SVAC ntuple variable, TkrNumClusters, see JIRA http://jira.slac.stanford.edu/browse/SVAC-63. This bug affects all 2 and 4 tower data and MC.
The bug has been fixed in EngineeringModelRoot v1r3p19. We would like to put this into the pipeline asap (in particular before the 6 tower SVAC data runs this week).
Because of the nature of the bug we need to reprocess the 2 and 4 tower data set. In the current pipeline setup this means reprocessing everything from digi on (including recon). We will look closer at disk space issues before we do this.
Plus a booboo: this code was inadvertently included in the live pipeline code:
The patch allows selecting geometry files other than the "flight" one, based on the number of towers present. These are necessary because configurations with 10, 12, or 14 towers have "voids" in the grid. Fixes http://jira.slac.stanford.edu/browse/SVAC-60.
Code Versions
Engineering Model (sim/recon) v5r0608p2
System Tests for this version
System Tests Result
FRED version
Pipeline tag
v1r0p2
GRITS tag (web browsing and task configuration)
glast-ground v0r3p7
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)
online/svac (task defs, scripts):
pipeline tasks:
online: v2r1p2
svac pipeline code and tasks:
code/tasks v3r1p18
ISOC code and tasks:
v0r5p0
Apps that run in pipeline:
eLog: v2r2p6
ConfigTables: v3r1p4
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p19(SVAC tuple) **changed**
Approval: 27 June: Bill, Richard, Steve. Eduardo out of town.
EngineeringModel v5r0608p2
calibGenCAL v3r6p13
Zach Fewtrell
Improves FLE dac settings generation for online as well as FLE threshold
measurements for offline.
The problem was such:
LEX8 ADC scale is used to characterize & measure FLE thresholds.
Hardware LEX8 channel doesn't have the dynamic range to handle the upper
ranges of the FLE trigger.
The solution was to continue to use LEX8 ADC units, but to extrapolate past
the hardware limit when needed..
Also includes a new validation tool, tholdCIVal.
Changes are only to python scripts not to C++ applications
Code Versions
Engineering Model (sim/recon) v5r0608p2 **changed**
System Tests for this version
System Tests Result
FRED version
Pipeline tag
v1r0p2
GRITS tag (web browsing and task configuration)
glast-ground v0r3p7
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)
online/svac (task defs, scripts):
pipeline tasks:
online: v2r1p2
svac pipeline code and tasks:
code/tasks v3r1p18
ISOC code and tasks:
v0r5p0
Apps that run in pipeline:
eLog: v2r2p6
ConfigTables: v3r1p4
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p17 (SVAC tuple)
Approval: unanimous 23 June; Steve wondering about the saturation fix.
CAS handles the web login to the pipeline configuration pages.
From Matt:
The upgrade of CAS to Tomcat seems to have gone without a hitch. Pipeline Front End and System Tests should not notice any difference. Please let me know id you notice any problems.
For reference (and CCB purposes?) the new CAS server is tagged in CVS as v1r0p3, and is running on glast05 under Tomcat 5.5.9.
EngineeringModel v5r0608p1
GlastPolicy v6r4
astro v1r9p1
Trigger v4r0p1
flux v8r18
celestialSources v1r0p4
FluxSvc v6r23p4
CalDigi v2r1p2
CalibSvc v0r21p3
ntupleWriterSvc v3r6p1
merit v6r20
Gleam v5r13p1
userAlg v6r4p5
CRflux v1r3p1
HepRepCorba v1r5p2
All of the above were picked up from GlastRelease v6r8.
Removed FluxDisplay just as GlastRelease has since FRED does not use it.
calibUtil v1r4
Joanne Bogart
Adds new application, calibCoverage, which is used to verify that the metadata database has a sensible set of calibrations.
TkrRecon v10r5p8em0
Tracy Usher
Branch created to include fix for kal theta multiple-scattering angle which had an artificial cutoff which had been necessary to keep the Kalman energy from being computed as an infinite value if the angle turned out to be zero. Now both a "raw" and "modified" value are used such that the "real" multiple-scattering angle with no cutoff is now reported.
GlastPolicy/RootcintPolicy v5r0p0
Heather Kelly
Added new pattern to allow more than just one shared library to be built in a ROOT package. This was done to fulfill a request by Johann that RootAnalysis build a LeaningTower shared library.
commonRootData v2r1p0
Heather Kelly
Uses new version of RootcintPolicy that includes new pattern for creating multiple shared libraries.
mcRootData v2r16p0
Heather Kelly
Uses new version of RootcintPolicy that includes new pattern for creating multiple shared libraries.
digiRootData v8r4
Heather Kelly
Uses new version of RootcintPolicy that includes new pattern for creating multiple shared libraries.
Also adds in new error summary class to store LDF errors.
reconRootData v5r6p3
Heather Kelly
Uses new version of RootcintPolicy that includes new pattern for creating multiple shared libraries.
calibRootData v1r7p1
Joanne Bogart
Uses new RootcintPolicy that includes new pattern for creating multiple shared libraries.
IExternal v4r1p2
Navid Golpayegani
Pick up more recent IExternal/ROOT which was updated to load the TreePlayer shared library along with the other ROOT GUI libraries.
calibGenTKR v2r6
Hiro Tajima and Joanne Bogart
Updates muonCalibTot which now reads hot and dead strips xml files
and remove them from disconnected strips. Hot strips are read from data directory since they are used to mask hot strips during data taking. The dead strips xml file can be specified via job options. This tag requires the latest facilities tag v2r12.
(Hiro) and enhance xml2root application so that it can append to or replace information in a pre-existing ROOT file. (Joanne)
facilities v2r12
Joanne Bogart
Adds new utility Util::expandEnvVarOS.
calibGenCAL v3r6p11
Zach Fewtrell
Addresses JIRA http://jira.slac.stanford.edu/browse/CAL-10, http://jira.slac.stanford.edu/browse/CAL-11, http://jira.slac.stanford.edu/browse/CAL-12
RootAnalysis v8r5
Heather Kelly and Michael Kuss
Creates a shared library for Leaning Tower at Johann's request.
RootTreeAnalysis was fixed to be ACLIC-able and includes a new macro that demonstrates how to compile RootTreeAnalysis using ACLIC at the ROOT command line.
Addresses JIRA http://jira.slac.stanford.edu/browse/SVAC-59 by removing code in LeaningTower that slowed down alignment.
xmlGeoDbs v1r22p4
Anders Borgland
Add 4, 6 and 8 tower geometries to the latAssembly.
xmlBase v5r2
Joanne Bogart
Adds parser set up method to support xml files using Schema as well as those using the old-style DTD
ldfReader
Heather Kelly
Handle Error Summary which may appear in TEM data in the LDF.
LdfConverter v1r13
Heather Kelly
Address JIRA http://jira.slac.stanford.edu/browse/SVAC-57
Handle Error Summary which may appear in TEM data in the LDF
LdfEvent v2r13
Heather Kelly
Create a new class to store the error summary which may appear in the TEM data in the LDF.
RootIo v15r8p1em1
Heather Kelly
Reads and writes error summary data to/from the TDS to/from ROOT.
LatIntegration v2r29
Anders Borgland
Introduced new jobOptions parameter that allows a warning to be printed in the GEM condition summary is zero. This addresses JIRA http://jira.slac.stanford.edu/browse/SVAC-57.
CalXtalResponse v0r5p1
Zach Fewtrell
Fix in response to apparent CAL ADC shift reported by Anders Borgland (SVAC). Repairs included fixes to retrieving constants from the GlastDetSvc for the geometry, where the fSement variable was left un initialized. This caused the ADC shift, since fSegment specifies the volId index for a crystal segment. This information is vital for determining the
asym->pos calculation in CalDigi.
Code Versions
Engineering Model (sim/recon) v5r0608p1 **changed**
System Tests for this version
System Tests Result
FRED version
Pipeline tag
v1r0p2
GRITS tag (web browsing and task configuration)
glast-ground v0r3p7
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)
online/svac (task defs, scripts):
pipeline tasks:
online: v2r1p2
svac pipeline code and tasks:
code/tasks v3r1p18
ISOC code and tasks:
v0r5p0
Apps that run in pipeline:
eLog: v2r2p6
ConfigTables: v3r1p4
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p17 (SVAC tuple)
Approval: unanimous 21 June, 2005
Due to hardware problems on glast04, the scheduler cron job has been de-activated on glast04 and initiated on glast02. A demo task has been run to demonstrate successful transfer.
ConfigTables v3r1p4 was not installed correctly. The code has been moved to the proper location and the following runs (all test runs, 151 of 'em) of configReport-v3r1p7 were deleted and resubmitted:
135002253
135002254
135002255
135002256
135002257
135002258
135002260
135002261
135002262
135002263
135002264
135002265
135002266
135002267
135002269
135002270
135002271
135002272
135002273
135002274
135002275
135002276
135002277
135002278
135002279
135002281
135002282
135002283
135002284
135002285
135002286
135002287
135002288
135002289
135002290
135002291
135002292
135002293
135002294
135002295
135002296
135002297
135002298
135002299
135002300
135002301
135002302
135002303
135002304
135002305
135002306
135002307
135002308
135002309
135002310
135002311
135002312
135002313
135002314
135002315
135002316
135002317
135002318
135002319
135002320
135002321
135002322
135002323
135002324
135002325
135002326
135002327
135002328
135002329
135002330
135002331
135002332
135002333
135002334
135002335
135002336
135002337
135002338
135002339
135002340
135002341
135002342
135002343
135002344
135002345
135002346
135002347
135002348
135002349
135002350
135002351
135002352
135002353
135002354
135002355
135002356
135002357
135002358
135002359
135002360
135002361
135002362
135002363
135002364
135002365
135002366
135002367
135002368
135002369
135002370
135002371
135002372
135002373
135002374
135002375
135002376
135002377
135002378
135002379
135002380
135002382
135002383
135002384
135002385
135002386
135002387
135002388
135002389
135002390
135002391
135002392
135002393
135002394
135002395
135002396
135002397
135002398
135002399
135002400
135002401
135002402
135002403
135002404
135002405
135002406
135002407
Warren Focke
Changed svac/online task output to u16 partition from u14.
Code Versions
Engineering Model (sim/recon) v4r060302p23
System Tests for this version
System Tests Result
FRED version
Pipeline tag
v1r0p2
GRITS tag (web browsing and task configuration)
glast-ground v0r3p7
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)
online/svac (task defs, scripts):
pipeline tasks:
online: v2r1p2
svac pipeline code and tasks:
code/tasks v3r1p18 **changed**
ISOC code and tasks:
v0r5p0
Apps that run in pipeline:
eLog: v2r2p6
ConfigTables: v3r1p4
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p17 (SVAC tuple)