Blog

CCB request 20051121

CCB Request for EngineeringModel v5r0703p2

GlastPolicy\RootcintPolicy v5r0p3
Heather Kelly
Update to vc_proj_library fragment for creating rootcint files for packages where there are more than one library created..specifically for RootAnalysis' Leaning Tower library

AcdDigi v1r14p2
Heather Kelly
Patch to edge effects for faces 1 and 3

LatIntegration v2r43
Anders Borgland

Added full LAT surface muon MC system test.
Limit some system tests to 100k events to avoid time out Correct sequence for the new Cal Recon

RootAnalysis v8r8p1
Michael Kuss, Heather Kelly Johann Cohen-Tangui Add johann's mergeFiles macro Updates to LeaningTower geometry Patches for windows

Em1 v2r5p1
Heather Kelly
Update JO file for test routine by removing old FluxDisplay

Em2 v1r5p1
Heather Kelly
Update JO file for test routine by removing old FluxDisplay

calibGenCAL v3r8p3
Zach Fewtrell and Dan Wood

Added tools 'adc2nrgVal' and 'biasVal' and 'adcsmooth', 'genGainSettings'
python scripts now support new intNonlin format (calCalib_v2r3.dtd) extrapolate all ciFit splines to saturation point.

calibGenTKR v2r7p4
Hiro Tajima

CalXtalResponse v0r7p5
Zach Fewtrell

fixed bug reported by philippe: only visible in ideal-mode high-asymmetry xtal deposits near 1050 MeV support new intNonlin data w/ optional per-channel dac values.
removed local caching of TDS RangeBase pointers in order to simplify the code.
fixed bug, found by Richard Dubois. when few lines of code were moved from computeEnergy function to execute function, the return when below_threshold flag is set became invalid, because it caused skipping of all subsequent digis - now replaced by continue statement.
fixed bug where wild hit positions were not clipped to xtal boundaries.
XtalRecTool combins EneTool & PosTool. FLE & FHE triggers now calculated. Also ULDs are now properly checked against ped subtracted adc values.

There were picked up as part of GR v7r3

IExternal/GaudiInterface v0r120p8
Toby Burnett
Patch to allow VC8 release builds

IExternal/ldfExt v0r561p2
Navid Golpayegani
Patch to allow VC8 release builds

xmlGeoDbs v1r25p4
Joanne Bogart
Updated ACD geometry to include tile shrinkage at operating temperature

astro v1r11p4
Toby Burnett and James Chiang
Move to use the tip packages and upgrades associated with HEALpix facilities v2r12p3 Joanne Bogart Added two new methods unsigned to string and string to unsigned.

idents v2r15
Heather Kelly
Add support for N/A in the AcdIds
JIRA ACD-4

detModel v2r18p2
Joanne Bogart
Updates to handle ACD screws

Event v11r8
Tracy Usher, Heather Kelly and assorted others move to new CalRecon Updates to ACD TDS structures

GlastSvc v9r16
Joanne Bogart
Fix to pushShape to handle ACD screws

commonRootData v2r4
Heather Kelly
support NA in AcdId
JIRA ACD-4

mcRootData v2r19p0
Tracy Usher
Add McTrajectory

digiRootData v8r7
Heather Kelly
Store AEM header parity bits
Modification of AcdDigi methods after consulting Alex Moiseev (GSFC).

reconRootData v8r6p1
Tracy Usher and Heather Kelly
Updates for new CalRecon
Updates for AcdRecon - ribbon active distance, corner DOCA, etc

RootIo v17r9p1
Tracy Usher and Heather Kelly
New CalRecon, updates to AcdRecon

Trigger v4r2
Toby Burnett
Enable Ritz throttle

flux v8r21p2
Toby Burnett
Include improvements to Earth from Dirk Petry Limit chime to 1 TeV

celestialSource v1r0p5
Jim Chiang

tag for GRBobsmanager

FluxSvc v6r28p1
Toby Burnett
Fix bug to StartTimeEnvVar
Provide access to ScienceTools sources

OnboardFilter v1r1p8gr0
Julie McEnery

calibUtil v1r5p2
Joanne Bogart

Add per-range sdacs attribute to <intNonlin>

CalibData v0r12p3
Joanne Bogart

Allow for array of sdacs per channel for intNonlin

CalibSvc v0r21p6
Joanne Bogart

Bring main branch up to snuff wrt intNonlin calibs

CalDigi v2r3p1
Zach Fewtrell

Save GltDigi Cal trigger info per xtal-face instead of per tower.
Save FLE &FHE trigger info to TDS. Uses CalXtalResponse >=
v0r7

CalRecon v6r1p15
Tracy Usher
New Cal Recon

TkrDigi v2r4p7
Leon Rochester

Bug-fix in GeneralHitRemovalTool
Changes for compatibility with modified IGeometry interface for ACD screws Add GeneralHitRemovalTool, and associated changes Does all hit removal (except for noise) including hit truncation
SiStripList: check if both coordinates are inside active area

TkrUtil v3r5p8
Leon Rochester

Make towerFactor a Property for TkrQueryClustersTool Changes for compatibility with modified IGeometry interface Fix TkrSplitsSvc, and test program TkrSplitsSvc now handles hit truncation consts

TkrRecon v10r8p1
Leon Rochester and Tracy Usher

Update default cluster cut from 500 to 2000 new code for truncated hits Changes for compatibility with modified IGeometry interface for ACD screws Add protection in Combo Pat Rec against "bad" cal axes Add alignment correction to recon Tag to go with new CalRecon output
ComboFindTrackTool: implement MC energy and pointing option

cancel hit-limiting for poorly measured Cal tracks

new method to provide limit and position prediction Remove tabs (I meant to do this earlier!)
FindTrackHitsTool: fix minor radlen bug

Utilities: implement TkrPoints wrt a reference point

ComboFindTrackTool: use new TkrPoints

Limit CAL finding to cone around CAL-track

Fix minor radlen and layer-numbering bug

AcdRecon v1r12p5
Heather Kelly and Eric Charles
Add Corner DOCA
Add ActTkrIntersectTool

DetDisplay v3r1p7
Joanne Bogart
Change to IGeometry interface to support ACD screws

G4Generator v5r11p2
Tracy Usher and Joanne Bogart

New McPositionHits, order McTrajectories by generation Updated IGeometry interface for ACD screws

ntupleWriterSvc v3r8
Toby Burnett

Add access function to an existing tuple item.
Make a table of bad values; provide property to allow bad entries anyway Have RootTupleSvc reject tuple entries with non-finite values

G4Propagator v2r1p11
Tracy Usher

fix the problem with ray parallel to surface Addresses JIRA GEANT-5

merit v6r27p2
Toby Burnett

Support for new GlastClassify trees
Add new Doxygen documentation for inclusion in the user workbook

Gleam v6r7p3
Toby Burnett

add FT1Alg to triggered sequence
Turn on CalMipFinder
Add AcdUtil for new geometry service

AnalysisNtuple v2r9p1
Leon Rochester
Updated Doxygen documentation for inclusion in th user workbook

TkrSurplusHitRatio: #hits outside/#hits inside energy/angle-dependent cone
TkrSurplusHCInside: #hits inside cone
TkrUpstreamHC: #hits inside a cylinder upstream of the track
Tkr1CoreHC: total # extra hits in small circles around each hit on track
1
Deprecated: TkrTotalHits – same as TkrSurplusHCInside Add new ACD variables for 3D Active Distance and CornerDOCA.

userAlg v6r4p6
Toby Burnett

minor fix for merit properties

CRflux v1r5
Toby Burnett

adjust date from 2000 to 2001 for solar flux

ldfReader v2r5
Heather Kelly

Handle AEM header parity bit (CMD/Data error)

LdfEvent v2r13p2
Heather Kelly
Fix compiler warnings in EventSummaryData

LdfConverter v1r17p1
Heather Kelly

Set n/a for AcdIds
Handle ACD Header Parity from AEM contribution

rootTestData v3r4
Heather Kelly
New CalRecon

HepRepSvc v0r14p7
Leon Rochester and Joanne Bogart
Update IGeometry interface for ACD screws
Upgraded information for clusters and track hits
Bullet-proof TrackFiller for bad TkrIds

rdbModel v2r2p3
Joanne Bogart

Support for unsigned auto_increment fields

svac/EngineeringModelRoot v1r8p3
Anders W. Borgland

  • Updated the SVAC ntuple to work with the new CAL Recon
  • Added new CAL MIP finder variables: SVAC-79
  • Added ACD digi and recon variables: SVAC-77
  • Added TKR_ACD intersection variables: SVAC-78
  • Store histograms for TKR in a separate tree in the SVAC root file
    to speed up TKR calibrations: SVAC-81
  • Increased max files size for SVAC ntuple to 500 GB.

Code Versions

Engineering Model (sim/recon)v5r0703p2

System Tests for this version

System Tests Result

System tests:

anders

FRED version

0.98

Pipeline tag

v1r0p2

GRITS tag (web browsing and task configuration)

glast-ground v0r3p7
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)

online/svac (task defs, scripts):

pipeline tasks:

online: v2r3p0

svac pipeline code and tasks:

code/tasks v3r3p3 **changed**
pipelineDatasets v0r3

ISOC code and tasks:

v0r5p0

Apps that run in pipeline:

eLog: v2r2p7
ConfigTables: v3r1p5
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p20(SVAC tuple)

Approval:

SVAC tasks/scripts at v3r3p3 to switch Oracle from SLACPROD to GLASTP.

Code Versions

Engineering Model (sim/recon)v5r0608p6

System Tests for this version

System Tests Result

System tests:

anders

FRED version

0.98

Pipeline tag

v1r0p2

GRITS tag (web browsing and task configuration)

glast-ground v0r3p7
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)

online/svac (task defs, scripts):

pipeline tasks:

online: v2r3p0

svac pipeline code and tasks:

code/tasks v3r3p3 **changed**
pipelineDatasets v0r3

ISOC code and tasks:

v0r5p0

Apps that run in pipeline:

eLog: v2r2p7
ConfigTables: v3r1p5
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p20(SVAC tuple)

Approval:

Change of Oracle server

SCS moved our SLACPROD Oracle tables to our shiny new dedicated GLASTP server on Saturday Nov 5. We have switched the pipeline and web interface to it as of Sunday and run a DC2 task in it. All seems well.

During this interregnum period, Navid modified the batch sub system to use the u15 partition for temporary storage of batch log files.

This pl/sql mod is to fix handling of foreign datasets. The pipeline was retrieving all runs from the foreign task, not just the one run needed at a time. This was shown to work in test on Dan's demo foreign task pipeline (and subsequently in prod on our reprocessing task).

Here's the diff between new and old. ('--' and everything after is a pl/sql comment)

The new query uses the runname to restrict the selected dataset list.

Dan

glast01:dflath> diff pdbprocedures.sql ~glastdpf/pdb/pdbprocedures.sql 949c949,950
< and T.Task_PK = DS.Task_FK;

> and
> T.Task_PK = DS.Task_FK;
2293c2294
< where TaskProcess_PK = TaskProcess_FK_in;

> where TaskProcess_pk = TaskProcess_FK_in;
2296,2314c2297,2310
< select T.TaskName, DS.DatasetName, DSI.FilePath, DSI.FileName, R.Run_PK, R.RunName --, DSI.DSInstance_PK
< from DSInstance DSI, Task T, Dataset DS, Run R
< where DSI.Run_FK in
< (select Run_PK from Run where RunName =
< (select RunName
< from Run
< where Run_PK = Run_FK_in
< )
< and Task_FK != Task_FK_in
< )
< and DSI.Dataset_FK in
< (select Dataset_FK
< from TP_DS
< where TaskProcess_FK = TaskProcess_FK_in
< and RW = 'R' – note: with task_fk != task_fk_in cut, above, this is probably redundant
< )
< and DSI.Dataset_FK = DS.Dataset_PK
< and DS.Task_FK = T.Task_PK
< and R.Run_PK = DSI.Run_FK;

> – select DS.DATASETNAME, DSI.FILEPATH, DSI.FILENAME, T.BaseFilePath, T.TaskName, R.Run_PK, R.RunName;
>
> select T.TaskName, DS.DATASETNAME, DSI.FILEPATH, DSI.FILENAME, R.Run_PK, R.RunName
> from Dataset DS, DSInstance DSI, Task T, Run R
> where DS.Dataset_PK in
> (select Dataset_FK from TP_DS
> where TaskProcess_FK = TaskProcess_FK_in
> and RW = 'R')
> and DSI.Run_FK = R.Run_PK
> and R.Run_PK = Run_FK_IN
> and DSI.Dataset_FK = DS.Dataset_PK
> – and T.Task_PK = DS.Task_FK
> and T.Task_PK = R.Task_FK
> and T.Task_PK != Task_FK_in;

CCB Request 20051021

SVAC pipeline tasks/scripts v3r3p2, implementing the following changes:

----------------------------------------------------------------

Change some oracle-related environment variables, do both cleanup and kludgification of the methods used to set them. Changes were necessary due to changes in SLAC's default Oracle environment. Changes adress JIRA
PIT-18 .

Changes to partially adress JIRA SVAC-73 (don't interpret bad DB queries as valid but empty data).

Final part of the fix for SVAC-60 (geometry issues w/ 10, 12, 14 towers).

Clean up some cruft left over from when configReport used the schema.

Reduce recon chunk size for VdG runs.

configReport-v3r3p2/ConfigTables was modified (in the GINO DB) to change the LSF queue from short to medium.

Code Versions

Engineering Model (sim/recon)v5r0608p6

System Tests for this version

System Tests Result

System tests:

anders

FRED version

0.98

Pipeline tag

v1r0p2

GRITS tag (web browsing and task configuration)

glast-ground v0r3p7
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)

online/svac (task defs, scripts):

pipeline tasks:

online: v2r3p0

svac pipeline code and tasks:

code/tasks v3r3p2 **changed**
pipelineDatasets v0r3

ISOC code and tasks:

v0r5p0

Apps that run in pipeline:

eLog: v2r2p7
ConfigTables: v3r1p5
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p20(SVAC tuple)

Approval:

For the record:

2005-10-20

Run 135004697 was deleted from tasks svacTuple-v3r3p2 reconReport-v3r3p2 recon-v3r3p2, and recon recreated. The run initially failed recon ianppropriately due to NFS/LSF issues. On rollback, the first process in the task failed due to the presence of files left over from the first run, but this failure was not detected, and downstream tasks were launched inappropriately.

2005-10-18

Run 141000346 was deleted from all tasks where it existed
(configReport-v3r3p1 digiReport-v3r3p1 digitization-v3r3p1 online-v2r3p1
updateELogDB-v3r3p1) and a new GINO run of online-v2r3p1 created for that data-taking run. This was necessary because of a (reproducible) failure to export the run from the DMZ. The data were copied by hand for the new injection.

2005-10-17

Run 134001212 had spent soem 8h finalizing in digitization-v3r3p1.
Deleted and resubmitted succesfully.

Strange thing about the logfile: The LSF output was at the beginning of the file, not at the end as it usual is.

2005-10-14

  • Run 140003114 was stuck in updateELogDB.
  • Run 398001916 had been a week in preparing for digiReportUrl in digiReport-v3r3p1
  • Run 141000315 was stuck in digiRootFile in digitization-v3r3p1.

Deleted and resubmitted.

2005-10-12

In order to clean up after 2005/10/11's pipeline problems:

Runs 140002363 and 134001171 of configReport-v3r3p1 were deleted from GINO's database and resubmitted.

157 runs were deleted from all SVAC tasks where they were present, and restarted. Runs were:

399002655 398001964 398001987 141000329 141000333 141000335 141000334 141000330 141000332 399002687 399002683 399002684 399002686 399002685
399002681 399002682 399002679 399002680 399002674 399002677 399002678
399002672 399002675 399002676 399002673 399002671 399002668 399002669 399002670 399002666 399002667 399002665 399002661 399002660 399002663
399002662 399002656 399002658 399002657 399002654 399002651 399002653
399002652 399002647 399002650 399002646 399002649 399002648 399002644
399002645 399002643 399002642 399002641 399002638 399002632 141000328
399002639 399002633 399002636 399002637 399002635 134001192 134001206
399002631 134001208 134001200 134001205 134001203 134001194 134001201
134001199 134001195 134001196 134001198 398002030 398002027 398002028
398002029 134001184 134001183 134001188 134001189 134001180 134001182
399002628 399002630 134001181 399002627 399002624 399002626 399002625
398002026 134001177 134001178 398002025 134001175 398002023 398002019
398002022 398002021 398002020 398002014 398002016 398002017 398002015
398002011 398002013 398002012 398002009 398002010 398002008 134001167
398002007 398002006 398002002 398002003 398002005 398002001 398001999 398002000 398001998 398001992 398001995 398001996 398001997 398001993
398001994 398001991 398001989 398001990 398001988 398001981 398001982
398001986 398001984 398001983 398001978 398001980 398001977 399002621
399002623 399002622 398001973 398001975 398001974 398001972 398001976
398001971 398001966 398001968 398001969 398001963 398001970 398001965
398001967 399002619 399002620

LSF Upgrade 20050911

LSF was upgraded from version 5 to 6. Some changes to the log file format engendered some mods to the LSFoutput.pm module in the pipeline.

CCB Action 20050818

svac updates

svacPipeline v3r3p1:
Use glastdata SLF group for chunk jobs in parallel recon.
Increase chunk size for parallel recon.
Add more logging to recon.

ConfigTables v3r1p5:
Don't crash on runs with no TEMs.

EngineeringModelRoot v1r3p20

Sets the maximum Root file size to 500 GB instead of 1.8 GB.

Will see how 8 tower data taking goes. If we hit the file size limit we would need to add this tag before reprocessing.

Code Versions

Engineering Model (sim/recon)v5r0608p6

System Tests for this version

System Tests Result

System tests:

I have checked all system tests histograms for EM p3 wrt EM p1.
Two changes were seen for MC. One is under investigation by Leon
(increased active area in the geometry increases unexpectedly the number
of MC hits), the other was due to a change in the ideal flight mode and
affected only one one MC system test that do not use calibrations.

There were no differences in system tests histograms between p4
and p3.

There were some differences in two MC system tests between p5 and
p4 because of new CAL calibration constants I put in yesterday in
preparation of datataking.

There were no differences in system tests histograms between p6
and p5. Note that I have added 4 and 6 tower data and 6 tower allgamma
system tests as of p6. They are currently not available for viewing -
there is a JIRA out on it from Julie.

anders

FRED version

0.98

Pipeline tag

v1r0p2

GRITS tag (web browsing and task configuration)

glast-ground v0r3p7
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)

online/svac (task defs, scripts):

pipeline tasks:

online: v2r3p0

svac pipeline code and tasks:

code/tasks v3r3p1 **changed**
pipelineDatasets v0r3

ISOC code and tasks:

v0r5p0

Apps that run in pipeline:

eLog: v2r2p7
ConfigTables: v3r1p5 **changed**
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p20(SVAC tuple) **changed**

Approval: Richard 18 Aug 2005 (everyone else absent!)

CCB Action 20050810

EngineeringModel v5r0608p6

RootIo v15r8p1em6: Remove memory leak/overwrite - 'Reset TProcessID ObjectNumber in RootIoSvc rather than in mcRootWriter. Put in fix to clear commonData maps in RootIoSvc'

ldfConverter v1r15: Add two JO to be able to be able to jump directly to a specific event in a Fits file.

xmlGeoDbs v1r22p7
Leon Rochester and Anders Borgland
Address JIRA http://jira.slac.stanford.edu/browse/GEO-6
Add support for 10, 12, 14 towers and 8 tower plus ACD configurations.

GlastSvc v9r13p1
Leon Rochester
Addresses JIRA http://jira.slac.stanford.edu/browse/GEO-5, where non-zero alignment constants sometimes caused non-physical strip numbers to be generated.

RootIo v15r8p1em4
Heather Kelly
Minor fix (one line) for windows, add explicit return of Gaudi StatusCode. Also modifies the default file split size from 25 GB to 500 GB.

CalXtalResponse v0r6p6
Zach Fewtrell
Fixed bug in mixed diode XtalDigi that had bad position estimates for diode deposit hits (CsI hits were fine).
Fixed a minor bug in recent CalTuple code (bad var init). The bad code was so young that it never made it into a tagged EM or GR release, so net bugfixes for EM & GR tags is just 1.
RECALIBRATION: ideal mode flight noise & LAC thresholds have been updated to better approximate expected flight noise levels.

TkrUtil v3r4p1
Leon Rochester
Fix for JIRA http://jira.slac.stanford.edu/browse/TKR-17
The default nominal constants have been adjusted to match the average for Tkr A and B. The constant fCPerMip has been changed to 5.0, to match the value used by Hiro in generating the ToT constants. The constant MeVPerMip has been changed from 0.155 to 0.113. Was using the mean energy, but should have been using the most probable value.

calibGenTKR v2r6p2
Hiro Tajima

Fix row and col definition of tower. Add several plots to monitor
tot fit parameters.

ldfReader v2r4p2
Heather Kelly
Fix for JIRA http://jira.slac.stanford.edu/browse/ROOT-15, where the log messages contained only the 17 bit event sequence rather than the new 32 bit event number.

LdfConverter v1r14
Heather Kelly
In response to JIRA http://jira.slac.stanford.edu/browse/LDF-25, adds a check to LdfCalDigiCnv for all four readouts in ALLRANGE mode.

HepRepCorba v1r6
Riccardo Giannitrapani
Addresses JIRA http://jira.slac.stanford.edu/browse/FRED-40

calibUtil v1r5p1
Joanne Bogart

Addresses JIRA http://jira.slac.stanford.edu/browse/CALIB-15. Lay groundwork for insert-latest and
supersede functions in rdbGUI.

rdbModel v2r2p2
Joanne Bogart

Addresses JIRA http://jira.slac.stanford.edu/browse/CALIB-15. Lay groundwork for insert-latest and
supersede functions in rdbGUI.

RootIo v15r8p1em4
Heather Kelly
Minor patch (change one constant) to modify the default file split size from 25 GB to 500 GB as requested by I&T.

calibGenCAL v3r7p2
Zach Fewtrell

ciFit & runMuTrigEff will now accept CI files w/ any multiple of
nXtalsPerTower hits per event. i.e. they accept parallel
calibrations. Also runMuTrigEff will intelligently skip the 'scan
events' which are sometimes intersperved w/ data events, & can
slightly mess up the event numbering & binning
runSuiteParallel script added which runs ciFit, muonCalib, muTrig,
***Merge.py, ***Validate.py on a full set of parallel calibration
data. Also the C++ applications now expand more environment
variables in the config options file in order to support the
runSuiteParallel script.

LatIntegration v2r33
Anders Borgland
Contains new system tests (4,6 tower data + all_gamma MC)
Include writing CAL ntuple as the default in the jobOptions files.
Reduce TKR noise in jobOptions file for Monte Carlo runs, addresses JIRA http://jira.slac.stanford.edu/browse/TKR-18.

svac updates

svacPipeline v3r3p0:

Add CAL tuple as ouput from recon (SVAC-69) run digitization on xlong (http://jira.slac.stanford.edu/browse/SVAC-70)

pipelineDatasets v0r3:

increase max tree size to 500 GB (http://jira.slac.stanford.edu/browse/SVAC-71)

Code Versions

Engineering Model (sim/recon)v5r0608p6 **changed**

System Tests for this version

System Tests Result

System tests:

I have checked all system tests histograms for EM p3 wrt EM p1.
Two changes were seen for MC. One is under investigation by Leon
(increased active area in the geometry increases unexpectedly the number
of MC hits), the other was due to a change in the ideal flight mode and
affected only one one MC system test that do not use calibrations.

There were no differences in system tests histograms between p4
and p3.

There were some differences in two MC system tests between p5 and
p4 because of new CAL calibration constants I put in yesterday in
preparation of datataking.

There were no differences in system tests histograms between p6
and p5. Note that I have added 4 and 6 tower data and 6 tower allgamma
system tests as of p6. They are currently not available for viewing -
there is a JIRA out on it from Julie.

anders

FRED version

0.98

Pipeline tag

v1r0p2

GRITS tag (web browsing and task configuration)

glast-ground v0r3p7
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)

online/svac (task defs, scripts):

pipeline tasks:

online: v2r3p0

svac pipeline code and tasks:

code/tasks v3r3p0 **changed**
pipelineDatasets v0r3 **new**

ISOC code and tasks:

v0r5p0

Apps that run in pipeline:

eLog: v2r2p7
ConfigTables: v3r1p4
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p19(SVAC tuple)

Approval: unanimous 13 Aug 2005

Dan upgraded the PROD pipeline to incorporate Navid's batch interface. This was tested out with bulk MC in the test db, and all major tasks have been tried in prod now and work.

CCB Action 20050803

svac changes

svacPipeline V3r2p0:
--------------------
Parallel recon (http://jira.slac.stanford.edu/browse/SVAC-38 and http://jira.slac.stanford.edu/browse/SVAC-68)
Run digiReport on xlong (http://jira.slac.stanford.edu/browse/SVAC-66)
Recon EM hardware without calibrations (http://jira.slac.stanford.edu/browse/SVAC-64)

eLog v2r2p7:
------------
Deal with multi-digit bay numbers (http://jira.slac.stanford.edu/browse/SVAC-65)

onlinePipeline v2r3p0:
--------------------
Support >2G FITS file writing (http://jira.slac.stanford.edu/browse/PIT-12)
Support >8G TAR file writing (http://jira.slac.stanford.edu/browse/PIT-14)

Code Versions

Engineering Model (sim/recon) v5r0608p1

System Tests for this version

System Tests Result

FRED version

0.98

Pipeline tag

v1r0p2

GRITS tag (web browsing and task configuration)

glast-ground v0r3p7
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)

online/svac (task defs, scripts):

pipeline tasks:

online: v2r3p0

svac pipeline code and tasks:

code/tasks v3r2p0 **changed**

ISOC code and tasks:

v0r5p0

Apps that run in pipeline:

eLog: v2r2p7 **changed**
ConfigTables: v3r1p4
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p19(SVAC tuple)

Approval: 04 Aug unanimous

The output disk for the online and svac was changed from u14 to u19. Igor has switched the mirroring to u19 from u16.

From Anders:

We have another request to the CCB. Because of large LDF file sizes some runs time out in the batch queue for the online task (in ldf2fits). More details in JIRA http://jira.slac.stanford.edu/browse/OSYS-27. We have a new tag than changes the queue from short to long: online v2r2p0

This tag also has a fix for logic bug in the wrapper script (still JIRA OSYS-27). The check to see if '$rc' has been defined has been moved out so that we only use it in case it actually has been defined.

We are planning to switch to a new pipeline tomorrow for the EngineeringModelRoot fix and would like to do this change at the same time.

From Jim Panetta:

There's a new online pipeline tag containing the following fixes:

1) Check of value of $rc in wrapper fails with !defined error
Reorder if statement, checking for defined($rc)

2) Move running of ldf2fits job to long queue from short queue.
Several long ldf2fits jobs have failed for mysterious reasons,
LSF not saying anything about the failure. Investigation showed
that the slac normalized CPU time for these failed jobs was
almost exactly 20 minutes, indicating that LSF probably terminated
them.

This tag is V02-02-00, or v2r2p0, depending on naming conventions.
Anders or Warren will install the new tag by:

a) cvs update -r V02-02-00 in the onlinePipeline area
b) uploading the online-v2r2p0.xml file from the above area

From Warren:

The change to the wrappers (16 of mine, 1 of Jim's) consists of replacing:

---------------------------------------------------
my $rc = $ex->execute();

if ($rc == 0) {
#terminated successfully:
exit(0);
} elsif ( defined($rc) ) {
#your app failed, interpret return code
#and then exit non-zero

#(do some stuff here if you want)
exit($rc);
} else {
---------------------------------------------------

with:

---------------------------------------------------
my $rc = $ex->execute();

if ( defined($rc) ) {
if ( $rc == 0 )

Unknown macro: { #terminated successfully exit(0); }

else

Unknown macro: { #your app failed, interpret return code #and then exit non-zero #(do some stuff here if you want) exit($rc); }

} else {
---------------------------------------------------

SVAC tasks will be v3r1p23.

Code Versions

Engineering Model (sim/recon) v5r0608p2

System Tests for this version

System Tests Result

FRED version

0.98

Pipeline tag

v1r0p2

GRITS tag (web browsing and task configuration)

glast-ground v0r3p7
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)

online/svac (task defs, scripts):

pipeline tasks:

online: v2r2p0 **changed**

svac pipeline code and tasks:

code/tasks v3r1p23 **changed**

ISOC code and tasks:

v0r5p0

Apps that run in pipeline:

eLog: v2r2p6
ConfigTables: v3r1p4
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p19(SVAC tuple)

Pipeline update 20050630

Changes made in the PROD server today: report from Dan Flath -

(1) Added table Archive:
(a) Columns; Archive_PK, ArchiveStatus_FK
(b) Constraints; PK_Archive (Pri Key),
FK_ArchiveStatus_Archive (Foreign Key Archive.ArchiveStatus_FK -> ArchiveStatus.ArchiveStatus_PK)
(c) Indexes; Archive_ArchiveStatus_idx (on Archive.ArchiveStatus_FK)
(d) Sequences; Archive_Seq (for Primary Key values)

(2) Removed from Run table:
(a) Columns; ArchiveFileName, ArchiveStatus_FK
(b) Constraints; FK_ArchStat_Run (Foreign Key Run.ArchiveStatus_FK -> ArchiveStatus.ArchiveStatus_PK)

(2) Added to Run table:
(a) Columns; Archive_FK
(b) Constraints; FK_Archive_Run (Foreign Key Run.Archive_FK -> Archive.Archive_PK)
(c) Indexes; Run_Archive_idx (on Run.Archive_FK)

The short story is that these changes will reduce data duplication and keep the schema highly normalized (the current state of things). All archive info in the Run table is now in an ancilliary table and linked by a foreign key relationship.

I'll have to update the stored procedures and archive code. I'll update again later when that's done.

Thanks,

Dan

CCB Action 20050627

svac changes

Dario discovered a bug in the filling of an SVAC ntuple variable, TkrNumClusters, see JIRA http://jira.slac.stanford.edu/browse/SVAC-63. This bug affects all 2 and 4 tower data and MC.

The bug has been fixed in EngineeringModelRoot v1r3p19. We would like to put this into the pipeline asap (in particular before the 6 tower SVAC data runs this week).

Because of the nature of the bug we need to reprocess the 2 and 4 tower data set. In the current pipeline setup this means reprocessing everything from digi on (including recon). We will look closer at disk space issues before we do this.

Plus a booboo: this code was inadvertently included in the live pipeline code:

The patch allows selecting geometry files other than the "flight" one, based on the number of towers present. These are necessary because configurations with 10, 12, or 14 towers have "voids" in the grid. Fixes http://jira.slac.stanford.edu/browse/SVAC-60.

Code Versions

Engineering Model (sim/recon) v5r0608p2

System Tests for this version

System Tests Result

FRED version

0.98

Pipeline tag

v1r0p2

GRITS tag (web browsing and task configuration)

glast-ground v0r3p7
grits-gino-web version 0.55 (v0r5p5)
grits-gino version 0.95 (v0r9p5)
grits-gino-xml version 1.42 (v1r4p2)
grits-common version 0.32 (v0r3p2)

online/svac (task defs, scripts):

pipeline tasks:

online: v2r1p2

svac pipeline code and tasks:

code/tasks v3r1p18

ISOC code and tasks:

v0r5p0

Apps that run in pipeline:

eLog: v2r2p6
ConfigTables: v3r1p4
TestReport: v3r2p7 (digi & recon reports)
EngineeringModelRoot: v1r3p19(SVAC tuple) **changed**

Approval: 27 June: Bill, Richard, Steve. Eduardo out of town.