The Problem
Way back when, there were two ROOT collection classes: TObjArray and TClonesArray. TObjArrays had to be used for those objects that may allocate memory, as some of our MC and Recon objects do. TClonesArray was more efficient, but reserved for those objects of fixed length. And so..we used a mixture of TObjArrays and TClonesArrays.
Three or so years ago, we found that the processing speed when reading/writing ROOT was unacceptably slow. This problem was traced to our use of TObjArrays and their inefficient deletion mechanism. The fix that was utilized mimiced the deletion algorithm used by TClonesArray where objects ready for deletion are collected and deleted in bulk in one shot rather than deleting smaller collections of objects for every event. The details of our algorithm involve large fixed length arrays to store the objects marked for deletion. The code has the known weakness that if we overflow the fixed-length array..a crash would ensue..so we made these arrays large.
Over the years we have run into examples where runs have crashed (albeit rarely) due to an overflow of one of these arrays. Here is an example:
FluxSvc INFO 40% complete: event 400, time=launch+ 10.9414
!!!Warning: limit for posHits is greater than 150000
Then crash!
Our quick fix, has been to increase the size of our fixed length arrays to allow for runs to carry on. Not pretty but keeps us moving along.
A Proposal
Meanwhile, TClonesArrays have evolved tsuch that they can contain objects that allocate memory. Without this restriction, we could modify our ROOT structures to use TClonesArrays in all cases and ditch TObjArrays once and for all. This affects MC, Digi, and Recon...so changes would involve mcRootData, digiRootData, reconRootData.
Then the question is do we require backward compatibility such that current ROOT files need to be readable by the new versions of the xxxRootData packages? Possible, just a bit more messy..