...
My preference is for some form of 2. 1. doesn't attempt to address the root of the problem. 3. involves possibly significant changes to RM operation (but I defer to Tom as to whether this is a real concern).
LATEST tags appear to be much less subject to problems than HEAD. I suspect this is because the LATEST tags are made bit by bit. At the very least, the SConsFiles part of each container, in particular the SConstruct file, should receive its ( HEAD or release ) tag last to minimize contention. (RM is watching SConstruct to determine when there is a new tag to be built. When it finds one, it submits several jobs which will do check-outs. Check-outs require a read lock; tagging requires a write lock.)
...