...
The current structure of these containers is not suitable for GitHub because certain files (SConstruct, allExternals.scons, ..) are shared by all containers via sym links into their root directories. These files need to be moved into a subpackage which can be treated like other subpackages. Such a scheme is under development with the new subpackage SConsShared.
Syncing repositories
Because of the container reorganization needed to use SConsShared I don't believe there is any hope of moving development for all containers to GitHub initially; it will have to be done piecemeal. Neither will modifying ReleaseManager (or substitute) to work from GitHub happen instantaneously. That means we will have to live with the "same" source in both GitHub and CVS. We need reliable syncing mechanisms for code and tags.
(Aug. 29, 2017) UPDATE: an informal poll reveals that developers are willing to tolerate an outage of weeks in order to get this done "all at once". This means we don't have to put effort into figuring out how to reliably keep git and CVS in sync.
Physical structure
Much of the organization and all the sharing is handled in the CVS repository by means of sym links. There are top-level directories ScienceTools-scons and GlastRelease-scons. There are also top-level directories for subpackages such as astro, Likelihood, etc. In almost all cases the subpackage directories are sym-linked into the container directory. Shared subpackages like astro and celestialSources are sym-linked into both. The one exception is containerSettings. Each container has its own version of this subpackage, which is a regular subdirectory of the container directory. When you check a container out of CVS, sym links are followed, so you end up with all source files in regular subdirectories.
...
- Put subsubpackages in their own repositories in Git; use some existing tool, like subtrees or subrepos, to keep track of the "real" structure
- Put subsubpackages in their own repositories in Git; use homegrown scripting to bridge the gap between git structure and CVS structure
Eliminate subsubpackages by changing organization in CVS and updating references as needed (but we still need an existing or custom homegrown tool which handles the two levels)
Git or third-party "Sub" tools
...
Both subtree and subrepo appear to handle branches reasonably well. In the more recent subtree doc the description of how to push changes to the subtree back to the remote is a bit daunting, especially since this is likely to be a common operation for developers. This was one of the motivations for subrepo subrepo doesn't have this issue, in fact fixing it was one - perhaps the chief - motivation for writing it.
I don't see any discussion of tags in the documentation of any of these tools but, since we've traditionally used home-grown tools rather than naked CVS commands for tagging already (C++ program stag for individual subpackages; tagCollector.py for release and release candidate tags), perhaps this isn't so important.
My tentative conclusion is that we can eliminate submodule and subtree from consideration; subrepo should be investigated further. In particular we should see how it does with a three-level hierarchy.
See also this discussion of reorganization and beginning experiments with subrepro.
Scripting bridge
Each subpackage or subsubpackage (that is, anything with an SConscript in its top directory) goes in a separate repository, as with previous strategy, but we write our own custom tool to help with the scenarios we expect to encounter, and not much more.
...