Status
Script tagCollector.py has been committed to CVS (grits-tools/python/tagCollector.py). This page has been updated for version 1.16. The only difference between 1.16 and 1.14 was to add version information of the script itself to the output and the only difference between 1.14 and 1.13 was a bug fix. There have been no interface changes since 1.13.
The most recent version of the tagCollector.py script should be used. On SLAC linux, the most recent version is maintained at
/afs/slac.stanford.edu/g/glast/ground/scripts/tagCollector.py
Interface changes from previous versions
Version 1.13 accepts the value NONE for the --parent option. In this case, --new=HEAD is not allowed, but you can specify a positive integer, e.g. --new=123 in order to create a tag for HEAD 123.
Version 1.11 has the additional flag --fake. The options --real and --verbose are now also true flags, taking no argument.
Functionality
Currently handles these activities:
- Make a new release tag which is identical to current HEAD build
- Make a new HEAD, not based on any previous HEAD.
- Make a new HEAD by changing some package tags, adding new packages or removing packages as compared to current HEAD
There must be a file packageList.txt, listing all packages and versions belonging to the build at the top level (e.g. ScienceTools-scons/packageList.txt). It is used to determine what gets the new tag in cases 1. and 2. above. For 3., tagCollector.py will update the packageList.txt belonging to the old HEAD appropriately .
At some point would like to add ability to make a new tag along a branch by updating, adding or removing packages relative to an existing tag on that branch.
Call Interface, Usage
Here is the help output:
jrb@noric11 $ python tagCollector.py --help
Usage: tagCollector.py [options] container, e.g.
python tagCollector.py --new=HEAD ScienceTools
python tagCollector.py --new=HEAD --real ScienceTools
python tagCollector.py --parent=NONE --new=35 GlastRelease
python tagCollector.py --new=ScienceTools-09-24-00 ScienceTools
CVS Version: $Id: tagCollector.py,v 1.16 2011/07/07 00:56:18 jrb Exp $
See also https://confluence.slac.stanford.edu/x/GgFyBQ
Options:
-h, --help show this help message and exit
-p PARENT, --parent=PARENT
HEAD or NONE. Someday also (branched) release tag to
act as base [default: HEAD]
-n NEWTAG, --new=NEWTAG
Specify tag to be created: HEAD (interpreted as next
available HEAD), positive integer (number of HEAD), or
literal tag name. Required option
-u UPGRADEFILE, --upgrade_list=UPGRADEFILE
File listing packages to upgrade, add or remove; used
only when creating a new HEAD [default: upgrade.txt]
-r, --real Really do it [default: not set]
-f, --fake Just print what we *would* do [default: yes]
-v, --verbose If set show all cvs command output [default: not set]
In this version PARENT can only be HEAD on NONE.
Here is a sample upgrade file:
# Upgrade for ScienceTools // Comments can start with a '#' as above or with a slash u dataSubselector dataSubselector 08-02-01 u tip tip-02-17-01 # One can also add or remove packages from HEAD, e.g. # a myNewPackage myNewPackage-00-00-01 # r obsoletePackage
and here is part of the output when it was used as the value for upgrade_list.
jrb@noric11 $ python tagCollector.py --new=HEAD --fake ScienceTools
tagCollector.py was invoked with argument ScienceTools and options
parent= HEAD
new= HEAD
upgrade_list= upgrade.txt
for_real= False
verbose= False
CVS version: $Id: tagCollector.py,v 1.16 2011/07/07 00:56:18 jrb Exp $
about to issue command
cvs -d /nfs/slac/g/glast/ground/cvs rlog -h ScienceTools-scons/SConstruct
lastHeadTag is: ScienceTools-HEAD-1-896
about to issue command
cvs -d /nfs/slac/g/glast/ground/cvs rlog -h ScienceTools-scons/packageList.txt
about to issue command
cvs -d /nfs/slac/g/glast/ground/cvs rlog -h ScienceTools-scons/packageList.txt
about to issue command
cvs -d /nfs/slac/g/glast/ground/cvs co -r ScienceTools-HEAD-1-896 ScienceTools-scons/packageList.txt
Found changed package: tip
Found changed package: dataSubselector
about to issue command
cvs -d /nfs/slac/g/glast/ground/cvs update -A ScienceTools-scons/packageList.txt
about to issue command
cvs -d /nfs/slac/g/glast/ground/cvs commit -m "tagCollector.py updating pkg list"
ScienceTools-scons/packageList.txt
Just fooling; it wasn't for real
about to issue command
cvs -d /nfs/slac/g/glast/ground/cvs rlog -h ScienceTools-scons/containerNotes.txt
No containerNotes.txt file found for ScienceTools-scons
about to issue command
cvs -d /nfs/slac/g/glast/ground/cvs rtag ScienceTools-HEAD-1-897 ScienceTools-scons/ChangeLog
Just fooling; it wasn't for real
about to issue command
cvs -d /nfs/slac/g/glast/ground/cvs rtag ScienceTools-HEAD-1-897 ScienceTools-scons/packageList.txt
Just fooling; it wasn't for real
about to issue command
cvs -d /nfs/slac/g/glast/ground/cvs rtag -r xmlBase-05-05-10 ScienceTools-HEAD-1-897 xmlBase
Just fooling; it wasn't for real
about to issue command
cvs -d /nfs/slac/g/glast/ground/cvs rtag -r astro-03-11-11 ScienceTools-HEAD-1-897 astro
Just fooling; it wasn't for real
?(followed by many similar lines tagging all the individual packages comprising ScienceTools.
The last one is always SConsFiles:)
about to issue command
cvs -d /nfs/slac/g/glast/ground/cvs rtag -r SConsFiles-00-12-01 ScienceTools-HEAD-1-897 SConsFiles
Just fooling; it wasn't for real
If the option --real had been used, the cvs commands appearing in the output all would have been executed. Since it was False for this run, only the read-only cvs commands, like the initial cvs rlog, were executed.
The overall strategy in the make-new-HEAD-from-previous case shown above is
- check out version of packageList.txt belonging to parent tag
- merge changes requested in upgrade file into packageList.txt; commit new packageList.txt
- tag top-level files (packageList.txt, ChangeLog, containerNotes.txt) belonging to no subpackage with new HEAD tag
- tag each package listed in packageList.txt with new HEAD tag, saving SConsFiles for last
The function to make a release tag from HEAD is similar except that no new packageList.txt file needs to be generated. Just use the one from the parent tag as is. For the case --parent=NONE, use the most recent packageList.txt
packageList.txt
This file lists all packages comprising the container, somewhat analogous to the CMT requirements file for the container package. It is not normallly edited by hand.
Interface change
As of version 1.10 of tagCollector.py, this file is a required part of every container package.
<cvs-path-to-package> <tag>
e.g.,
flux flux-08-40-07
The file may also contain blank lines or comments (any line beginning with #), which are ignored.
The file is not used as input to the tagging operation, but it is updated to reflect the changes specified in the user's upgrade file, thereby providing a record of exactly which tags are in each head build or release tag.
containerNotes.txt
If the container has a file containerNotes.txt at the top level, as of version 1.9 (committed to CVS March 10, 2011) the tagCollector script will give it some special handling for HEAD tags: the most recently committed version of containerNotes.txt will get the new HEAD tag. (The file isn't required; if it doesn't exist, tagCollector will put out an informational message but will keep going.) The intent is to give the owner of the container a place to document the contents of the new HEAD. There are no constraints on the file, but it is probably most useful as a log (new information is appended or inserted; old entries are left alone) containing at a mininum the name or user id of whoever is making the new tag and a timestamp. When creating release tags, tagCollector treats containerNotes.txt just like everything else: everything with the most recent HEAD tag will also get the new release tag.
Still To Do
- test adequately Creating next HEAD tag has been tested for Toy-scons DONE Testing deemed adequate as of March 10, 2011.
- add as output a file which lists all the packages and their tags belonging to the new tag just created. The file probably should go at the top level (e.g., in the ScienceTools-scons directory). One possibility in the new-HEAD case is to create it after the temp. tag has been made, submit to CVS, move temp tag to the new version, then make the final new HEAD tag. However, if we plan to also support tag collecting from RMViewer, it would have to go through a similar procedure. DONE The file is called packageList.txt.
- add ability to make release tags along a branch
Sticky Points
The way CVS tagging is currently set up (in particular, the script $CVSROOT/CVSROOT/tagger), you would have to be logged in as glast or glastrm to run the script in "for real" mode.
Proposal: Modify tagger script to allow creation and deletion of tmp tags with names satisfying a pattern. Should be something like HEAD pattern except replace HEAD with string HEADtmp, possibly HEADtmp<stuff> where <stuff> is an arbitrary and optional alphanumeric string if, e.g., we want to distinguish different attempts of creating HEAD of a particular version. DONE but, as of 1.11, temp tags are no longer used.
Certain assumptions are hard-coded in the script when, ideally, they should come from the RM database: for example, exact format of tags. These things don't change very fast, if at all, but if they do, someone will have to remember to update the script.