Confluence will be down for maintenance June 14 2024 at 6AM PT.
Version | Date | Description of Changes |
---|---|---|
v0.0 | 4/??/24 | Initial draft work by Marcio and Jeremy |
v0.1 | 5/2/24 | Applied changes from last meeting. Made note of "interface" GitHub organizations, removed lcls-daq from the organization list, noted down additional discussion points about team setup. |
v0.2 | 5/3/24 | Added section about READMEs and GitHub pages. Added note that the "Other aspects to consider" is mostly noting capabilities that we probably want to apply in a more localized way. (we don't want to enforce a certain branching workflow for the whole lab!) |
v0.3 | 5/3/24 | Added section on security, for discussion in next meeting. |
Term | Description |
---|---|
Organization | A location on GitHub where many repositories and teams can be stored. Translates into a URL when browsing or cloning a repository. |
Working Copy | A clone of a Git repository that you can edit and compile. |
Repository | A location where Git history and code are stored. These are added as "remotes" on a local working copy. |
GHE | GitHub Enterprise |
HLA | High-Level Applications |
Fork | A copy of a repository from one organization to another. For example, github.com/slac-epics/asyn would be a fork of github.com/epics-modules/asyn. GitHub keeps track of forks so the upstream code has the link clearly visible. |
Upstream | Repository that is the original basis of a fork. |
We've analyzed different ideas:
Given that slaclab has hundreds of repositories, a naming conflict was very probable. We decided that we needed some level of repository grouping without creating too many repositories.
One idea of grouping would be to have one GitHub organization per SLAC directorate. This idea was abandoned because:
For managing repositories in a per-directorate basis, a best approach would be to use GitHub teams.
Pros and cons for this model:
Open source projects that take on a life of their own outside of SLAC may want to consider being under a dedicated GitHub organization. A good example of this is the archiver appliance, which can be found here: https://github.com/archiver-appliance
These organizations are intended to "interface" with the community outside of SLAC, allowing community contributors or maintainers to be brought in without needing to worry about slaclab/slac-epics permissions.
Even though these organizations are outside of slaclab, they could still be brought under the SLAC GitHub enterprise umbrella. This would allow them to use the additional resources afforded under our plan.
Here are some examples of SLAC open source projects that receive outside collaboration and/or have their own dedicated organization:
The most obvious approach for organizing teams in GitHub would be to use its hierarchical team configuration to mimic what we have at SLAC. Using the embedded group in TID as an example:
This way, repositories maintained by a specific group can easily be configured to the correspondent team.
The easiest way to configure this model is to have the group leaders add their group members to the correct team in GitHub. This would share the effort among several people, reducing the load.
Another model that can use GitHub Teams is creating teams per system. Examples:
Both models can coexist as permission to individual repositories can receive multiple teams and individuals.
A questions for the next meeting:
As GitHub doesn't allow the distribution of repositories in a hierarchy like file systems do, one way to ease the search is by the use of Topics. Topics are like labels that can be set in each repository. A repository can have multiple Topics.
Once this is set, if you are interested in LLRF, for example, you would search by the LLRF Topic and see only the repositories related to that Topic.
Topics cross organizations, so having more than one organization doesn't impact this search mechanism. For example, checking the rtems topic returns https://github.com/topics/rtems. slaclab is one organization that shows up in the search results, but there are others.
At this moment GitHub allows for searching a Topic in one organization or all organizations available in GitHub. There's no way to configure a search for a group of organizations. To improve the success in searches we could prepend "slac-" to all our Topics, like slac-timing, slac-atca, slac-llrf, etc. This way we ensure that a broad search in GitHub would bring repositories related only to organizations related to SLAC.
Currently we have 2 ticket systems in use for software development/bug tracking: CATER and Jira. GitHub brings its own ticket system called Issues.
CATER won't go away for a long time. So, what do we do regarding Jira and GitHub issues? The use cases could be:
Do we want to keep track of tickets in 3 different tools?
NOTE (5/1/24): Jerry K. indicated that EED is looking to move away from Jira. Other groups that have a heavier dependence on Jira may not want to move away.
Overall, this seems like a department-specific decision rather than one that can be made for the entire lab.
If we end up creating multiple different organizations for SLAC-related projects, how do we keep track of things? A potential solution to this issue is to use GitHub organization READMEs and GitHub pages for organization level documentation.
We could create an organization-level README on slaclab that contains links to other SLAC orgs and some basic information. We could also create a documentation page for the entire slaclab org that contains links to and information about relevant GitHub organizations and projects.
The pcdshub organization is a good example of this type of setup:
GitHub pages simply publishes HTML. They can either be written manually or generated with a software package like Spinx or Hugo.
In the above example, pcdshub.github.io is using Sphinx as the documentation generator and ReStructured Text for the source files.
In TID we've been following SLAC's legal request of adding a specific LICENSE file to each repository's top directory, plus a disclamer text in all .c, .cpp., .h, .hpp, .py, .vhd, etc files. There's a Python script that we run that do this automatically: https://github.com/slaclab/surf/blob/pre-release/scripts/apply_slac_license.py. As this comes from SLAC legal, I believe that this would be extended to all code available in SLAC's GitHub organizations.
The problem arises for external code that we fork in our repos. It is very common that the forked code has its own license that we can't modify. TID directors' orientation in this case is that the repository must be made private.
I believe that we need to talk with SLAC legal again to verify more use cases.
NOTE (5/1/24): SLAC legal is probably concerned about licensed code from other sources (i.e. vxWorks), not open source software. It is probably fine to keep open source projects we fork public.
Some thoughts about security:
This section covers workflow specific guidelines for the usage of GitHub enterprise. These are not intended to be applied lab wide! Instead, these serve as examples of controls that may be applied on a per-department/group/division basis.
Should we standardize for repository naming or keep each team to define them freely? Use cases:
Should the entire SLAC follow the same workflow, with standard names for branches, and standard rules for using each branch? What if different departments have conflicting requirements?
Settings > Rules > Rulesets
[0-9]+.[0-9]+.[0-9]+
The work outlined here is outside of the scope of this document and should probably be done on a per directorate/department/group basis.
ipmiComm and ek9000 module could be used as the reference implementation for these things.
These two modules should implement most or all of the recommended CI checks and whatnot, and adhere to the standards we define.