Version History

Definitions

GitHub Organization

Analysis of proposals

We've analyzed different ideas:

• Single organization for all the code created at SLAC:
  • We tried this and soon we started to face problems with naming conflicts.
  • To solve this we would need to modify the name of the repositories appending something that would group them. For example, all EPICS support modules would have "epics-" prepended. This way asyn would become epics-asyn.
  • The naming change was not ideal because we would like to fork an external module and have the name coincide with the upstream code.

Given that slaclab has hundreds of repositories, a naming conflict was very probable. We decided that we needed some level of repository grouping without creating too many repositories.

One idea of grouping would be to have one GitHub organization per SLAC directorate. This idea was abandoned because:

• It creates a virtual wall, psychologically discouraging collaboration between directorates.
• It breaks related repos into different organizations. For example, EPICS modules would be spread in different GitHub organizations.
• It difficults transfer of ownership if this ever happens.

For managing repositories in a per-directorate basis, a best approach would be to use GitHub teams.

Current proposal

• slac-epics
  • Name is still undecided. Other options: slac-epics-support, slac-epics-infra
  • Components that do not belong in slac-epics-apps, but depend on the EPICS build system and/or EPICS base
  • Examples:
    • EPICS base
    • EPICS modules
    • EDM
    • ALH
    • eco_tools
    • Striptool
    • Gateway
    • eget services
• slac-epics-apps
  • EPICS applications of all kinds, usually they will be specific to a part of the lab rather than shared
  • Examples:
    • EPICS IOCs from all directorates, like EED and LCLS.
    • EPICS HLAs
    • PyDM and EDM screens
    • Matlab code that uses EPICS
  • NOTE (5/1/24): Likely want to merge this with slac-epics
• slaclab
  • Everything else.

Pros and cons for this model:

• Pros
  • Separate from slaclab org creating less name contention.
  • Only people who need access will be added to each organization.
  • Already used for our EPICS software (currently mirrors, though).
  • Easy to find related code because the code is grouped in an organization with an easily identifiable name.
• Cons/Problems to Solve
  • An additional organization to manage; teams need to be mirrored between organizations.
    • Seems to be solved by GHE.
  • Yet another organization to keep track of
    • We should collect links somewhere, maybe in an organization-wide README or GitHub hosted documentation for quick access.

Exceptions to consider

Open source projects that take on a life of their own outside of SLAC may want to consider being under a dedicated GitHub organization. A good example of this is the archiver appliance, which can be found here: https://github.com/archiver-appliance

These organizations are intended to "interface" with the community outside of SLAC, allowing community contributors or maintainers to be brought in without needing to worry about slaclab/slac-epics permissions.
Even though these organizations are outside of slaclab, they could still be brought under the SLAC GitHub enterprise umbrella. This would allow them to use the additional resources afforded under our plan.

Here are some examples of SLAC open source projects that receive outside collaboration and/or have their own dedicated organization:

• EPICS archiver: has its own organization - https://github.com/archiver-appliance. This organization has 8 repositories.
• PyDM and NALMS could fit in slac-epics (or slac-epics-infra, slac-epics-support), but their location in slaclab is already used by many outside SLAC.

GitHub Teams

The most obvious approach for organizing teams in GitHub would be to use its hierarchical team configuration to mimic what we have at SLAC. Using the embedded group in TID as an example:

This way, repositories maintained by a specific group can easily be configured to the correspondent team.

The easiest way to configure this model is to have the group leaders add their group members to the correct team in GitHub. This would share the effort among several people, reducing the load.

Another model that can use GitHub Teams is creating teams per system. Examples:

• Timing team: responsible for everything that is related to the timing system.
• EPICS team: responsible for everything that is related to EPICS.

Both models can coexist as permission to individual repositories can receive multiple teams and individuals.

A questions for the next meeting:

• Ownership matters, but how much?
  • GitHub is a platform designed for open source projects, our ownership model should reduce any barriers to contribution as much as possible.
• Do we need to mirror the "bureaucratic" hierarchy of SLAC, or should we organize on a per-project basis? (i.e. LCLS-II, LCLS-II HE, etc.)
• Organization admin issues.
  • Admins can see everything, even code that may fall under NDA or other restrictions.
  • Some admins no longer work at SLAC, need to keep a better eye on the admin list.

GitHub Topics

As GitHub doesn't allow the distribution of repositories in a hierarchy like file systems do, one way to ease the search is by the use of Topics. Topics are like labels that can be set in each repository. A repository can have multiple Topics.

Once this is set, if you are interested in LLRF, for example, you would search by the LLRF Topic and see only the repositories related to that Topic.

Topics cross organizations, so having more than one organization doesn't impact this search mechanism. For example, checking the rtems topic returns https://github.com/topics/rtems. slaclab is one organization that shows up in the search results, but there are others.

At this moment GitHub allows for searching a Topic in one organization or all organizations available in GitHub. There's no way to configure a search for a group of organizations. To improve the success in searches we could prepend "slac-" to all our Topics, like slac-timing, slac-atca, slac-llrf, etc. This way we ensure that a broad search in GitHub would bring repositories related only to organizations related to SLAC.

GitHub Issues

Currently we have 2 ticket systems in use for software development/bug tracking: CATER and Jira. GitHub brings its own ticket system called Issues.

CATER won't go away for a long time. So, what do we do regarding Jira and GitHub issues? The use cases could be:

1. CATER is kept as it is used today. No changes.
2. GitHub issues for SLAC maintained repositories that have external collaborators.
3. GitHub issues for tickets clearly related to one single repository.
4. Jira for tickets that cross multiple repositories or that are unrelated to work for any repository. Could GitHub Projects be used for this, instead?

Do we want to keep track of tickets in 3 different tools?

NOTE (5/1/24): Jerry K. indicated that EED is looking to move away from Jira. Other groups that have a heavier dependence on Jira may not want to move away.
Overall, this seems like a department-specific decision rather than one that can be made for the entire lab.

GitHub Pages and READMEs

If we end up creating multiple different organizations for SLAC-related projects, how do we keep track of things? A potential solution to this issue is to use GitHub organization READMEs and GitHub pages for organization level documentation.

We could create an organization-level README on slaclab that contains links to other SLAC orgs and some basic information. We could also create a documentation page for the entire slaclab org that contains links to and information about relevant GitHub organizations and projects.

The pcdshub organization is a good example of this type of setup:

• https://github.com/pcdshub (Note the README.md at the top of the page)
• https://pcdshub.github.io/ (Note the sidebar with links to the slaclab org)
  • Source code: https://github.com/pcdshub/pcdshub.github.io
  • Provides a summary of some of their projects

GitHub pages simply publishes HTML. They can either be written manually or generated with a software package like Spinx or Hugo.
In the above example, pcdshub.github.io is using Sphinx as the documentation generator and ReStructured Text for the source files.

Licenses

In TID we've been following SLAC's legal request of adding a specific LICENSE file to each repository's top directory, plus a disclamer text in all .c, .cpp., .h, .hpp, .py, .vhd, etc files. There's a Python script that we run that do this automatically: https://github.com/slaclab/surf/blob/pre-release/scripts/apply_slac_license.py. As this comes from SLAC legal, I believe that this would be extended to all code available in SLAC's GitHub organizations.

The problem arises for external code that we fork in our repos. It is very common that the forked code has its own license that we can't modify. TID directors' orientation in this case is that the repository must be made private.

I believe that we need to talk with SLAC legal again to verify more use cases.

NOTE (5/1/24): SLAC legal is probably concerned about licensed code from other sources (i.e. vxWorks), not open source software. It is probably fine to keep open source projects we fork public.

Other aspects to consider

This section covers workflow specific guidelines for the usage of GitHub enterprise. These are not intended to be applied lab wide! Instead, these serve as examples of controls that may be applied on a per-department/group/division basis.

Repository naming

Should we standardize for repository naming or keep each team to define them freely? Use cases:

If we have slac-epics and slac-epics-apps:

• No prefixes for modules, base, or other applications.
• Prefix IOC applications with ioc-?

Standard rulesets

Should the entire SLAC follow the same workflow, with standard names for branches, and standard rules for using each branch? What if different departments have conflicting requirements? 

Settings > Rules > Rulesets

• Mirror TID-ID workflows
  • Git workflow for the Embedded Group
• Match branches:
  • main
  • pre-release
  • Release tags (in the format [0-9]+.[0-9]+.[0-9]+ 
    • NOTE: GitHub uses fnmatch, not regexp for this matching.
• Allow bypass:
  • Maintainers and repository owners
• Restrict creations
• Restrict updates
• Restrict deletions
• Require a pull request before merging
  • Require at least 1 approval before merge
• Require status checks to pass
  • Only if CI is enabled

Future work

The work outlined here is outside of the scope of this document and should probably be done on a per directorate/department/group basis.

Establish Standards for Locally Developed C/C++ Projects

ipmiComm and ek9000 module could be used as the reference implementation for these things.
These two modules should implement most or all of the recommended CI checks and whatnot, and adhere to the standards we define.

• GitHub actions for CI. Verify that everything compiles to avoid broken code being merged.
• Set -Werror, minimize ignored warnings
  • Define standard set of enabled/disabled warnings for C/C++ projects
• Some level of automated testing
  • Testing should be run with a runtime analyzer too
    • asan
    • valgrind
  • Testing may not always be possible due to the nature of EPICS
• Static code analysis with clang-tidy
  • Define standard set of clang-tidy checks
• Code formatting with clang-format
  • Define standard clang-format configuration (ideally one that closely matches our current, most common style)
• pre-commit hook for code formatting
  • Only if using code formatting.
  • Mostly concerned about Python projects here, clang-format can be finicky with pre-commit.

Establish Standards for Documentation

• Generate documentation using:
  • Sphinx (Restructured Text) (any, but mostly Python)
  • Doxygen or clang-doc (C, C++)
  • LaTeX (any)
• Publish documentation to GitHub pages
  • See asyn and friends for a CI setup here: https://github.com/epics-modules/asyn/blob/master/.github/workflows/publish-docs.yml

Document Contribution Workflow on GitHub Pages

• Like ECS does: https://pcdshub.github.io/development.html
• They also briefly describe their various repositories so newbies can figure out where to find what.
• Links in the side bar too.

Standard Labels

• Labels can be applied to issues and pull requests so that they can be filtered.
• Repositories may have custom labels, but you can also define them organization-wide.
• Depending on the workflow for certain software components, different labels may be needed.

Commit Naming Guidelines

• Commit naming guidelines may be helpful to enforce consistent and detailed change descriptions.
• ECS enforces their own commit guidelines. Detailed here: https://pcdshub.github.io/development.html#commit-guidelines.
• Now that slac-epics contains some software that may be from ECS (i.e. eco_tools), should we enforce a controls-wide commit naming scheme?
  • Maybe just describe the model used by a particular software packaged in the README?