This page is documentation of the github actions runner we plan to use

Possible setup #1 - GitHub enterprise


Possible setup #2 - Self hosted runner

I made a self-hosted runner that runs on s3df on a repository named BuildSystem slaclab/BuildSystem: BuildSystem for EED software development and configuration management (github.com). The actions themselves doesn't do much, since this is a proof of concept.

But also communicates with github, and posts build results there under the 'Action' tab.

  1. Here is setup
      
  1. And then run, and manually trigger the workflow using Github CLI (I installed github CLI on my goenv)

    Result on github repo 'Actions'

To think about: Should we use github CLI to be the base tool for our CLI? It seems to have features we need like triggering a workflow manually. But it also allows you to create your own commands. Maybe we can make a simplified wrapper around the CLI - And make only certain options visible to the end user?

Resources:

Adding self-hosted runners - GitHub Docs

If go this route:

  1. Add the self-hosted runner at the organization level (So every repo just needs to enable their self-hosted runner to the 'buildsystem' so they're able to communicate to the buildsystem).
  2. (optional) configure the self-hosted runner as a service


Possible setup #2.1 - Self hosted runner in a container

  1. This is the same as #2 except it runs in an apptainer container

  2. Run the workflow. (The first x means not completed, then it completes)


  3. The output of the runner is a bit verbose, so its better to look at the results on github actions

This works too but i am not sure if is scalable, because in the definition file (build_system_runner.def), i cd into the actions-runner/ directory which is already preconfigured for 1 runner. If multiple runners are made, then may have to configure all of them.

Resources: 

Provide runner as a Docker Image · Issue #367 · actions/runner (github.com)

Package actions-runner (github.com)

Possible setup #3 - Self hosted runner with jobs in containers

May be the optimal route if we host the build system on our own hardware (or virtual). 

This solves 2 problems:

  1. No bottleneck on the self hosted runner since the runner will just spin up containers to do the jobs, not the runner itself. 
  2. Not having the runner rely on github resources that can potentially cost money if overused. Also not having a self-hosted runner for each project, we can host the runner organization wide (like propose #2) and have every project enable their actions runner to 'build-system'.

Running jobs in a container - GitHub Docs

Creating a Docker container action - GitHub Docs

Possible setup #4 - Self hosted runner autoscaling (kubernetes)

May also be optimal route, but need to look into it. Surface level: Runners are created/deleted based off usage. This route involves kubernetes infrastructure.

Autoscaling with self-hosted runners - GitHub Docs

About Actions Runner Controller - GitHub Docs



Setting up organization wide runner

  1. Made personal organization for sandbox 'ad-build-test'
  2. Added a runner called runner1 in runner group ad-build-test-runner
  3. then i forked the build system repo to ad-build-test
  4. and the actions runner was automatically set to the organization runner
  5. So i clones the repo on s3df, did a gh repo set-default to set the repo to ad-build-test/BuildSystem
  6. then did a gh workflow run to see if it worked and it did.
  7. So i then created another runner called runner2, but ran the same commands, the only difference was i changed the actions-runner folder to actions-runner2



  8. So I ran 2 different workflows (content doesn't matter) And since workflow 'CI' occupied runner1, running workflow 'CI-2' didn't queue on runner1, the job moved to next available runner, since runner2. 

This proves that we can have multiple runners to satisfy users requests

Drawback:

  1. Since repos on slaclab are public, we would have to enable the 'Allow public repositories' on the runner groups. (Which may have a security risk if non-slac users fork the repo and trigger the runners) 
    1. (Update, we can allow public repos, just as long as the runners are only allowed to run within the organization, which is allowed for enterprise - Managing access to self-hosted runners using groups - GitHub Enterprise Cloud Docs) To solve this, in the organization's 'General actions permissions', set the policy to 'Allow <organization_name> actions, and select non-ad-build-test, actions and reusable workflows'. So only people within the organization can access the runners. And can include other actions like default github ones

Workflow Access

  1. In the github organization settings, we can specify 'runner groups'. Where each group can take jobs from any workflow, or certain workflows chosen by administrator. 
    1. This could be useful if we want 'nightly builds' to be their own group, so there will always be runners available in other runner groups. 
    2. You could also add labels to the runners, and then on the workflow job yaml you can specify the label to determine which runner to run on.
    3. Choosing the runner for a job - GitHub Docs


  • No labels