Versions Compared

Old Version 3

changes.mady.by.user Nisperos, Patrick

Saved on

compared with

New Version 4

changes.mady.by.user Nisperos, Patrick

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Runs on hardware in s3df

    Code Block
    Allocatable:
  cpu:                64
  ephemeral-storage:  152933498761
  hugepages-1Gi:      0
  hugepages-2Mi:      2816Mi
  memory:             259679512Ki
  pods:               220
System Info:
  Machine ID:                 92faa81e90af4e65ba73d3007e42519e
  System UUID:                ce9ba000-5727-11ed-8000-3cecefd8e38e
  Boot ID:                    96386228-b4ab-4836-b764-b22d4dfc0cda
  Kernel Version:             4.18.0-372.32.1.el8_6.x86_64
  OS Image:                   Red Hat Enterprise Linux 8.6 (Ootpa)
  Operating System:           linux
  Architecture:               amd64
  Container Runtime Version:  containerd://1.6.31
  Kubelet Version:            v1.28.8
  Kube-Proxy Version:         v1.28.8
  2. ad-build-dev is used for build system development, while ad-build is the production build system for users.

How to access

ad-build-dev cluster: https://k8s.slac.stanford.edu/ad-build-dev

...

  1. Get the build system container running on the kluster, see if you can use the actions/actions-runner-controller: Kubernetes controller for GitHub Actions self-hosted runners
    1. Lets do it vanilla first (running build system container) 
      1. Create the image using base image: Package actions-runner (github.com)
        1. TODO: push the docker image to a registry so anyone can pull it
          1. From where the dockerfile is 
          2. 'docker build --tag pnispero/gh-runner-image:latest .'
          3. This step may change (make a docker account, then create a access token, which will allow you to login on your shell)
          4. 'docker push pnispero/gh-runner-image:latest'
          5. Output: pnispero/gh-runner-image - Docker Image | Docker Hub

        2. Dockerfile (Here temporarily, these are the only 2 files you need to get this to work)

          Code Block
          # base
FROM ubuntu:22.04

# set the github runner version
ARG RUNNER_VERSION="2.316.0"

# update the base packages and add a non-sudo user
RUN apt-get update -y && apt-get upgrade -y && useradd -m docker

# install python and the packages the your code depends on along with jq so we can parse JSON
# add additional packages as necessary
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
    curl jq build-essential libssl-dev libffi-dev python3 python3-venv python3-dev python3-pip

# cd into the user directory, download and unzip the github actions runner
RUN cd /home/docker && mkdir actions-runner && cd actions-runner \
    && curl -O -L https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz \
    && tar xzf ./actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz

# install some additional dependencies
RUN chown -R docker ~docker && /home/docker/actions-runner/bin/installdependencies.sh

# copy over the start.sh script
COPY start.sh start.sh

# make the script executable
RUN chmod +x start.sh

# since the config and run script for actions are not allowed to be run by root,
# set the user to "docker" so all subsequent commands are run as the docker user
USER docker

# set the entrypoint to the start.sh script
ENTRYPOINT ["./start.sh"]

          start.sh

          Code Block
          #!/bin/bash

ORGANIZATION=$ORGANIZATION
ACCESS_TOKEN=$ACCESS_TOKEN

# Generate organization registration token
REG_TOKEN=$(curl -L \
  -X POST \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer ${ACCESS_TOKEN}" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  https://api.github.com/orgs/${ORGANIZATION}/actions/runners/registration-token | jq .token --raw-output)

cd /home/docker/actions-runner

./config.sh --url https://github.com/${ORGANIZATION} --token ${REG_TOKEN}

cleanup() {
    echo "Removing runner..."
    ./config.sh remove --unattended --token ${REG_TOKEN}
}

trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM

./run.sh & wait $!
      2. do 'docker image ls' to ensure its there
      3. Then you must be an organization administrator, and make a personal access token with the "admin:org" and "repo" scope to create a registration token for an organization (REST API endpoints for self-hosted runners - GitHub Docs)
      4. Copy the token, and use it in the next step

      5. Run the docker image

        Code Block
        docker run \
  --env ORGANIZATION=<ORG> \
  --env ACCESS_TOKEN=<PERSONAL-TOKEN> \
  --name runner1 \
  runner-image

        Replace <ORG> with the organization name
        Replace <PERSONAL-TOKEN> with the token you created above

      6. And now your runner should be registered and running
      7. When done testing make sure to 'ctrl+c' and  'stop' and 'remove' the container
    2. Start the image using kubectl for our ad-build kubernetes cluster you created above
      1. For now, use a local registry
      2. Code Block
        # Start the image with environment variables
kubectl run gh-runner1
         kubectl create deployment runner1 
        --image=pnispero/gh-runner-image --env="ORGANIZATION=<ORG>" --env="ACCESS_TOKEN=<PERSONAL-TOKEN>"

        Replace <ORG> with the organization name
        Replace <PERSONAL-TOKEN> with the token you created above

  2. Then we can use that for building buildroot. One of the workflows will be it checking out on /scratch/ in s3df, then build, and output results there.
    Resource (Slightly altered its files): How to containerize a GitHub Actions self-hosted runner | by Alessandro Baccini | Medium

...