Page about component & deployment databases

Component Database

Resources

Backend API/CLI to the MongoDB component database: eed-web-application/build-environment-builder (github.com)

API Documentation: Swagger UI (stanford.edu)

Potential Schema

/* new schema for component db v2*/
 
// Note - Not all fields are populated depending on the api endpoint
component_schema
{
    "id": "",                   // Primary Key of json object
    "name": "",                 // Name of component (corresponds to name of repository)
    "organization": "",         // Name of organization
    "description": "",          // Description of component
    "version": "",              // Version of component schema (NOT the repo version)
    "environment": "",          // Build/Runtime environment - ex: ['rhel7', 'rhel8', 'matlab'] which may or may not contain components
    "url": "",                  // URL/filepath to the component object
    "approvalRule": "",         // Determines what kind of approval was done - ex: ['all', 'none'] determined by approval rule
    "approvalGroup": [          // Names of experts on the component (possibly group names in addition to individuals)
      "",
      "",
    ],
    "testingCriteria": "",      // Determines acceptance criteria for testing - ex: ['all', '90%']     
    "dependsOnComponents": [    // Component dependencies
      {"name": "", "tagname": ""},
      {"name": "", "tagname": ""}
    ],
    "buildInstructions": ""     // Determines how to build component - ex: ['make', 'build_component.sh']
                                // May be a simple make, or refer to a script within the component repo
}
// Q: can a component have 2 or more environments (os)?
// note - maybe the environment moves to an additional dependsOnComponents entry - maybe the first entry?
 
/* Example */
{
  "id": "663e591ce77eef11eb66a54f",
  "name": "mps_central_node",
  "organization": "slaclab",
  "description": "central node for machine protection system",
  "version": "1",
  "environment": "rhel7.9",
  "url": "s3df/slaclab/mps_central_node",
  "approvalRule": "all",
  "approvalGroup": [       
      "Jerry Katzung",
      "Patrick Nisperos",
    ],
   "dependsOnComponents": [
    {"name": "slaclab/epics", "tagname": "epics-72"}, 
    {"name": "slaclab/adcore", "tagname": "adcore-45"}
   ],
   "buildInstructions": "make"
}
 

// Note - useful when we want to report to developers where certain branches are
// note - expected to be dynamic
branches_in_development_schema
{
    "id": "",                   // id of entry
    "name": "",                 // name of repo
    "organization": "",         // name of organization
    "branchName": "",           // name of branch
    "approvalStatus": "",       // Status of approval - ex: ['approved', 'rejected', 'in progress']
    "testingStatus": "",        // Status of testing - ex: ['pass', 'fail', 'in progress']
    "overallStatus": ""         // Status at the final state - ex: ['merged', 'rejected', 'deleted']
}
// Q: does testLocation information get tracked in here?
 
 
deployment_schema
{
    "id": "",                   // id of entry
    "name": "",                 // name of repo
    "organization": "",         // name of organization
    "tagName": "",              // name of tag
    "hostId": "",                   // Host hardware id from hardware_host_schema
    "deploymentStatus": ""          // Status at the final state - ex: ['provisional', 'active']
}
// Q: Do we add host name as well for convenience? Or does the gui we use to view database query for us?
// Q: how do we cleanly revert if multiple components are upgraded (like ioc and epics)?  // Note - when in provisional state, during pamm ansible will look at the name/org/tag to install the packages
// Use case: developer updates the deploymentStatus if deployed app is good, or not good
    // if good: then ansible leaves it alone
    // if bad: then engineer reverts tagname to previous tag and marks active, then ansible reverts to previous tags and installs previous package
 
 
hardware_host_schema
{
    "id": "",                   // id of entry
    "name": "",                 // name of hardware
    "location": "",             // location of hardware
    "owner": "",                // owner or POC
    "status": "",               // Status - ex: ['inactive', 'active']
    "specialHardware": [        // Any special hardware besides host machine - ex: ['camera', 'optical interface', TBD]
        "",
        ""
    ]
}

Other Notes

  1. cbs - core build system
  2. docker-compose.yml file is used to create the database locally (for development purposes). Where you would use a 'docker compose up' command to execute

README.md (Since there isn't one on the repo)

How to build:

Disclaimer: I've only tried this on vscode, since the repo already made support to create the development environment for vscode using (devcontainer.json)

  1. Open up the repo in vscode, and you will be prompted to open up the devcontainer.
    1. Click yes, give it a few minutes to setup, it will run docker-compose.yml which will setup the mongodb locally
  2. Now that your development environment is setup, open up a terminal in vscode. Which should open you in the container, at folder '/workspace'
  3. Do a 'make build'

    At the moment there is errors
    1. I solved the vcs status error by
      1. do a git status
      2. follow the command it outputs
      3. Got rid of the error:
    2. I solved the malformed import path "-o": leading dash by
      1. Edit line 8 in makefile 
        Original: go build ./... -o cbs
        New: go build -o cbs ./...
    3. Then it should be able to run
  4. Example

How to run:

TBD

  1. Component list
    1. ./build-environment-builder component -h
    2. Specify a label (endpoint), but seems that no labels are configured. Check configure.go with the "add your map here" I think you specify the yaml
    3. Seems like this only works in vscode dev container, because the server is running there (update: Maybe not actually, since when you do docker ps, you'll see the containers that were started because of vscode (Which runs docker compose up))
      1. I first did a './build-environment-builder configure endpoint --label=dev --url=http://cbs:8080/api/' to create a label (endpoint)
      2. Then when i try to use it when listing components: './build-environment-builder component list --label=dev', it seems to connect to server, but gives error, probably because i have no components in it
      3. tip: do an 'echo=$EDITOR' and if you dont have anything specified, set it to vim. Otherwise, in this next step, when you create a component, it'll open up in nano.
      4. './build-environment-builder component list --label=dev'
      5. Left off here

Deployment Database

TBD



  • No labels