At EED we have two k8s cluster accel-webapp-dev and accel-webapp respectively for testing and production deployment. In this article we are going to describe how CD/CD has been implemented to achieve CI/CD (Continuous Integration and Continuous Delivering). each one environment is identified by his hostname accel-webapp-dev.slac.stanford.edu and accel-webapp.slac.stanford.edu.
Overview
This document outlines provides a detailed description of the continuous integration (CI) and continuous deployment pipeline for Project A. The pipeline is designed to manage deployments from development through to production using Kubernetes clusters, with human intervention required for promotion to the production environment(CD) pipeline applied across all projects. Our pipeline orchestrates the workflow from development to production, utilizing Kubernetes clusters to ensure smooth transitions and scalability. We've established a process that necessitates manual approval for production deployments, enhancing control and stability. The structure comprises two distinct Git repositories for each application: the Development Project for active development and the Deployment Project for managing deployment processes.
Pipeline Stages
All the The deployment workflow is composed by three stages segmented into three primary phases: Development, Pre-Production, and Production.
Development Stage
Within the Development Project, the development phase adheres to the trunk-based development model, maintaining the main branch as the most stable and release-ready version of the code. This approach emphasizes short-lived branches for small features, which are then merged into the main branch post-verification, thereby streamlining the merging and integration steps. This strategy is instrumental in achieving CI/CD objectives, consequently enhancing software delivery and organizational performance.
To support this methodology, two pipelines have been instituted:
Merge Request Pipeline
Executed within the Development Project, this pipeline automates the following tasks:
- Compile: Compilation of the codebase.
- Test: Execution of automated tests to validate code quality and functionality. This includes initializing the required backend services using a Docker Compose architecture.
- Merge: Automatic merging of the merge request (MR) by the system upon successful completion of the prior steps.
Main Branch Update Pipeline
Triggered by updates to the main branch within the Development Project, this pipeline performs the following:
- Compile: Code compilation.
- Test: Automated testing to verify code quality and functionality, which involves spinning up backend services using Docker Compose.
- Build Docker Image: Creation of a Docker image from the compiled and tested code.
- Notify Deployment Pipeline: Invocation of a subsequent pipeline within the Deployment Project to manage deployment readiness.
| Info |
|---|
By maintaining distinct pipelines for merge requests and main branch updates, we ensure rigorous validation at every stage, thereby preserving the integrity and deployability of the |
Deployment Stage
the deployment stage lives within the deployment project that is a git repository that host both the test and production kubernetes configuration. This design permit to have different authorization from who develop the application and who needs to manage the deployment- **Source**: The source code resides in `Project repository` hosted on GitHub/GitLab, this i where the developer push change request for updating software functionality, the "main branch is portected" force the creation of merge request, that will be merged when all check will be verified with static error checking and dynamic error checking give by the unit tests.
- **Compile**: The code is compiled in this stage.
- **Test**: Automated tests are run to ensure code quality and functionality.
- **Build Docker Image**: A Docker image is created from the compiled code.
- **Promote Pre**: The image is then tagged for the pre-production stage.
Pre-Production Stage
- **Automated Deployment**: Using Argo CD, the Docker image is automatically deployed to the Kubernetes (K8s) Pre-Production Cluster (accel-webapp-dev.slac.stanford.edu).
- **Components**:
- **ProjectA Artifact**: Deployed application artifact.
- **MongoDB**: The pre-production database instance.
- **MariaDB**: Another pre-production database instance.
- **Argo CD**: The continuous deployment tool that synchronizes the cluster state with the repository state.
Production Stage
- **Wait Promotion**: Deployment to production requires human intervention for approval.
- **Promotion**: Once approved, the configuration from the pre-production is promoted to the production Git repository.
- **Automated Deployment**: Argo CD automatically deploys the promoted configuration to the Kubernetes (K8s) Production Cluster (accel-webapp.slac.stanford.edu).
- **Components**:
- **ProjectA Artifact**: Deployed application artifact in production.
- **MongoDB**: The production database instance.
- **MariaDB**: Another production database instance.
- **Argo CD**: Manages the deployment and ensures the production cluster state is in sync with the production repository.
...