Skip to content

CI CD for Backstage

Jump to bottom
Chloe edited this page Jun 11, 2024 · 2 revisions

Introduction

This document outlines the Continuous Integration (CI) and Continuous Deployment (CD) workflows we use in our project. These workflows help us automate the process from code submission to deployment, allowing our project to meet code quality standards and to be reliably deployed.

CI Workflows

Workflow Overview

The CI workflow is triggered by any push or pull request to any branch within the repository. It makes sure that the code meets our standards for quality and security before it is ever merged into the main branch.

CI Workflow Visual Diagram

+-------------+     +------------------+     +----------------+     +-----------------+
|             |     |                  |     |                |     |                 |
| Push/PR ────┼───► | Setup and Test   ├────►| Validate       |────►| Validate Branch |
|             |     |                  |     | Commit Messages|     | Names           |
+-------------+     +------------------+     +----------------+     +-----------------+

Workflow Steps

  1. Setup and Test
  • Environment: Runs on latest Ubuntu env.
  • Node Versions: Tests are run across multiple Node.js versions (18.x, 20.x) to make sure they are compatible.
  • Process Involed:
    • Checkout Code: Checks out the code for the push or pull request.
    • Node.js Setup: Configures the specified Node.js versions using a matrix strategy to test across multiple versions.
    • Dependency Installation: Installs all necessary dependencies with Yarn so the build can proceed.
    • Build and Test Execution: Executes the build and run all defined tests so no breaking changes are introduced.
  1. Validate Commit Messages
  • Purpose: Checks that all commit messages meet the standard, allowing for better readability and easier tracking. This act as a secondary safeguard besides the Git hooks.
  • Steps Involved:
    • Checkout Code: Ensures the full git history is available for checks.
    • Commit Log Review: Fetches commit messages from the history and validates each against our predefined pattern (ADD:, FIX:, DEL:, UPD:). If a commit message does not follow this pattern, the workflow fails, preventing merging.
  1. Validate Branch Names
  • Purpose: Enforces naming conventions for branches to maintain a clean and organized repository. This act as a secondary safeguard besides the Git hooks.
  • Steps Involved:
    • Determine Branch Name: Extracts the branch name from the pull request.
    • Naming Convention Check: Validates the branch name against the allowed patterns (main, feature/, hotfix/). If a branch name does not comply, the workflow fails to prevent further actions on an incorrectly named branch.

Security and Best Practices

  • Secure Workflow Execution: All the jobs are run on isolated virtual environments to minimize risks.
  • Code Quality Assurance: Integrated testing and linting (commented out for now, but will be enabled in a future version) to maintain higher code quality.
  • Branch and Commit Message Hygiene: Enforces documentation and our team naming conventions for all contributions, allowing for easier code reviews and history tracking.

CD Workflows

Workflow Overview

The CD workflow is triggered on pushes to the main branch or via workflow_dispatch. It manages the deployment of the application to production environments, making sure that our latest and most stable versions are always available to the students.

CD Workflow Visual Diagram

+------------+     +-------------------+     +-----------------+
|            |     |                   |     |                 |
| Push ──────┼───► | Create & Push     ├────►| Deploy Image    |
|            |     | Docker Image      |     | on Cloud Run    |
+------------+     +-------------------+     +-----------------+

Workflow Steps

  1. Create and Push Docker Image
  • Permissions: Requires an id-token with write access and contents with read access.
  • Environment: Runs on latest Ubuntu env.
  • Process Involed:
    • Checkout: Checks out the latest code from the main branch.
    • Setup QEMU: Prepares the QEMU environment on the runner for cross-platform build support using Docker actions.
    • Setup Docker Buildx: Configures Docker Buildx for a better build performance.
    • Google Cloud Authentication: Authenticates with Google Cloud using a service account to gain appropriate permissions for subsequent actions.
    • Login to Google Artifact Registry (GAR): Logs into GAR using an access token, allowing Docker to push the built image.
    • Build and Push Docker Image: Builds the Docker image and pushes it to GAR. The image is tagged with the SHA of the commit triggering the workflow, ensuring immutability and traceability of the deployed image.
  1. Deploy Image on Cloud Run
  • Dependencies: This job needs the successful completion of the previous create-and-push-imagejob.
  • Permissions: Similar to the first job, requires id-token and contents access.
  • Environment: Runs on latest Ubuntu env.
  • Process Involed:
    • Checkout: Checks out the source code.
    • Google Cloud Authentication: Similar to the previous job, authenticates with Google Cloud.
    • Deploy to Cloud Run: Deploys the Docker image from GAR to Cloud Run, configuring the service with necessary environmental variables and connecting to Cloud SQL instances. This step uses Google's deploy-cloudrun GitHub action.

Environment Variables and Secrets

  • POSTGRES_PASSWORD: The password for the Postgres database.
  • GITHUB_TOKEN: Used for operations that require GitHub authentication.
  • GOOGLE_CLIENT_ID and GOOGLE_CLIENT_SECRET: Credentials for OAuth authentication with Google services.

Security and Best Practices

  • Secure Handling of Secrets: All of our sensitive credentials are handled as secrets and are not hardcoded in the workflow files, allowing them to adhere to best practices for cloud security.
  • Immutable Tags for Docker Images: Since images are tagged with commit SHA, each deployment is traceable and immutable, so risks associated with mutable tags are reduced.

Hosting Situation

Our Backstage application is currently hosted on Google Cloud run, which gives us a chance to deploy securely directly from container images, and allows for scalability. Here's an overview how our CD integrates with GCR:

Deployment Workflow Overview

  1. Image Creation and Registry Storage:
    • The Docker image is built and pushed to Google Artifact Registry (GAR) every time changes are pushed to the main branch.
    • The image is then tagged with the Git commit SHA, so that each version is uniquely identifiable and traceable.
  2. Deployment to Cloud Run
    • Once the image is available in GAR, it's then deployed to GCR.
    • GCR can then scale the application depending on the traffic, so resources and performance are used accordingly.
    • Env variables and connections to other GC services like Cloud SQL are configured at this stage as well.

Security and Performance

  • Managed Authentication: We use Google's Workload Identity to securely connect GitHub Actions with Google Cloud services, so all our deployment process can be secure without exposing any sensitive credentials.
  • Environment Isolation: With GCR each deployment is isolated in its own environment, so the risk of conflicts and ensuring stability is lower.

Future Considerations and Current Challenges

While GCR meets our current MVP needs, we are aware of certain instability issues with the deployment workflow which occasionally fails. Addressing this instability is a high priority for the next stages of this project.

Clone this wiki locally