Add Workflow for Multi-Architecture Docker Image Build, Push, and Manifest Management

[Bcoderx6]

This PR introduces a comprehensive GitHub Actions workflow to build, push, and manage Docker images for multiple architectures, ensuring broad platform support and seamless integration. Key features include:

Trigger Conditions:
Executes on push and pull_request events for branches (master, develop, and feature/**) and tags matching the pattern v*...
Implements concurrency to prevent overlapping runs for the same workflow or pull request.
Workflow Steps:

Build Stage:
Leverages QEMU and BuildX to enable cross-platform builds.
Constructs Docker images for eight platforms, including linux/amd64, linux/arm64, and others.
Tags images with platform-specific suffixes.

Push Stage:
Pushes the platform-specific images to Docker Hub.
Manifest Management:

Creates and pushes multi-architecture manifest lists for key tags (deps, dev, runtime, cli).
Includes support for version-specific tags derived from GitHub tag references.
Maintains a latest tag pointing to the CLI image.

Testing Stage:
Validates the CLI image by creating a test Dockerfile that runs a basic script using metacallcli.
Tests across all supported architectures.

Cleanup Stage:
Removes platform-specific tags from Docker Hub after manifest creation to minimize clutter.
Environment Variables:

Centralizes key variables like Docker registry, username, image name, and BuildKit version for easier management.
Secrets Management:

Securely uses DOCKER_HUB_USERNAME and DOCKER_HUB_ACCESS_TOKEN for authentication.

Notes:
This workflow ensures compatibility across a wide range of platforms, facilitates efficient Docker image management, and automates testing and cleanup. It lays the foundation for scalable and robust multi-architecture Docker builds.

@viferga

[viferga]

@Bcoderx6 I gave some reviews, please have a look.

[viferga]

Docker username is a secret that must be provided by GitHub secrets.

[viferga]

I think all those environment variables are already set in the script.

[viferga]

Can you transform all those commands in a for using the matrix.platform as an array?

[viferga]

Same here, can't you get this directly from matrix.platform?

[Bcoderx6]

@viferga, I’ve fixed those issues. Please have a look.

[viferga]

I was also thinking about something like this:

name: Build and Push Docker Image for Multiple Architectures

on:
  pull_request:
  push:
    branches:
      - master
      - develop
      - multiArchitectureFeature
    tags:
      - 'v*.*.*'

concurrency:
  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
  cancel-in-progress: true

env:
  DOCKER_REGISTRY: docker.io
  DOCKER_USERNAME: mryash
  IMAGE_NAME: core
  BUILDKIT_VERSION: 0.13.0

jobs:
  pre-build:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.platforms.outputs.result }}
    steps:
    - uses: actions/github-script@v6
      id: platforms
      with:
        result-encoding: string
        script: |
          // List of platforms
          const platform = [
            "linux/amd64",
            "linux/arm64",
            "linux/riscv64",
            "linux/ppc64le",
            "linux/s390x",
            "linux/386",
            "linux/arm/v7",
            "linux/arm/v6",
            // "linux/mips64le",
            // "linux/mips64",
          ];
          const docker = platforms.map(p => p.replaceAll('/', '-'));

          return JSON.stringify({ platform, docker });
  build:
    name: Build
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        platform: ${{ fromJson(needs.pre-build.outputs.matrix).platform }}

    steps:
      - name: Checkout Repository
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3

      - name: Set up Docker BuildX
        uses: docker/setup-buildx-action@v3
        with:
          version: v${{ env.BUILDKIT_VERSION }}

      - name: Login to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKER_HUB_USERNAME }}
          password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}

      - name: Build MetaCall Docker Images
        env:
          METACALL_PLATFORM: ${{ matrix.platform }}
        run: |
          ./docker-compose.sh platform

      - name: Tag Platform Images
        run: |
          platform_tag=$(echo "${{ matrix.platform }}" | tr '/' '-')
          echo "Platform Tag: ${platform_tag}"
          for tag in "deps" "dev" "runtime" "cli"; do
            docker tag ${DOCKER_USERNAME}/${IMAGE_NAME}:${tag} \
              ${DOCKER_REGISTRY}/${DOCKER_USERNAME}/${IMAGE_NAME}:${tag}-${platform_tag}
          done

      - name: Create Test Dockerfile
        run: |
          cat <<EOF > Dockerfile
          FROM ${DOCKER_USERNAME}/${IMAGE_NAME}:cli
          RUN echo "console.log('abcde')" > script.js
          RUN metacallcli script.js
          EOF

      - name: Build and Test Image
        run: |
          set -exuo pipefail
          platform_tag=$(echo "${{ matrix.platform }}" | tr '/' '-')
          docker build --platform ${{ matrix.platform }} -t test-image .
          docker run --rm --platform=${{ matrix.platform }} test-image | grep "abcde"

      - name: Push Platform Images
        run: |
          platform_tag=$(echo "${{ matrix.platform }}" | tr '/' '-')
          for tag in "deps" "dev" "runtime" "cli"; do
            echo "Pushing image for tag: ${tag} with platform: ${platform_tag}"
            docker push ${DOCKER_REGISTRY}/${DOCKER_USERNAME}/${IMAGE_NAME}:${tag}-${platform_tag}
          done

  manifest:
    name: Create and Push Manifest Lists
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Login to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKER_HUB_USERNAME }}
          password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}

      - name: Create and Push Manifest Lists
        run: |
          for tag in "deps" "dev" "runtime" "cli"; do
            echo "Creating manifest for tag: $tag"
            platform_tags=""
            platforms=("linux-amd64" "linux-arm64" "linux-riscv64" "linux-ppc64le" "linux-s390x" "linux-386" "linux-arm-v7" "linux-arm-v6")
            for platform in "${platforms[@]}"; do
              platform_tags="${platform_tags} ${DOCKER_REGISTRY}/${DOCKER_USERNAME}/${IMAGE_NAME}:${tag}-${platform}"
            done
            docker manifest create ${DOCKER_REGISTRY}/${DOCKER_USERNAME}/${IMAGE_NAME}:${tag} ${platform_tags} --amend
            docker manifest push ${DOCKER_REGISTRY}/${DOCKER_USERNAME}/${IMAGE_NAME}:${tag}
          done

      - name: Create Version Specific Tags
        if: startsWith(github.ref, 'refs/tags/')
        run: |
          VERSION=${GITHUB_REF#refs/tags/v}
          tags=("deps" "dev" "runtime" "cli")
          platforms=("linux-amd64" "linux-arm64" "linux-riscv64" "linux-ppc64le" "linux-s390x" "linux-386" "linux-arm-v7" "linux-arm-v6")
          for tag in "${tags[@]}"; do
            platform_tags=""
            for platform in "${platforms[@]}"; do
              platform_tags="${platform_tags} ${DOCKER_REGISTRY}/${DOCKER_USERNAME}/${IMAGE_NAME}:${tag}-${platform}"
            done
            docker manifest create ${DOCKER_REGISTRY}/${DOCKER_USERNAME}/${IMAGE_NAME}:${VERSION}-${tag} ${platform_tags} --amend
            docker manifest push ${DOCKER_REGISTRY}/${DOCKER_USERNAME}/${IMAGE_NAME}:${VERSION}-${tag}
          done

          # Create and push the 'latest' tag for CLI
          cli_platform_tags=""
          for platform in ${{ matrix.platform }}; do
            cli_platform_tags="${cli_platform_tags} ${DOCKER_USERNAME}/${IMAGE_NAME}:cli-${platform}"
          done
          docker manifest create ${DOCKER_REGISTRY}/${DOCKER_USERNAME}/${IMAGE_NAME}:latest ${cli_platform_tags} --amend
          docker manifest push ${DOCKER_REGISTRY}/${DOCKER_USERNAME}/${IMAGE_NAME}:latest

  cleanup:
    name: Cleanup Platform Specific Tags
    needs: [build, manifest]
    runs-on: ubuntu-latest
    if: always()
    steps:
      - name: Remove Platform-Specific Tags
        run: |
          platforms=("linux-amd64" "linux-arm64" "linux-riscv64" "linux-ppc64le" "linux-s390x" "linux-386" "linux-arm-v7" "linux-arm-v6")
          tags=("deps" "dev" "runtime" "cli")

          for platform in "${platforms[@]}"; do
            for tag in "${tags[@]}"; do
              tag_to_delete="${tag}-${platform}"
              echo "Deleting tag: ${tag_to_delete}"

              curl -X DELETE \
                -H "Authorization: Bearer ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}" \
                "https://hub.docker.com/v2/repositories/${DOCKER_USERNAME}/${IMAGE_NAME}/tags/${tag_to_delete}/"
            done
          done

Then we can replace all the:

platforms=("linux-amd64" "linux-arm64" "linux-riscv64" "linux-ppc64le" "linux-s390x" "linux-386" "linux-arm-v7" "linux-arm-v6")

By something like:

${{ fromJson(needs.pre-build.outputs.matrix).docker }}

But we need to do .join('" "') on it.