Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MDBF-770 - GitHub Reusable Workflows #535

Merged
merged 4 commits into from
Aug 12, 2024

Conversation

RazvanLiviuVarzaru
Copy link
Collaborator

@RazvanLiviuVarzaru RazvanLiviuVarzaru commented Aug 8, 2024

@grooverdan @fauust @vladbogo .
The core idea is to use GitHub reusable workflows to divide image builds into groups of parent workflows.

Based on the "matrix" in bbw_build_container.yml, parent workflows were created to reflect Dockerfile usage. Each workflow monitors changes in the corresponding Dockerfile, allowing for more granular control over which images are built.

A master workflow, "build-workflow-dispatcher," is available to trigger all the child workflows manually or when the template is changed.

Rollout Plan:

  • In bbw_build_container_template.yml, the PUSH to QUAY/GHCR is commented out so we can push this commit and perform regression tests against the current bbw_build_container workflow.
  • All pending contributions to bbw_build_container should be rebased and adapted to this new workflow by either changing the template file or the corresponding parent workflow.
  • Our forks should be synced after this is merged.
  • when we are ready to activate PUSH to registries, we uncomment QUAY/GHCR blocks of code in the template file and disable the old workflow
  • the old workflow should stay disabled for a while in case of a rollback

Pending contributions that should be adapted:

If I'm missing something please let me know.

The core idea is to use GitHub reusable workflows to divide image builds into groups of parent workflows.

Based on the "matrix" in bbw_build_container.yml, parent workflows were created to reflect Dockerfile usage. Each workflow monitors changes in the corresponding Dockerfile, allowing for more granular control over which images are built. A master workflow, "build-workflow-dispatcher," is available to trigger all the child workflows manually or when the template is changed.

Rollout Plan:
- In bbw_build_container_template.yml, the PUSH to QUAY/GHCR is commented out so we can push this commit and perform regression tests against the current bbw_build_container workflow.
- All pending contributions to bbw_build_container should be rebased and adapted to this new workflow by either changing the template file or the corresponding parent workflow.
- Our forks should be synced after this is merged.
- when we are ready to activate PUSH to registries, we uncomment QUAY/GHCR blocks of code in the template file and disable the old workflow
- the old workflow should stay disable for a while in case of a rollback
@RazvanLiviuVarzaru
Copy link
Collaborator Author

RazvanLiviuVarzaru commented Aug 8, 2024

Right now there's a lot of noise in the checks section of this Pull Request.
This is because files are defined for the first time. A similar run will happen when we first merge to DEV.
After that on Pull Request only affected workflows will trigger.

Copy link
Member

@grooverdan grooverdan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to get this done and merged use.

Rebasing existing work on this can occur afterwards.

Could enable GHCR as the first proof to have some images to test from the build, leaving QUAY for afterwards.

In favour of a quick cleanup. Things are in code history for reverting, if they are maintained for possible revert it tends to linger.

@RazvanLiviuVarzaru
Copy link
Collaborator Author

Happy to get this done and merged use.

Rebasing existing work on this can occur afterwards.

Could enable GHCR as the first proof to have some images to test from the build, leaving QUAY for afterwards.

In favour of a quick cleanup. Things are in code history for reverting, if they are maintained for possible revert it tends to linger.

@grooverdan
If I enable GHCR all images will be rebuilt based on the first trigger of "Dispatch all builds.." but I assume that's no problem, since that's our backup system and provides us some insights on the builds as you mentioned. And I want to test that secrets:inherited is really working.

I would keep bbw_build_container.yml for a while, not very much, until we enable QUAY also on these workflows.
So the next pull request will be:

  • remove build_bbw_container.yml
  • enable push on quay for new workflows.

How does it sound?

@RazvanLiviuVarzaru
Copy link
Collaborator Author

RazvanLiviuVarzaru commented Aug 8, 2024

@grooverdan So, I've enabled push to GHCR:
b56e3e4

I will double check everything to see if I have the right images, tags, platforms, and other parameters right.
I would like a review from @vladbogo and @fauust also . This is kind of major change so additional pair of eyes won't hurt.

@fauust If it's ok for you, that OpenSuse/SLES logic for moving files to be available for any container, I would provide it in a different patch as there are already many things to test. --> solved in: acffa3e

- avoid race conditions and building from different commits when both Dockerfiles and the template file are modified.
- define input for the user for the dispatcher workflow
@RazvanLiviuVarzaru
Copy link
Collaborator Author

One commit to address conditions on which dispatcher runs in parallel with other workflows (Triggered by docker files).
666a608

@RazvanLiviuVarzaru
Copy link
Collaborator Author

For the failures reported in checks

@RazvanLiviuVarzaru RazvanLiviuVarzaru merged commit 119dc50 into MariaDB:dev Aug 12, 2024
22 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants