Consolidate multi-arch docker builds #7902

ddelange · 2022-12-15T07:04:35Z

Move docker builds from release.yaml to docker-builds.yaml
Delete republish-images.yaml:
- there are no longer single-arch artifacts uploaded.
- re-trigger by flipping the this is a prerelease checkbox on the Github Release

Example

One commit on top of #7901, diff: ddelange/prefect@docker-builds-release...docker-builds-consolidation

Steps 7-8 ref #7740 (comment) for multi arch docker images, closes #5388

Checklist

This pull request references any related issue by including "closes <link to issue>"
- If no issue exists and your change is not a small fix, please create an issue first.
This pull request includes tests or only affects documentation.
This pull request includes a label categorizing the change e.g. fix, feature, enhancement

netlify · 2022-12-15T07:04:44Z

✅ Deploy Preview for prefect-orion ready!

Built without sensitive environment variables

Name	Link
🔨 Latest commit	`b246223`
🔍 Latest deploy log	https://app.netlify.com/sites/prefect-orion/deploys/63d164a3ee32640008ad1b50
😎 Deploy Preview	https://deploy-preview-7902--prefect-orion.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

…r-builds-release * 'main' of https://github.com/ddelange/prefect: (131 commits) Fix docker-builds.yaml templating syntax (PrefectHQ#8114) Rename Worker pools -> work pools (PrefectHQ#8107) Build multi-arch images for commits on main (PrefectHQ#7900) [Issue PrefectHQ#456] Change example of `infra_override` in docs/concepts/deployment (PrefectHQ#8101) Bump @playwright/test from 1.29.1 to 1.29.2 in /orion-ui (PrefectHQ#8105) Bump @prefecthq/orion-design from 1.1.53 to 1.1.54 in /orion-ui (PrefectHQ#8104) Update deployment docs to include tag and idempotency key (PrefectHQ#7771) Add `BaseWorker` and `ProcessWorker` (PrefectHQ#7996) Add Peyton and Serina as global code owners (PrefectHQ#8098) Add release notes for 2.7.7 (PrefectHQ#8091) Add youtube badge (PrefectHQ#8089) Adds `MAX_RRULE_LENGTH` (PrefectHQ#7762) Limit task run cache key size (PrefectHQ#7275) Add --match flag to work queues documentation (PrefectHQ#7768) Modify disable ssl setting tests to allow any for headers and timeout (PrefectHQ#8086) Add test for allow_failure and quote (PrefectHQ#8055) Adds `experimental_field` decorator (PrefectHQ#8066) add docs on migrating block documents (PrefectHQ#8085) Add Redoc documentation for REST API reference (PrefectHQ#7503) Allow disabling SSL verification (PrefectHQ#7850) ...

…refect into docker-builds-consolidation * 'docker-builds-release' of https://github.com/ddelange/prefect: (131 commits) Fix docker-builds.yaml templating syntax (PrefectHQ#8114) Rename Worker pools -> work pools (PrefectHQ#8107) Build multi-arch images for commits on main (PrefectHQ#7900) [Issue PrefectHQ#456] Change example of `infra_override` in docs/concepts/deployment (PrefectHQ#8101) Bump @playwright/test from 1.29.1 to 1.29.2 in /orion-ui (PrefectHQ#8105) Bump @prefecthq/orion-design from 1.1.53 to 1.1.54 in /orion-ui (PrefectHQ#8104) Update deployment docs to include tag and idempotency key (PrefectHQ#7771) Add `BaseWorker` and `ProcessWorker` (PrefectHQ#7996) Add Peyton and Serina as global code owners (PrefectHQ#8098) Add release notes for 2.7.7 (PrefectHQ#8091) Add youtube badge (PrefectHQ#8089) Adds `MAX_RRULE_LENGTH` (PrefectHQ#7762) Limit task run cache key size (PrefectHQ#7275) Add --match flag to work queues documentation (PrefectHQ#7768) Modify disable ssl setting tests to allow any for headers and timeout (PrefectHQ#8086) Add test for allow_failure and quote (PrefectHQ#8055) Adds `experimental_field` decorator (PrefectHQ#8066) add docs on migrating block documents (PrefectHQ#8085) Add Redoc documentation for REST API reference (PrefectHQ#7503) Allow disabling SSL verification (PrefectHQ#7850) ...

…er-builds-consolidation * 'main' of https://github.com/prefecthq/prefect: (34 commits) Minor API docstring formatting fixes, mostly lists (PrefectHQ#8196) Build multi-arch images for releases (PrefectHQ#7901) feat: Adds "head" flag to `flow-run logs` (PrefectHQ#8003) Updates to Automations and Notifications documentation (PrefectHQ#8140) Bump @prefecthq/vue-compositions from 1.2.3 to 1.2.9 in /orion-ui (PrefectHQ#8175) Bump eslint from 8.31.0 to 8.32.0 in /orion-ui (PrefectHQ#8174) Update Postgres tests to use Docker instead of GHA services (PrefectHQ#8125) Update `black` and `isort` versions in pre-commit (PrefectHQ#8167) Chore: Bump orion and prefect design to 1.2.0 (PrefectHQ#8163) Mark `test_process_streams_stderr` as flaky on Python 3.11 (PrefectHQ#8161) Add release notes for 2.7.8 (PrefectHQ#8160) Add some host_config arguments to DockerContainer (PrefectHQ#8033) Skip flaky test `test_dependent_events_in_two_loops_do_not_deadlock` (PrefectHQ#8159) Update jsonschema requirement from <4.0.0,>=3.2.0 to >=3.2.0,<5.0.0 (PrefectHQ#8152) Fix detection of conda installation in test (PrefectHQ#8149) Add `httpx.WriteError` to client retryable exceptions (PrefectHQ#8145) Add documentation for cache refreshes (PrefectHQ#8156) Add YouTube video to welcome page (PrefectHQ#8090) Add social links (PrefectHQ#8088) bump orion-design, release flow run timeline (PrefectHQ#8153) ...

zanieb · 2023-01-18T17:22:59Z

Hm, there are some warnings on the latest merge: https://github.com/PrefectHQ/prefect/actions/runs/3950856517

ddelange · 2023-01-18T18:01:03Z

its this conditional that will cause empty output of that step (no prod tags on commit push to main). didnt know that throws a warning, it can be ignored...

zanieb · 2023-01-19T22:37:51Z

We've got an example run from a release now: https://github.com/PrefectHQ/prefect/actions/runs/3962751282/jobs/6789813811

ddelange · 2023-01-20T04:52:35Z

nice, the build command in L143 LGTM!

just fyi since recent versions of the action, there is also --provenance being pushed unless disabled, an interesting feature that might break some dev/custom setups ref docker/buildx#1509 (comment) especially ones that use docker manifest commands (my comment further down).

I don't think it causes trouble for docker pull though, otherwise you'd probably already have seen a bug report of some sort.

zanieb · 2023-01-25T17:05:23Z

.github/workflows/release.yaml

-          python -m pip install --upgrade pip
-          pip install --upgrade --upgrade-strategy eager -e .[dev]
+          python -m pip install -U pip setuptools wheel
+          pip install -U -e .[dev]


Can you revert these changes? I don't think they belong in this pull request.

zanieb · 2023-01-25T17:06:36Z

.github/workflows/release.yaml

-        run: git merge origin/main --no-ff -Xtheirs -m "Merge branch 'main' into temporary docs branch"
+        run: git merge origin/main --no-ff -m "Merge branch 'main' into temporary docs branch"


Can you revert this change? This merge strategy is needed and the change doesn't belong here anyway.

ouch something went wrong with merge conflicts. on it

No problem! Thanks :)

zanieb

Let's give it a go :) Thanks for your hard work on this!

…erm-testing * 'main' of https://github.com/prefecthq/prefect: (77 commits) Update roles and permissions in documentation (PrefectHQ#8263) Add Prefect Cloud Quickstart tutorial (PrefectHQ#8227) Remove needless log Update comment for consistency Reorder migrations for clarity Refactor cancellation cleanup service Uses canonical `CANCELLING` states for run cancellations (PrefectHQ#8245) Add cancellation cleanup service (PrefectHQ#8128) Improve engine shutdown handling of SIGTERM (PrefectHQ#8127) Create a `CANCELLING` state type (PrefectHQ#7794) Update KubernetesJob options (PrefectHQ#8261) Small work pools UI updates (PrefectHQ#8257) Removes migration logic (PrefectHQ#8255) Consolidate multi-arch docker builds (PrefectHQ#7902) Include nested `pydantic.BaseModel` secret fields in blocks' schema (PrefectHQ#8246) Improve contributing documentation with venv instructions (PrefectHQ#8247) Update Python tests to use a single test matrix for both databases (PrefectHQ#8171) Adds migration logic for work pools (PrefectHQ#8214) Add `project_urls` to `setup.py` (PrefectHQ#8224) Add `is_schedule_active` to client `Deployment` class (PrefectHQ#7430) ...

ddelange requested a review from zanieb as a code owner December 15, 2022 07:04

ddelange force-pushed the docker-builds-consolidation branch from e8f887f to 2994f03 Compare December 15, 2022 07:17

ddelange changed the title ~~Docker builds consolidation~~ Consolidate multi-arch docker builds Dec 15, 2022

Build multi-arch images for commits on main

2db6388

ddelange force-pushed the docker-builds-consolidation branch from 697b309 to 049373a Compare December 15, 2022 11:45

ddelange added 3 commits December 16, 2022 06:53

Deprecate CACHE_IMAGE

157dd9b

Build multi-arch images for releases

54c49bd

Remove docker builds from release.yaml

52c0107

ddelange force-pushed the docker-builds-consolidation branch from 049373a to 52c0107 Compare December 16, 2022 05:56

ddelange added 2 commits January 9, 2023 20:48

ddelange requested review from serinamarie and peytonrunyan as code owners January 9, 2023 19:50

ddelange mentioned this pull request Jan 13, 2023

Build multi-arch images for releases #7901

Merged

3 tasks

ddelange requested a review from a team as a code owner January 18, 2023 17:17

Delete merge leftover

c3a473e

Merge branch 'main' into docker-builds-consolidation

ba52250

zanieb reviewed Jan 25, 2023

View reviewed changes

zanieb added the enhancement An improvement of an existing feature label Jan 25, 2023

ddelange added 3 commits January 25, 2023 18:15

Merge branch 'main' into docker-builds-consolidation

ca52102

Fix merge remnants

7e6b9b7

Make diff cleaner

b246223

ddelange removed the request for review from serinamarie January 25, 2023 17:19

ddelange requested review from zanieb and removed request for peytonrunyan January 25, 2023 17:19

zanieb approved these changes Jan 25, 2023

View reviewed changes

zanieb merged commit ee49d48 into PrefectHQ:main Jan 25, 2023

github-actions bot pushed a commit that referenced this pull request Jan 26, 2023

Consolidate multi-arch docker builds (#7902)

64e974b

github-actions bot pushed a commit that referenced this pull request Jan 26, 2023

Consolidate multi-arch docker builds (#7902)

e7a3d24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Consolidate multi-arch docker builds #7902

Consolidate multi-arch docker builds #7902

ddelange commented Dec 15, 2022 •

edited by zanieb

Loading

netlify bot commented Dec 15, 2022 •

edited

Loading

zanieb commented Jan 18, 2023

ddelange commented Jan 18, 2023 •

edited

Loading

zanieb commented Jan 19, 2023

ddelange commented Jan 20, 2023 •

edited

Loading

zanieb Jan 25, 2023

zanieb Jan 25, 2023

ddelange Jan 25, 2023

zanieb Jan 25, 2023

zanieb left a comment

		run: git merge origin/main --no-ff -Xtheirs -m "Merge branch 'main' into temporary docs branch"
		run: git merge origin/main --no-ff -m "Merge branch 'main' into temporary docs branch"

Consolidate multi-arch docker builds #7902

Consolidate multi-arch docker builds #7902

Conversation

ddelange commented Dec 15, 2022 • edited by zanieb Loading

Example

Checklist

netlify bot commented Dec 15, 2022 • edited Loading

✅ Deploy Preview for prefect-orion ready!

zanieb commented Jan 18, 2023

ddelange commented Jan 18, 2023 • edited Loading

zanieb commented Jan 19, 2023

ddelange commented Jan 20, 2023 • edited Loading

zanieb Jan 25, 2023

Choose a reason for hiding this comment

zanieb Jan 25, 2023

Choose a reason for hiding this comment

ddelange Jan 25, 2023

Choose a reason for hiding this comment

zanieb Jan 25, 2023

Choose a reason for hiding this comment

zanieb left a comment

Choose a reason for hiding this comment

ddelange commented Dec 15, 2022 •

edited by zanieb

Loading

netlify bot commented Dec 15, 2022 •

edited

Loading

ddelange commented Jan 18, 2023 •

edited

Loading

ddelange commented Jan 20, 2023 •

edited

Loading