Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: generate releases.json on release event #1563

Merged
merged 2 commits into from
Jan 30, 2023

Conversation

crazy-max
Copy link
Member

@crazy-max crazy-max commented Jan 28, 2023

We are currently using the GitHub API in our setup-buildx-action to check for latest and tagged releases to make sure they exist before download. But this requires using a token to avoid rate-limit. It's fine for public runners but GHES runners don't have the github.token populated automatically (see docker/setup-buildx-action#194). They need to create a PAT.

This PR will solve this issue by generating and pushing a releases.json file in this repo when we publish a GitHub Release, see https://github.com/docker/buildx/actions/runs/4033493181/jobs/6934050707#step:3:5

This file will then be fetched through raw.githubusercontent.com endpoint on setup-buildx-action repo. This endpoint is better served for our purpose with 5000 requests per hour compared to the GitHub API endpoint that is limited to 60 requests per hour (unauth) and 1000 request per hour when authenticated.

Also ignore .github/releases.json file on pull request event as an action in a workflow run can't trigger a new workflow run anyway. See https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#example-using-more-than-one-event

We could also generate this file within the setup-buildx-action repo if we don't want that in Buildx repo. Let me know what you think about it.

Copy link
Collaborator

@jedevc jedevc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This SGTM - since this is the only GitHub API we use as part of the action, it makes sense to access the actual release data in the same way as we access the releases themselves. There is going to always be some inconsistency here though - e.g. when tagging a new release, the binaries for that release can't be in the JSON file in the commit we tag (since I think the artifact URL is unpredictable?). That feels kind of annoying, so we should work out how we manage the file - maybe we should only allow modifying it on the master branch (and verify this through CI)?

We could maybe bring this into our actions utility tools when we do that? It seems generic enough that we might also want to use it in some of our other actions, e.g. for buildkit.

Slightly frustrating that this change was made right before the weekend 😢

@crazy-max
Copy link
Member Author

There is going to always be some inconsistency here though - e.g. when tagging a new release, the binaries for that release can't be in the JSON file in the commit we tag (since I think the artifact URL is unpredictable?). That feels kind of annoying, so we should work out how we manage the file - maybe we should only allow modifying it on the master branch (and verify this through CI)?

This will be fine. Currently we are creating a draft GitHub Release:

So the workflow updating the releases.json will not be triggered until we publish the release. Will look like this: master...crazy-max:buildx:ci-releases-json

@crazy-max crazy-max marked this pull request as draft January 28, 2023 18:18
@crazy-max crazy-max changed the title hack: generate releases.json from GitHub Releases ci: generate releases.json on release event Jan 28, 2023
@crazy-max crazy-max force-pushed the releases-json branch 2 times, most recently from 2648807 to 130da48 Compare January 28, 2023 20:34
@crazy-max
Copy link
Member Author

Updated to remove the script, bake target and new Dockerfile so we only use GitHub Actions to generate the releases JSON file.

with:
base: master
branch: releases-json/${{ github.event.release.name }}
commit-message: "update .github/releases.json"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
commit-message: "update .github/releases.json"
commit-message: "github: update .github/releases.json"

Or some other prefix, for consistency.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

github prefix sounds good


jobs:
generate:
uses: crazy-max/.github/.github/workflows/releases-json.yml@002654044825b3c2b9856af61b8a2aaf389706b1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, good to have something reusable 😅

Tiny nit - could we potentially have this produce indented JSON, it would be useful if it was human-readable.

Copy link
Member Author

@crazy-max crazy-max Jan 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tiny nit - could we potentially have this produce indented JSON, it would be useful if it was human-readable.

Yes might be better, done.

Relevant commit: crazy-max/.github@2a596c9

We are currently using the GitHub API in our setup-buildx-action
to check for latest and tagged releases to make sure they exist
before download. But this requires using a token to avoid
rate-limit. It's fine for public runners but GHES runners don't
have the `github.token` populated automatically. They need to
create a PAT.

This PR will solve this issue by generating and pushing a
`releases.json` file in this repo when we publish a GitHub Release
that will then be fetched through `raw.githubusercontent.com`
endpoint on `setup-buildx-action` repo. This endpoint is better
served for our purpose with 5000 requests per hour compared to the
GitHub API endpoint that is limited to 60 requests per hour (unauth)
and 1000 request per hour when authenticated.

Also ignore .github/releases.json file on pull request event as an
action in a workflow run can't trigger a new workflow run anyway.
See https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#example-using-more-than-one-event

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants