Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: respect GITHUB_TOKEN to download artifacts from GHCR #7580

Merged
merged 5 commits into from
Dec 4, 2024

Conversation

knqyf263
Copy link
Collaborator

@knqyf263 knqyf263 commented Sep 24, 2024

Description

The troubleshooting documentation currently suggests setting the GITHUB_TOKEN to avoid rate limiting. However, this information was outdated, created when the image subcommand didn't even exist. We've identified that in the latest version of Trivy, the GITHUB_TOKEN is not correctly utilized when downloading the vulnerability database, checks bundles and the Java database. The token helps pull these assets from private repositories. Note that it currently doesn't help with rate limiting.

This PR addresses the issue by implementing the correct usage of GITHUB_TOKEN for GHCR.

Notes

The docker login ghcr.io functionality is working as expected and remains unchanged.

Checklist

  • I've read the guidelines for contributing to this repository.
  • I've followed the conventions in the PR title.
  • I've added tests that prove my fix is effective or that my feature works.
  • I've updated the documentation with the relevant information (if needed).
  • I've added usage information (if the PR introduces new options)
  • I've included a "before" and "after" example to the description (if the PR is a user interface change).

Signed-off-by: knqyf263 <knqyf263@gmail.com>
@knqyf263 knqyf263 self-assigned this Sep 24, 2024
Signed-off-by: knqyf263 <knqyf263@gmail.com>
@knqyf263
Copy link
Collaborator Author

knqyf263 commented Sep 24, 2024

Even GITHUB_TOKEN doesn't seem to help. I locally faced the same error with my token.

$ GITHUB_TOKEN=$(gh auth token) mage test:fixtureContainerImages
{unset gho_XXXXXXXXXXXXXXXXXXXXXXXXX}
Error: GET https://ghcr.io/v2/aquasecurity/trivy-test-images/tags/list?n=1000: TOOMANYREQUESTS: retry-after: 71.346µs, allowed: 44000/minute

@schnatterer
Copy link

schnatterer commented Sep 30, 2024

Does docker login work? How is it used within trivy? Does trivy read
~/.docker/config.json?
I am asking because I'd like to use it from within the trivy container where there is no docker. Can I just mount
~/.docker/config.json:/root/.docker/config.json?

@knqyf263
Copy link
Collaborator Author

First of all, GITHUB_TOKEN doesn't seem to help for GHCR at the moment.

If you still want to try a token, you can just mount config.json as you said. If you use an OS-specific credential store, it will probably not work.

e.g.

$ jq .credsStore ~/.docker/config.json
"osxkeychain"

@schnatterer
Copy link

I tried to test it, but now I get

docker login ...
docker pull ghcr.io/aquasecurity/trivy:0.55.2
Error response from daemon: toomanyrequests: retry-after: 516.474µs, allowed: 44000/minute

So maybe it's not the rate limit on the user but a repo-specific limit on the ghcr.io/aquasecurity/trivy repo?
I added this theory to your discussion at https://github.com/orgs/community/discussions/139074.

Copy link

This PR is stale because it has been labeled with inactivity.

@github-actions github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and will be auto-closed. label Nov 30, 2024
Signed-off-by: knqyf263 <knqyf263@gmail.com>
GITHUB_TOKEN didn't help pull images from GHCR

Signed-off-by: knqyf263 <knqyf263@gmail.com>
Signed-off-by: knqyf263 <knqyf263@gmail.com>
@knqyf263 knqyf263 marked this pull request as ready for review December 4, 2024 07:16
@knqyf263 knqyf263 removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and will be auto-closed. label Dec 4, 2024
@knqyf263 knqyf263 requested a review from DmitriyLewen December 4, 2024 07:16
@knqyf263
Copy link
Collaborator Author

knqyf263 commented Dec 4, 2024

@DmitriyLewen We found that this fix does not help with rate limiting, but it does help with downloads from private repositories, so I modified it and made it ready for review.

Copy link
Contributor

@DmitriyLewen DmitriyLewen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@knqyf263 knqyf263 added this pull request to the merge queue Dec 4, 2024
Merged via the queue into aquasecurity:main with commit 21b68e1 Dec 4, 2024
18 checks passed
@knqyf263 knqyf263 deleted the fix/github_token branch December 4, 2024 08:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants