Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: cache trivy-test-images #7598

Closed
knqyf263 opened this issue Sep 26, 2024 · 5 comments · Fixed by #7599
Closed

ci: cache trivy-test-images #7598

knqyf263 opened this issue Sep 26, 2024 · 5 comments · Fixed by #7599
Assignees
Labels
kind/testing Categorizes issue or PR as related to a unit/integration test.
Milestone

Comments

@knqyf263
Copy link
Collaborator

Description

Currently, we pull test images every time we run integration/module/VM tests on GitHub Actions.

img, err := crane.Pull(imgName)

if err = saveLayer(layer, filePath); err != nil {

These images are rarely changed and should be cached on GitHub Actions to mitigate 429 errors.

Downloading spring4shell-jre11...
Downloading spring4shell-jre[8](https://github.com/aquasecurity/trivy/actions/runs/11045901509/job/30685527564#step:5:9)...
Downloading almalinux-8...
Downloading alpine-3[9](https://github.com/aquasecurity/trivy/actions/runs/11045901509/job/30685527564#step:5:10)...
Error: GET https://ghcr.io/v2/aquasecurity/trivy-test-images/blobs/sha256:e7c96db7181be991f19a9fb6975cdbbd73c65f4a2681348e63a141a2192a5f[10](https://github.com/aquasecurity/trivy/actions/runs/11045901509/job/30685527564#step:5:11): TOOMANYREQUESTS: retry-after: 339.865µs, allowed: 44000/minute

cf. https://github.com/aquasecurity/trivy/actions/runs/11045901509/job/30685527564

@knqyf263 knqyf263 added the kind/testing Categorizes issue or PR as related to a unit/integration test. label Sep 26, 2024
@knqyf263 knqyf263 added this to the v0.56.0 milestone Sep 26, 2024
@gerhard
Copy link

gerhard commented Oct 8, 2024

Hi @knqyf263,

We started hitting this issue in our pipeline. What are your thoughts on this workaround?

@DmitriyLewen
Copy link
Contributor

Hello @gerhard
Related discussion - #7538

IIUC you use token from config.json in trivy container.
But it doesn't always help - #7580

@gerhard
Copy link

gerhard commented Oct 9, 2024

That was helpful @DmitriyLewen , thank you.

This comment confirms my initial suspicion of this being a rate limit on the ghcr.io/aquasecurity/trivy repository: #7580 (comment)

I'm thinking that we should go with --db-repository public.ecr.aws/aquasecurity/trivy-db as the short-term fix: https://github.com/orgs/community/discussions/139074#discussioncomment-10808081

@veselov
Copy link

veselov commented Oct 28, 2024

If your image has java stuff in it, also add --java-db-repository public.ecr.aws/aquasecurity/trivy-java-db

@DmitriyLewen
Copy link
Contributor

@veselov
small correction - if you have jar(or other java archives - e.g. war) file.
Because Trivy uses trivy-java-db only for them. (Trivy doesn't use trivy-java-db for pom.xml files).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/testing Categorizes issue or PR as related to a unit/integration test.
Projects
Status: No status
Development

Successfully merging a pull request may close this issue.

4 participants