Use local rekor and fulcio instances in e2e tests #3478
Conversation
| @@ -62,6 +79,8 @@ docker run -d -p 5000:5000 --restart always -e REGISTRY_STORAGE_DELETE_ENABLED=t | |||
| export COSIGN_TEST_REPO=localhost:5000 | |||
| go test -tags=e2e -v ./test/... -run TestSignVerifyClean | |||
|
|
|||
| # Use the public instance to verify existing images and manifests | |||
| unset SIGSTORE_REKOR_PUBLIC_KEY | |||
There was a problem hiding this comment.
Thoughts on keeping this? Should we sign local images in a private registry in order to test the below verify commands?
There was a problem hiding this comment.
Ideally this test would be entirely hermetic, would definitely improve performance, though we can investigate that as a follow up. The signature verification should be offline, though this test depends on ghcr and distroless maintaining signatures on its images.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #3478 +/- ##
==========================================
+ Coverage 39.57% 40.07% +0.50%
==========================================
Files 155 155
Lines 10040 10040
==========================================
+ Hits 3973 4024 +51
+ Misses 5600 5530 -70
- Partials 467 486 +19 ☔ View full report in Codecov by Sentry. |
- Use -v for clearer output - All the tests tagged 'e2e' are in ./test, no need to search the whole tree - There's no third_party directory, no need to grep it out - Clean up services no matter when the script exits Signed-off-by: Colleen Murphy <colleenmurphy@google.com>
Clean up the key pair secret after the test so that the test suite can be re-run locally if desired. Without this, the secret is set to 'immutable' and can't be overwritten by the next test run. Signed-off-by: Colleen Murphy <colleenmurphy@google.com>
In 7068357 the e2e tests moved from running on the locally-spun-up rekor instance to the public instance. This means test signatures are piling up in the public instance, and the tests may be taking longer than they need to since they are using an external service. This change moves back to using the local rekor instance, which the e2e has still been spinning up even though it has been going unused. Also now do the same for fulcio. Signed-off-by: Colleen Murphy <colleenmurphy@google.com>
| fi | ||
| done | ||
|
|
||
| echo "starting services" | ||
| export FULCIO_METRICS_PORT=2113 | ||
| for repo in rekor fulcio; do |
There was a problem hiding this comment.
Using a local instance of Fulcio is great - We don't have any tests currently using it, and we'll need to set up a mock oidc provider, but this will let us have much more rich e2e tests that exercise the certifciate issuance path
| @@ -62,6 +79,8 @@ docker run -d -p 5000:5000 --restart always -e REGISTRY_STORAGE_DELETE_ENABLED=t | |||
| export COSIGN_TEST_REPO=localhost:5000 | |||
| go test -tags=e2e -v ./test/... -run TestSignVerifyClean | |||
|
|
|||
| # Use the public instance to verify existing images and manifests | |||
| unset SIGSTORE_REKOR_PUBLIC_KEY | |||
There was a problem hiding this comment.
Ideally this test would be entirely hermetic, would definitely improve performance, though we can investigate that as a follow up. The signature verification should be offline, though this test depends on ghcr and distroless maintaining signatures on its images.
In 7068357 the e2e tests moved from running on the locally-spun-up
rekor instance to the public instance. This means test signatures are
piling up in the public instance, and the tests may be taking longer
than they need to since they are using an external service.
This change moves back to using the local rekor instance, which the e2e
has still been spinning up even though it has been going unused. Also
now do the same for fulcio.
This PR also includes minor cleanups in the e2e tests and test script.
Depends on sigstore/fulcio#1518
Relates to sigstore/sigstore-probers#105
Summary
Release Note
NONE
Documentation
Test changes only.