Remove lengthly operations from hermetic tests 🧪 #4590
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tekton-robot
added
release-note-none
tekton-robot
added
the
size/S
|
My plan is to try a couple of things to make sure the test is still actually testing what it should and then I'm thinking we could try this out and see if the flake stops happening. |
Also use Errorf instead of Fatalf between the two tests (the hermetic test and the non-hermetic tests) so that if one fails the other will still run. In tektoncd#4567 we see that the hermetic end to end test sometimes fails, specifically it seems to be the `not-hermetic-run-as-root` version of the test, and it seems like the failure is hitting the 1 minute timeout. Looking at the test, it seems to be doing an `apt-get update` which seems like an operation that would be in grave danger of sometimes taking a while (especially depending on what version of the latest ubuntu image is running) so although I'm not sure that's what is causing the problem, I want to try doing something that is less likely to take so long but still would require network access. I thought maybe that it was also trying to do somethign that required priviledged execution (i.e. running as root) but it seems like that's not something that hermetic mode drops anyway (looking at the TEP it seems to just be scoped to networking) so it doesn't feel like there is actually any need for that.
bobcatfish
force-pushed
the
hermetic_flake
branch
from
8c3ed46
to
2e520bb
Compare
|
Didn't get a chance to do as much investigation as I'd hoped, but I suggest we still give this a try - we can always reopen the flakey test issue and continue investigating if it persists. Fixes #4567 |
tekton-robot
removed
the
do-not-merge/work-in-progress
dibyom
approved these changes
Mar 21, 2022
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: dibyom The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
tekton-robot
added
the
approved
|
/lgtm |
|
/test pull-tekton-pipeline-alpha-integration-tests |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Just WIP for now b/c I want to try out a couple of things first :D
Also use Errorf instead of Fatalf between the two tests (the hermetic
test and the non-hermetic tests) so that if one fails the other will
still run.
In #4567 we see that the hermetic end to end test sometimes fails,
specifically it seems to be the
not-hermetic-run-as-rootversion ofthe test, and it seems like the failure is hitting the 1 minute timeout.
Looking at the test, it seems to be doing an
apt-get updatewhichseems like an operation that would be in grave danger of sometimes
taking a while (especially depending on what version of the latest
ubuntu image is running) so although I'm not sure that's what is causing
the problem, I want to try doing something that is less likely to take
so long but still would require network access, as well as something
that would require priviledged access - which I assume is why the update
was included, to capture the combo of network access and doing something
priviledged. I'm still a bit confused about why both of those elements
are present - I assume both are not allowed in hermetic mode but it
would probably make more sense to test them separately to be sure they
each fail, otherwise only one is covered (i.e. either the network call
is going to fail and halt things, or the priviledged operation)
Submitter Checklist
As the author of this PR, please check off the items in this checklist:
functionality, content, code)
Release Notes