Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce flakiness by ensuring three successful fetches before returning. #1197

Merged
merged 2 commits into from
Mar 15, 2018

Conversation

nat-henderson
Copy link
Contributor

@nat-henderson nat-henderson commented Mar 14, 2018

Fixes #1131.

@nat-henderson nat-henderson force-pushed the iam-wait branch 2 times, most recently from c01f074 to 9d447d6 Compare March 15, 2018 00:06
@nat-henderson nat-henderson changed the title WIP: Reduce flakiness by ensuring three successful fetches before returning. Reduce flakiness by ensuring three successful fetches before returning. Mar 15, 2018
@nat-henderson nat-henderson force-pushed the iam-wait branch 6 times, most recently from 18e6833 to 33e6fb2 Compare March 15, 2018 01:15
@nat-henderson
Copy link
Contributor Author

nat-henderson commented Mar 15, 2018

There's a serious risk of this making us run out of iam policy fetch quota, making the tests flakier, so most of this new code is to try to keep things on track even when we run out of quota.

Having run all the IAM tests twice, I got 0 flakes. I'll set it running a third time but I expect not to see any further flakes.

google/iam.go Outdated
@@ -64,6 +69,36 @@ func iamPolicyReadModifyWrite(updater ResourceIamUpdater, modify iamPolicyModify
log.Printf("[DEBUG]: Setting policy for %s to %+v\n", updater.DescribeResource(), p)
err = updater.SetResourceIamPolicy(p)
if err == nil {
fetchBackoff := 5 * time.Second
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume you came to this value from trial and error, right? It feels a bit high since that means that at minimum we've just added 15s to this resource, but I'll defer to your findings

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can try starting at 1 - increasing the backoff as we run out of quota might work out okay.

@nat-henderson
Copy link
Contributor Author

Yep, that seems to work fine. :)

@nat-henderson nat-henderson merged commit 2ea3562 into hashicorp:master Mar 15, 2018
@nat-henderson nat-henderson deleted the iam-wait branch March 15, 2018 18:15
ashish-amarnath pushed a commit to ashish-amarnath/terraform-provider-google that referenced this pull request Mar 20, 2018
modular-magician pushed a commit to modular-magician/terraform-provider-google that referenced this pull request Sep 27, 2019
This change should be a no-op - it only changes a handful of imports.
@ghost
Copy link

ghost commented Mar 29, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!

@ghost ghost locked and limited conversation to collaborators Mar 29, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add ability to wait for operation completion
2 participants