Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RedisCache - ensure FailedProvisioning=false when Provisioned=true #1395

Merged

Conversation

babbageclunk
Copy link
Member

What this PR does / why we need it:
A user reported a case where a RedisCache was provisioned and working but was marked as FailedProvisioning=true. This seems to have happened because it initially failed (possibly a temporary error that we aren't checking for), and then on a subsequent periodic reconciliation the ARM query reported that it was deployed successfully. Use the SetProvisioned helper method (and friends) to prevent this state.

Fixes #1394

Special notes for your reviewer:

If applicable:

  • this PR contains documentation
  • this PR contains tests

@babbageclunk babbageclunk changed the title Ensure FailedProvisioning=false when Provisioned=true RedisCache - ensure FailedProvisioning=false when Provisioned=true Mar 18, 2021
matthchr
matthchr previously approved these changes Apr 2, 2021
Copy link
Member

@matthchr matthchr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - left a comment on your TODO though suggesting a minor change

@@ -50,16 +50,14 @@ func (rc *AzureRedisCacheManager) Ensure(ctx context.Context, obj runtime.Object
instance.Status.Message = err.Error()
return false, err
}
instance.Status.Message = resourcemanager.SuccessMsg
instance.Status.State = string(newRc.ProvisioningState)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: I would put the state setting next to the message/provisioned/provisioning setting

return false, nil
} else if helpers.ContainsString(catchKnownError, azerr.Type) {
// TODO: Not sure why we want this to be provisioning=false?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It comes down to a question of what we want provisioning to mean... I think what the original ASO authors were trying to convey with it was: "I am currently waiting for Azure to actually make the thing", and so they didn't count "waiting in Kubernetes for some reason" as provisioning = true.

I don't think that's really a good interpretation though because it's not stable: We set provisioning = false here but also return false which means "try again" - causing the reconcile loop to come back through and it the SetProvisioning on line 71... so the resource is going to blip between provisioning/not until whatever knownError is resolved. I think it's clearer to just say that it's provisioning the whole time and also include the message if needed explaining what is going on.

Since we already set the message to include the error details on line 74 I think you can just delete this line.

A user reported a case where a RedisCache was provisioned and working
but was marked as FailedProvisioning=true. This seems to have happened
because it initially failed (possibly a temporary error that we aren't
checking for), and then on a subsequent periodic reconciliation the
ARM query reported that it was deployed successfully. Use the
SetProvisioned helper method to prevent this state.
@babbageclunk babbageclunk merged commit 8d221d2 into Azure:master Apr 7, 2021
@babbageclunk babbageclunk deleted the rediscache-provisioning-flags branch April 7, 2021 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug: RedisCache can be Provisioned and FailedProvisioning
2 participants