Skip to content
This repository has been archived by the owner on Feb 11, 2022. It is now read-only.

Fix missed error check in API call #54

Merged
merged 4 commits into from
Nov 13, 2020
Merged

Fix missed error check in API call #54

merged 4 commits into from
Nov 13, 2020

Conversation

vasilevp
Copy link
Contributor

@vasilevp vasilevp commented Nov 6, 2020

Could be the cause of #51 & #53 in case of a flaky timeout/API issue that is not caught in time causing state to become fragmented

Also prevent the Broker from functioning when there's several
broker-state apps in maintenance project
@vasilevp vasilevp added the bug Something isn't working label Nov 6, 2020
@vasilevp vasilevp self-assigned this Nov 6, 2020
@vasilevp
Copy link
Contributor Author

vasilevp commented Nov 6, 2020

Steps to repair the affected environments:

  1. Stop all Broker operations (i.e. stop the broker app)
  2. Choose one instance of the broker-state Realm app to be the "target" app (write down URL, ID or anything else unique about it)
  3. Copy all Values from all instances of the broker-state Realm app into the "target" (by hand or via a script)
    • TODO: write a script?
    • TODO: handle duplicates, if any
  4. Delete all broker-state apps except the "target"
  5. Update the Broker to version containing this fix and verify if the issue still occurs with known "bad" instances (c20bdcea-6bf1-4804-ab06-bed618054e7d is one)

@jasonmimick
Copy link
Contributor

Good find.
We missed checking the api calls to use proper context, possibly would be a good idea to have a linter warning in the go-client or something against that to prevent this issue.

realmClient, err := mongodbrealm.New(
context.TODO(),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[q] I see that context parameter is not used in New function - is this intentional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch - probably left over from an older version

nil,
mongodbrealm.SetBaseURL(realmURL),
mongodbrealm.SetAPIAuth(context.TODO(), key.PublicKey, key.PrivateKey),
mongodbrealm.SetAPIAuth(ctx, key.PublicKey, key.PrivateKey),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct me if I'm wrong but the context parameter which is passed everywhere is not used? mongodbrealm.NewRequest doesn't use the parameter passed. Is it planned for future?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm adding context everywhere (except mongodbrealm.New above) right now - not sure if I should do it in this PR or a separate one, as it's a big change and not necessarily related to this error

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok! 👍

Copy link

@antonlisovenko antonlisovenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR looks good - maybe in the future would be great to separate different changes to separate PRs (context and error handling)

The main concern is using errors.Wrap everywhere - maybe not an issue depending on the way the final error is handled and logged

@@ -254,16 +267,18 @@ func encodePlan(v dynamicplans.Plan) (string, error) {
b64 := base64.NewEncoder(base64.StdEncoding, b)
err := json.NewEncoder(b64).Encode(v)
if err != nil {
return "", err
return "", errors.Wrap(err, "cannot marshal plan")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

interesting, wasn't aware of changes inerrors regarding the wrapping.
Am I right that errors.Wrap will override the string message so the initial message will be lost (though the causing error will be saved internally). So loggers may miss that?
Maybe errors.Errorf("..%w") would be a better way?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Wrap() will actually append the inner message when Error() is called: https://godoc.org/github.com/pkg/errors#Wrap

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A real-life example of this would be the following error message:
Service broker error: cannot find plan for instance "c20bdcea-6bf1-4804-ab06-bed618054e7d": cannot fetch instance: cannot find instance in maintenance DB(s): no instances found

Here every : is basically a Wrap() call, going all the way to the top.

pkg/mongodbrealm/mongodbrealm.go Show resolved Hide resolved
Copy link

@antonlisovenko antonlisovenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@vasilevp vasilevp merged commit 042cf24 into master Nov 13, 2020
@vasilevp vasilevp deleted the fix-statestorage branch November 13, 2020 16:11
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants