Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

etcd: fix JWT renewal with user/pass authentication #6905

Merged
merged 2 commits into from
May 19, 2021
Merged

Conversation

awly
Copy link
Contributor

@awly awly commented May 18, 2021

Two changes here:

  1. update go.etcd.io/etcd to their latest module-compliant versions
  2. use a separate connection for peer version checking
    the implementation of b.client.Status (in the etcd library) internally causes the in-memory JWT store to be overwritten.
    this ends up breaking the JWT refresh logic (one store has the exired token but a different one gets the refreshed one).
    I will send an upstream fix too

# 2 works without # 1.
Since this PR is against master, I decided to keep # 1 anyway, for 7.0.
I will cherry-pick only # 2 to 6.2, without dependency changes.

FYI @fspmarshall because you had some memory issues with the etcd client; check if this updated version of the library fixes them

Fixes #6881

@Joerger
Copy link
Contributor

Joerger commented May 18, 2021

@klizhentas has shown concern about moving to a beta version of etcd for a production release #6183. However it doesn't seem clear when etcd plans on releasing 3.5 and this has pushed back a few other fixes as well. @awly Do you have a sense of how stable the current beta version is or how soon it'll be released officially?

Copy link
Collaborator

@r0mant r0mant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change lgtm but I agree I would be cautious vendoring an alpha dependency. Is it required for this change?

lib/backend/etcdbk/etcd.go Outdated Show resolved Hide resolved
 There is a data race in etcd that breaks the internal state in etcd client
implementation for some server setups (user/pass authentication with
JWTs).
@awly
Copy link
Contributor Author

awly commented May 18, 2021

You're right, let's wait for etcd 3.5 to be fully released before updating.
Reverted all dependency changes. PTAL

@awly awly requested a review from r0mant May 18, 2021 23:30
@awly awly enabled auto-merge (squash) May 19, 2021 16:53
@awly awly merged commit 11c62e8 into master May 19, 2021
@awly awly deleted the andrew/update-etcd branch May 19, 2021 17:07
awly pushed a commit that referenced this pull request May 19, 2021
There is a data race in etcd that breaks the internal state in etcd client
implementation for some server setups (user/pass authentication with
JWTs).
awly pushed a commit that referenced this pull request May 19, 2021
There is a data race in etcd that breaks the internal state in etcd client
implementation for some server setups (user/pass authentication with
JWTs).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

etcd: infinite reconnect loop when using JWT authentication
3 participants