Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix cluster naming for really long service names #4366

Merged
merged 5 commits into from
Aug 29, 2022

Conversation

kflynn
Copy link
Member

@kflynn kflynn commented Jul 31, 2022

Without this fix, the collision-handling code for a cluster with a really long service name will fail, because the really really long service name will push all the stuff that's supposed to disambiguate the cluster names off past the point of relevance. 🤦‍♂️

With this fix, we stick a cryptographic hash of the whole too-long cluster name into the truncated name, so that the stuff trimmed off the end is still correctly accounted for.

I pretty much have tests running on my M1 Pro Mac after this PR.

Fixes #4354. Best reviewed commit-by-commit:

  • 1545a2a: Clean up a duplicated entry in docs/releaseNotes.yml
  • c5a6bd8: fix a buglet where using query_parameters or regex_query_parameters would alter the internal name of the Mapping object. Should have no end-user-visible effect, but OMG was it making debugging weird.
  • 3a5f6ae: fix Envoy cluster creation issue with conflicting service names #4354 by changing the way we do hashing for collision resolution.
  • 607f005: big alterations to TestFakeCollision to better test everything.
  • cb2351e: more test changes so that my M1 Mac can run tests

Signed-off-by: Flynn emissary@flynn.kodachi.com

  • I made sure to update CHANGELOG.md.
  • This is unlikely to impact how Emissary performs at scale.
    It's going to add an SHA1 to every too-long cluster name being generated, but that should be a small fraction of cluster names.
  • My change is adequately tested.
  • I didn't need to update DEVELOPING.md.
  • The changes in this PR have been reviewed for security concerns and adherence to security best practices.

@kflynn kflynn requested a review from a team July 31, 2022 03:13
@LanceEa
Copy link
Contributor

LanceEa commented Aug 1, 2022

@kflynn - looks like you have a test failing.

Also, it make sense, can we just add a corresponding test to ensure it is working as reported in the issue.

Copy link
Contributor

@LanceEa LanceEa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leaving a comment to make github review like me 😄.

@kflynn
Copy link
Member Author

kflynn commented Aug 25, 2022

Hey @LanceEa! I modifiedsubstantially rewrote the TestFakeCollision test to do a better job of testing this; more info in the lead comment above. 🙂

Signed-off-by: Flynn <emissary@flynn.kodachi.com>
Signed-off-by: Flynn <emissary@flynn.kodachi.com>
…hat don't collide.

Signed-off-by: Flynn <emissary@flynn.kodachi.com>
Signed-off-by: Flynn <emissary@flynn.kodachi.com>
Signed-off-by: Flynn <emissary@flynn.kodachi.com>
Copy link
Contributor

@LanceEa LanceEa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kflynn - In general, I'm good with the changes. I just had a few questions so I make sure my understanding is correct.

short_name = name[0:40]
h = hashlib.new("sha1")
h.update(name.encode("utf-8"))
hd = h.hexdigest()[0:16].upper()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just trying to make sure I understand this due to my lack of python skills. Envoy limits us to 60 characters so when it is greater than that we need to shorten and merge clusters.

The first 39 characters + 1 static dash ("-") + 15 character hash so we will shorten it to 55 characters...So did I interpret my python code correctly? If so any reason for only allowing 55 characters rather than full 60?

Copy link
Member Author

@kflynn kflynn Aug 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name[0:40] is actually the first 40 characters rather than the first 39, and there's a two-character suffix at the end (-0), so it's 40 + "-" + 16 for the hash + -0 == 59 characters, which is close enough that I didn't want to push my luck. 😉

(The -0 is all but certainly unneeded at this point, too, but I still need to finish convincing myself of that.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that makes sense and understood. Yeah, I would think introducing the hash negates any reason for keeping the suffix around. Do you want to remove that as part of this PR or follow-up when you have had more time to think of the repercussions of that change?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The latter, yeah. Hopefully soon. 🤞

@LanceEa
Copy link
Contributor

LanceEa commented Aug 29, 2022

@kflynn - it looks good to me. Let me know what you think of removing the suffix as part of this PR?

also, I would like to get a second review, mostly for knowledge transfer so the other team members are aware of this naming convention.

@LanceEa
Copy link
Contributor

LanceEa commented Aug 29, 2022

@ddymko - Can you take a look at this, I think it would be good to understand this logic and how we consolidate clusters that have name collisions.

@ddymko ddymko self-requested a review August 29, 2022 13:28
Copy link
Member

@Alice-Lilith Alice-Lilith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks @kflynn 🙂

Copy link
Member

@ddymko ddymko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

@kflynn kflynn merged commit eeb9039 into master Aug 29, 2022
@kflynn kflynn deleted the flynn/dev/cluster-names branch August 29, 2022 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Envoy cluster creation issue with conflicting service names
4 participants