Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce dynamic throttling for NEG requests #2124

Merged
merged 9 commits into from
Jun 20, 2023
Merged

Introduce dynamic throttling for NEG requests #2124

merged 9 commits into from
Jun 20, 2023

Conversation

alexkats
Copy link
Member

@alexkats alexkats commented May 18, 2023 •
edited
Loading

The purpose of this whole PR is to introduce the dynamic throttling mechanism for NEG requests (List/Attach/DetachNetworkEndpoints) that should be more flexible than static QPS rate limiter

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label May 18, 2023
@k8s-ci-robot k8s-ci-robot requested review from aojea and code-elinka May 18, 2023 17:10
@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label May 18, 2023
@k8s-ci-robot
Copy link
Contributor

Hi @alexkats. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label May 18, 2023
@alexkats
Copy link
Member Author

/assign @swetharepakula
/assign @bowei

@cezarygerard
Copy link
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 22, 2023
bowei
bowei reviewed May 22, 2023
View reviewed changes
pkg/neg/syncers/backoff.go
NextDelay() (time.Duration, error)
// ResetDelay resets the retry delay
ResetDelay()
// DecreaseDelay returns the decreased delay for next retry
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is kind of confusing -- does this have a side effect and also returns a value?

When will this be called etc vs ResetDelay? I think the other methods are more intuitive and their documentation is sufficient, but DecreaseDelay is somewhat out of ordinary understanding.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember to make the same comment, I can't think on having a backoff handler that decrease, I can see how can be useful to have Reset() method to start over, but allowing to go up and down is indeed confusing

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intended usage for it is inside throttling strategy (which is introduced in later commits). The dynamic strategy tries to adjust the delay based on the previous request results, and that's the place where all three methods are used, including DecreaseDelay

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 24, 2023
swetharepakula
swetharepakula reviewed May 24, 2023
View reviewed changes
Copy link
Member

@swetharepakula swetharepakula left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, the logic looks good. Main comment is to comment both the code and the tests

pkg/neg/syncers/backoff_test.go Outdated Show resolved Hide resolved
pkg/neg/syncers/backoff_test.go Outdated Show resolved Hide resolved
pkg/neg/syncers/backoff_test.go Outdated Show resolved Hide resolved
pkg/neg/syncers/backoff_test.go Outdated Show resolved Hide resolved
pkg/neg/syncers/backoff_test.go Outdated Show resolved Hide resolved
pkg/ratelimit/ratelimit_test.go Show resolved Hide resolved
pkg/ratelimit/metrics/metrics.go Outdated Show resolved Hide resolved
pkg/ratelimit/metrics/metrics.go Outdated Show resolved Hide resolved
pkg/ratelimit/metrics/metrics.go Outdated Show resolved Hide resolved
pkg/ratelimit/metrics/metrics.go Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 2, 2023
@alexkats alexkats force-pushed the neg-dt branch 2 times, most recently from 4857901 to 9d82824 Compare June 2, 2023 19:21
@alexkats
Copy link
Member Author

alexkats commented Jun 2, 2023

/retest

swetharepakula
swetharepakula reviewed Jun 8, 2023
View reviewed changes
Copy link
Member

@swetharepakula swetharepakula left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one comment about the metrics, otherwise LGTM

swetharepakula
swetharepakula reviewed Jun 8, 2023
View reviewed changes
Copy link
Member

@swetharepakula swetharepakula left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one comment about the metrics, otherwise LGTM

@alexkats alexkats force-pushed the neg-dt branch 2 times, most recently from c06446c to 78ccdbc Compare June 20, 2023 09:36
swetharepakula
swetharepakula reviewed Jun 20, 2023
View reviewed changes
Copy link
Member

@swetharepakula swetharepakula left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Alex!

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 20, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alexkats, swetharepakula

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 20, 2023
@k8s-ci-robot k8s-ci-robot merged commit 80f2d99 into kubernetes:master Jun 20, 2023
@alexkats alexkats deleted the neg-dt branch June 21, 2023 21:17
@alexkats alexkats mentioned this pull request Dec 20, 2023
Closed
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@swetharepakula swetharepakula swetharepakula left review comments

@bowei bowei bowei left review comments

@aojea aojea Awaiting requested review from aojea

@code-elinka code-elinka Awaiting requested review from code-elinka

Assignees

@bowei bowei

@swetharepakula swetharepakula

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@alexkats @k8s-ci-robot @cezarygerard @bowei @swetharepakula @aojea