Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Yxjetcd rangestream #12343

Open
wants to merge 22 commits into
base: main
Choose a base branch
from
Open

Yxjetcd rangestream #12343

wants to merge 22 commits into from
+1,519 −290

Conversation

yangxuanjia
Copy link
Contributor

@yangxuanjia yangxuanjia commented Sep 28, 2020 •
edited
Loading

Please read https://github.com/etcd-io/etcd/blob/master/CONTRIBUTING.md#contribution-flow.

#12342
I will make an topic for resolve #12342 , and make test futhermore.
If maintainer think this PR is good, I will do the following things.(unit test, proxy rangestream and lease reagestream)

yangxuanjia and others added 22 commits September 27, 2020 15:56
@yangxuanjia
range stream
ea4d536
@yangxuanjia
range stream
97df433
@yangxuanjia
range stream
3ab92f6
@yangxuanjia
range stream
03e8774
@yangxuanjia
range stream
5821cb8
@yangxuanjia
range stream
7e29c80
@yangxuanjia
range stream
432fb0b
@yangxuanjia
Merge branch 'range_stream' into yxjetcd_rangestream
1d3ec5c
@yangxuanjia
range stream
544f936
https://github.com/etcd-io/etcd/pull/11776
adff7e9
@yangxuanjia
range stream
229d202
@yangxuanjia
range stream
65f0c84
@yangxuanjia
Merge branch 'jdmaster' into range_stream
ca5de71 
# Conflicts:
#	Documentation/dev-guide/api_reference_v3.md
#	api/etcdserverpb/etcdserver.pb.go
#	api/etcdserverpb/rpc.pb.go
#	etcdserver/etcdserverpb/gw/rpc.pb.gw.go
@yangxuanjia
Merge branch 'upstream_master' into jdmaster
dc4e771
@yangxuanjia
Merge branch 'jdmaster' into range_stream
e13f761
@yangxuanjia
Merge branch 'jdmaster' into range_stream
98b04b4 
# Conflicts:
#	Documentation/dev-guide/api_reference_v3.md
#	api/etcdserverpb/etcdserver.pb.go
#	api/etcdserverpb/rpc.pb.go
#	etcdserver/etcdserverpb/gw/rpc.pb.gw.go
@yangxuanjia
new
98ac428
@yangxuanjia
new
d395b4c
@yangxuanjia
new
f655751
@yangxuanjia
Merge branch 'range_stream' into yxjetcd_rangestream
0d070a3
@yangxuanjia
new
76dd18b
@yangxuanjia
Merge branch 'range_stream' into yxjetcd_rangestream
19065fb
@yangxuanjia yangxuanjia mentioned this pull request Oct 23, 2020
Closed
@stale
Copy link

stale bot commented Jan 19, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jan 19, 2021
@nate-double-u nate-double-u mentioned this pull request Feb 2, 2021
Merged
@stale stale bot removed the stale label Feb 5, 2021
@ptabor ptabor mentioned this pull request Feb 9, 2021
Closed
@gyuho
Copy link
Contributor

gyuho commented Apr 26, 2021

We will revisit this very soon, targeting etcd 3.6. /cc @chaochn47

@yangxuanjia
Copy link
Contributor Author

yangxuanjia commented Apr 27, 2021 •
edited
Loading

rangeStream we use it for k8s big range to resolve OOM problem.
I modify the inner rangeStream version include etcd, grpc-proxy.
combine rangeRequest prototype for range and rangeStream.
If need, I will update the code for this PR again.

this PR code version is old.

@gyuho gyuho mentioned this pull request Jun 15, 2021
Merged
@stale
Copy link

stale bot commented Aug 11, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Aug 11, 2021
@stale stale bot closed this Sep 2, 2021
chaochn47 added a commit to chaochn47/etcd that referenced this pull request Oct 11, 2021
@chaochn47
cherry-pick from etcd-io#12343
8fc7fd1
chaochn47 added a commit to chaochn47/etcd that referenced this pull request Oct 11, 2021
@chaochn47
cherry-pick from etcd-io#12343
754fe88
@ptabor ptabor reopened this Feb 23, 2022
@stale stale bot removed the stale label Feb 23, 2022
@chaochn47 chaochn47 mentioned this pull request Mar 10, 2022
Closed
@stale
Copy link

stale bot commented Jun 11, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jun 11, 2022
@stale stale bot closed this Jul 10, 2022
@linxiulei
Copy link

Is this still WIP?

I am working on #14810 that shares the idea on #12343 (comment) and it uses pagination so it won't break the queries.

@ptabor
Copy link
Contributor

ptabor commented Nov 29, 2022

It's kept in the backlog - it doesn't seem there is a maintainer that has capacity to proceed with the project,
but technically it looks like a right thing to do.
Improving the reliability is definitely higher priority than this.

@ptabor ptabor reopened this Nov 29, 2022
@stale stale bot removed stale labels Nov 29, 2022
@stale stale bot removed the stale label Nov 29, 2022
@ptabor ptabor mentioned this pull request Jan 19, 2023
Closed
@fuweid fuweid mentioned this pull request Jul 24, 2023
Merged
@k8s-ci-robot
Copy link

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
area/performance needs-rebase
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

k8s range lots of pods(about 300k pods) in 100 threads, cause etcd OOM
9 participants
@yangxuanjia @nate-double-u @wilsonwang371 @ptabor @anguslees @gyuho @linxiulei @k8s-ci-robot @serathius