Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server: drop read request if the leader is changed #10094

Merged
merged 5 commits into from
Sep 24, 2018

Conversation

nolouch
Copy link
Contributor

@nolouch nolouch commented Sep 14, 2018

The read request will block the following read request until the context timeout if meet network partition, and this PR will drop the read request if the leader is changed. ref: #9567 #9566

@gyuho gyuho requested review from gyuho and xiang90 September 14, 2018 07:38
@gyuho gyuho self-assigned this Sep 14, 2018
@@ -938,7 +941,9 @@ func (s *EtcdServer) run() {
s.compactor.Resume()
}
}

if newLeader {
s.leaderChanged <- struct{}{}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handle when linearizableReadLoop had returned from a stopped server?

select {
case s.leaderChanged <- struct{}{}:
default:
}

Copy link
Contributor

@gyuho gyuho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall approach looks good to me. Can you help fix tests? /cc @xiang90


if newLeader {
select {
case s.leaderChanged <- struct{}{}:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we want to do a broadcast style notification, we should close this channel not just sending one item.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean? seems I do not need to broadcast the leader change, it just consumes the channel in linearizableReadLoop. There are two possibilities of being blocked in the loop, one is no read request before the leader changed, and another is that there is a read request coming during the leader changed process. Although there are two places to consume this channel, it only consumes once. cc @xiang90

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The chan is named as leaderChanged. By reading the variable name, people will assume they will get notification when there is a leader change by using this chan. However, when this chan gets used by multiple routines, this chan only deliver one notification. There will be a problem.

I would rather make this chan a broadcast chan.

@nolouch nolouch force-pushed the drop-read branch 2 times, most recently from 4c0b00a to 694ea2c Compare September 17, 2018 15:56
@wenjiaswe
Copy link
Contributor

@wenjiaswe

@xiang90
Copy link
Contributor

xiang90 commented Sep 17, 2018

The general change looks good to me. defer to @gyuho after fixing the CI issue.

@gyuho
Copy link
Contributor

gyuho commented Sep 17, 2018

Thanks @xiang90. Will take a look at CIs this week.

@gyuho gyuho added this to the etcd-v3.4 milestone Sep 21, 2018
@xiang90
Copy link
Contributor

xiang90 commented Sep 23, 2018

@gyuho can you check the CI when you have time? thanks!

@gyuho
Copy link
Contributor

gyuho commented Sep 24, 2018

@xiang90 There were some flaky tests from clientv3. Not related. Merging on greens. Thanks!

@gyuho gyuho merged commit fb67483 into etcd-io:master Sep 24, 2018
@xiang90
Copy link
Contributor

xiang90 commented Sep 24, 2018

@gyuho Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants