-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
try fixing a PartitionConsumer's race condition #1156
Conversation
This fix may break existing users who expect close to drain the message channel and close to return. It may be possible that these users do not actively drain the message channel, so https://github.com/Shopify/sarama/blob/master/consumer.go#L454 may block, which prevents https://github.com/Shopify/sarama/blob/master/consumer.go#L482, which causes https://github.com/Shopify/sarama/blob/master/consumer.go#L431 to block. |
@georgeteo Yes, you are right. But I still think it's reasonable not draining messages in |
@imjustfly What are your view on this PR? is it still relevant? |
@varun06 Yes, I still think it's relevant. |
@sam-obeid you mind taking a quick look? |
@imjustfly I am worried about breaking part mentioned by @georgeteo |
@varun06 The 2nd commit in this PR fixed the compatibility breaking issue. You can check if it works. |
Looks good to me 👍 @bai please have a look. |
Thanks! |
@varun06 and @imjustfly please have a look at my comment here . Looks like there is a problem in the behaviour. |
@harshana-pickme will have a look today. Thanks for pointing out. |
When closing a
PartitionConsumer
, a race condition will happen if draining the channelmessages
. This causes user's consuming routine gets partial messages, which can lead to confusion.This problem is related to bsm/sarama-cluster#255