Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invalid stored offset makes consumer panic #56

Closed
horkhe opened this issue Apr 11, 2016 · 0 comments
Closed

Invalid stored offset makes consumer panic #56

horkhe opened this issue Apr 11, 2016 · 0 comments
Labels

Comments

@horkhe
Copy link
Member

horkhe commented Apr 11, 2016

If an offset stored for a topic partition is out of partition bounds then Kafka-Pixy panics on consumption attempt:

panic: </smartConsumer[1]/group-1[0]/test.1:0[0]> failed to start partition consumer: offset=1152, err=(kafka server: The requested offset is outside the range of offsets maintained by the server for the given topic/partition.)

goroutine 704 [running]:
panic(0x4ff6a0, 0xc820680240)
    /usr/local/go/src/runtime/panic.go:464 +0x3e6
github.com/mailgun/kafka-pixy/consumer.(*exclusiveConsumer).run(0xc82051e6c0)
    /Users/maximvladimirsky/workspace/goworld/src/github.com/mailgun/kafka-pixy/consumer/smart_consumer.go:292 +0x6d4
github.com/mailgun/kafka-pixy/consumer.(*exclusiveConsumer).(github.com/mailgun/kafka-pixy/consumer.run)-fm()
    /Users/maximvladimirsky/workspace/goworld/src/github.com/mailgun/kafka-pixy/consumer/smart_consumer.go:260 +0x20
github.com/mailgun/kafka-pixy/consumer.spawn.func1(0xc82051e738, 0xc82047f6d0)
    /Users/maximvladimirsky/workspace/goworld/src/github.com/mailgun/kafka-pixy/consumer/utils.go:16 +0x44
created by github.com/mailgun/kafka-pixy/consumer.spawn
    /Users/maximvladimirsky/workspace/goworld/src/github.com/mailgun/kafka-pixy/consumer/utils.go:17 +0x56
exit status 2
FAIL    github.com/mailgun/kafka-pixy/consumer  7.868s

That can happen when consumption of a topic stopped for period of time greater than the topic retention period.

@horkhe horkhe added the bug label Apr 11, 2016
@horkhe horkhe closed this as completed in 3d23f0a Apr 24, 2016
horkhe added a commit that referenced this issue Apr 24, 2016
Adjust partition offset if out of bounds (fixes #56)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant