Implemented:
- #135 Added support for Kafka v0.11.0.0 - v1.0.0
- Consumer.RebalanceTimeout was removed, so rebalancing is triggered as soon as membership status of a consumer group or subscription of a consumer group member changes.
- #138 Pending offsets are
committed to Kafka faster on rebalancing. That could take upto
consumer.offsets_commit_interval
before, but now it happens as soon as possible.
Fixed:
- #120 Consumption from a topic stopped for a group.
- #123 Inexplicable offset manager timeouts.
- #124 Subscription to a topic fails indefinitely after ZooKeeper connection loss.
- #140 Offset manager keeps loosing connection with a broker.
Implemented:
- Added HTTP API endpoints to list topics optionally with partitions and configuration.
- Ack timeout can now be greater then subscription timeout. So in absence of incoming consume requests for a topic, rebalancing may be delayed until ack timeout expires for all offered messages. If a new request comes while waiting for ack timeout to expire, then the subscription timer is reset for the topic.
- Posts can now be performed with content type
x-www-form-urlencoded
, in that case message should be passed in themsg
form parameter. - Structural logging with sirupsen/logrus and mailgun/logrus-hooks/kafkahook.
- Support for Kafka version 0.10.2.0.
Fixed:
- Explicit acks were fixed for HTTP API. Turned out values of
noAck
,ackPartition
, andackOffset
parameters had been ignored. - A race was found in tests that if a request comes while a topic expiration is in progress consumption from the topic may never resume.
- #100 Consumption from a partition stops if the segment that we read from expires.
Implemented:
- At-Least-Once delivery guarantee via synchronous production and explicit acknowledgement of consumed messages.
- Support for Kafka version 0.10.1.0.
Implemented:
- #81 Added capability to proxy to multiple Kafka/ZooKeeper clusters.
- #16 A YAML
configuration file can be passed to Kafka-Pixy with
--config
command line parameter. A default configuration file is provided for reference. - #87 Added support for gRPC API.
Fixed:
Bug fix release.
- #64 Panic in group consumer.
- #66 Group consumer closed while rebalancing in progress.
- #67 Request timeout errors logged by offset manager.
Major overhaul and refactoring of the implementation to make it easier to understand how the internal components interact with each other. It is an important step before implementation of explicit acknowledgements can be started.
During refactoring the following bugs were detected and fixed:
- #56 Invalid stored offset makes consumer panic.
- #59 Messages are skipped by consumer during rebalancing.
- #62 Messages consumed twice during rebalancing.
- #49 Topic consumption stops while ownership retained.
- #47 Support for Kafka 0.9.0.0. Note that consumer group management is still implemented via ZooKeeper rather than the new Kafka group management API.
This release aims to make getting started with Kafka-Pixy easier.
- #39 First time consume from a group may skip messages.
- #40 Make output of
GET /topics/<>/consumers
easier to read. - By default services listens on 19092 port for incoming API requests and a
unix domain socket is activated only if the
--unixAddr
command line parameter is specified. - By default a pid file is not created anymore. You need to explicitly specify
--pidFile
command line argument to get it created. - The source code was refactored into packages to make it easier to understand internal dependencies.
- Integrated with coveralls.io to collect test coverage data.
The very first version actually used in production.