KAFKA-2063: Preserve behaviour of fetch requests version 2 and below and add tests #5
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is joint work between dguy and enothereska. The work implements KIP-63. Overview of main changes: - New byte-based cache that acts as a buffer for any persistent store and for forwarding changes downstream. - Forwarding record path changes: previously a record in a task completed end-to-end. Now it may be buffered in a processor node while other records complete in the task. - Cleanup and state stores and decoupling of cache from state store and forwarding. - More than 80 new unit and integration tests. Author: Damian Guy <damian.guy@gmail.com> Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Matthias J. Sax, Guozhang Wang Closes apache#1752 from enothereska/KAFKA-3776-poc
…ert from null to logical values The `JsonConverter` class has `LogicalTypeConverter` implementations for Date, Time, Timestamp, and Decimal, but these implementations fail when the input literal value (deserialized from the message) is null. Test cases were added to check for these cases, and these failed before the `LogicalTypeConverter` implementations were fixed to consider whether the schema has a default value or is optional, similarly to how the `JsonToConnectTypeConverter` implementations do this. Once the fixes were made, the new tests pass. Author: Randall Hauch <rhauch@gmail.com> Reviewers: Shikhar Bhushan <shikhar@confluent.io>, Jason Gustafson <jason@confluent.io> Closes apache#1867 from rhauch/kafka-4183
…t field when target field is optional Author: Shikhar Bhushan <shikhar@confluent.io> Reviewers: Konstantine Karantasis <konstantine@confluent.io>, Jason Gustafson <jason@confluent.io> Closes apache#1865 from shikhar/kafka-4173
ijuma
force-pushed
the
kafka-2063-bound-fetch-response-size
branch
from
67544ce to
7534e9e
Compare
This PR implements KIP-78:Cluster Identifiers [(link)](https://cwiki.apache.org/confluence/display/KAFKA/KIP-78%3A+Cluster+Id#KIP-78:ClusterId-Overview) and includes the following changes: 1. Changes to broker code - generate cluster id and store it in Zookeeper - update protocol to add cluster id to metadata request and response - add ClusterResourceListener interface, ClusterResource class and ClusterMetadataListeners utility class - send ClusterResource events to the metric reporters 2. Changes to client code - update Cluster and Metadata code to support cluster id - update clients for sending ClusterResource events to interceptors, (de)serializers and metric reporters 3. Integration tests for interceptors, (de)serializers and metric reporters for clients and for protocol changes and metric reporters for broker. 4. System tests for upgrading from previous versions. Author: Sumit Arrawatia <sumit.arrawatia@gmail.com> Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk> Closes apache#1830 from arrawatia/kip-78
…ch-response-size * apache/trunk: KAFKA-4093; Cluster Id (KIP-78) KAFKA-4173; SchemaProjector should successfully project missing Struct field when target field is optional KAFKA-4183; Corrected Kafka Connect's JSON Converter to properly convert from null to logical values KAFKA-3776: Unify store and downstream caching in streams
|
Also merged trunk and fixed merge conflicts. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please review @nepal.