Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug that header values are not null-terminated #970

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jan-lindemann
Copy link
Contributor

There are currently two bugs in the producer code when dealing with buffers as header values.

The first issue is that the data of buffers passed as header values is cut off on the first null byte. According to the documentation (see https://kafka.apache.org/documentation/#recordheader) headers of Kafka messages can be arbitrary byte sequences. The bug is caused by the use of the copy constructor in std::string, which stops copying on the null byte. Using the copy constructor where the length of the byte sequence is passed explicitly will fix this issue.

A second issue is that a buffer is treated similar to a string and that its content is passed to the UTF8String constructor. This will modify the byte sequence, as this constructor will transform the passed data from the node internal UTF-16 string representation into a UTF-8 representation. Since buffers can be arbitrarily encoded data and not necessarily strings, buffers should not be passed to the UTF8String constructor. Instead, their value should be used as is.

@iradul
Copy link
Collaborator

iradul commented Oct 16, 2022

Thank you for the PR.
Fixing the first issue implies a fix for the second one since utf-16 should be treated as Buffer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants