Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When I was to send the message to kafka's service, the message content was lost when I have found it by kafkatool #4818

Open
2 of 4 tasks
hyhtemple opened this issue Aug 16, 2024 · 0 comments

Comments

@hyhtemple
Copy link

hyhtemple commented Aug 16, 2024

Description

Recently, I have met an issue that I want to send a lot of the message to kafka service, the format of the message is json , it looked like:
{
"EventSort": 16,
"Height": 60,
"ShotTime": "20240815162354",
"StoragePath": "http://xxx.xxx.x.xxx:xxxxx/download?picid=ft=Jpeg&sz=3849&st=c&cid=621202&fp=%2Fopt%2Fcloud%2F20240815%2F3_20240815162418_327853939905100000011.cts&ro=608923&po=873674",
"Type": "02",
"Data": "/9j/4AAQSkZJRgA......AKKKKACiiigD/2Q", // loaded the base64 data of the picture
"Width": 76,
"FileFormat": "Jpeg",
"DeviceID": "62122170001320000020",
"ImageID": "62122170001320000020022024081516235429186"
}

the field of data ' s value is base64 picture, it may be included 2~5 numbers inside of the Json, when I put the message to the Kafka by the Librdkafa's library, I found the object of the Json was lost and I was try saving the message to the locate file, the message in the content of the Json was complete in the file , but it was not complete in the kafka' service when I observered by kafkatool,
I don't known why it did, can you help me with something?

Checklist

  • librdkafka version 0.8.1
  • Apache Kafka version: kafka_2.11-1.0.0
  • librdkafka client configuration: queue.buffering.max.kbytes=2000000 queue.buffering.max.messages=1000000 batch.num.messages=1 request.timeout.ms value=2000 queue.buffering.max.kbytes=20000 queue.buffering.max.messages=10000 message.max.bytes=4194304
  • Operating system: CentOS7.6 x64
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant