get message publish timestamp

[hsiaand]

Hi

I am looking to get message publish timestamp on the returned json payload from consumer subscription.
Any option or ways to get this info?

[ppatierno]

It's not possible right now. As you can see it's not in the ConsumerRecord OpenAPI definition here https://github.com/strimzi/strimzi-kafka-bridge/blob/main/src/main/resources/openapi.json#L1512
It's also missing on the producer side https://github.com/strimzi/strimzi-kafka-bridge/blob/main/src/main/resources/openapi.json#L1838

[ppatierno]

Triaged on 13/6/2024: it makes sense to have this implemented to reflect what users already have when using the Kafka consumer/producer API.

[antonio-pedro99]

@ppatierno I will be working on this.

[ppatierno]

Cool, thanks @antonio-pedro99

[antonio-pedro99]

Cool, thanks @antonio-pedro99

Should we care about the ProducerRecord's timestamp too?

I think it makes sense to include the timestamp in the ConsumerRecord, to see when the message was created or appended to the log. Which is the case of @hsiaand.

For my initial work, messages are being retrieved just like below:

[
    {
        "topic": "bridge-quickstart-topic",
        "key": "my-key",
        "value": "sales-lead-0001",
        "partition": 0,
        "offset": 0,
        "timestamp": 1719663008510
    },
    {
        "topic": "bridge-quickstart-topic",
        "key": "my-key",
        "value": "sales-lead-0001",
        "partition": 0,
        "offset": 1,
        "timestamp": 1719663054984
    }
]

Whether the message timestamp value is when it was created or appended to the log, it depends on the message.timestamp.type which in turn is CreatedTime by default.

[ppatierno]

Should we care about the ProducerRecord's timestamp too?

Yes. We should provide the same UX we have with the Kafka Producer API which, as you can see here https://javadoc.io/doc/org.apache.kafka/kafka-clients/latest/org/apache/kafka/clients/producer/ProducerRecord.html, allows to specify the timestamp on the produced record.

[antonio-pedro99]

Should we care about the ProducerRecord's timestamp too?

Yes. We should provide the same UX we have with the Kafka Producer API which, as you can see here https://javadoc.io/doc/org.apache.kafka/kafka-clients/latest/org/apache/kafka/clients/producer/ProducerRecord.html, allows to specify the timestamp on the produced record.

I got it. My question is, should the users mention the timestamp while producing messages? This is something the bridge can handle it internally.

[ppatierno]

As I said, we should provide same UX experience as a Kafka Producer API, where the user provides the timestamp.
The same has to be supported when using the HTTP client. Of course, the timestamp isn't mandatory (as for the Kafka Producer API).