On UnknownHostException never send messages to DLQ

[lorenzoponsb]

When the connector loses the connection with Kusto, it never sends the messages to the DLQ.

I think the reason is that it only throws an exception if the KustoSinkConfig.BehaviorOnError.FAIL is activated:
https://github.com/Azure/kafka-sink-azure-kusto/blob/master/src/main/java/com/microsoft/azure/kusto/kafka/connect/sink/FileWriter.java#L196

But for the message to be inserted into the DLQ, it has to be different than BehaviorOnError.FAIL:
https://github.com/Azure/kafka-sink-azure-kusto/blob/master/src/main/java/com/microsoft/azure/kusto/kafka/connect/sink/TopicPartitionWriter.java#L170

[ag-ramachandran]

hello @lorenzoponsb

I think this was by design.

Reason was the following:

The DLQ was set up for errors like deserialization, one message cannot hold processing of all messages in the topic. So make it into bytes and move it out for analysis

In the case of errors like connectivity not being established, the connector will not process any offset (no messages will go through and hence the offset processed will not move for any message)
In summary, the idea of DLQ was to target specific message that was a bad message as opposed to infra level failures where all messages have to be moved to DLQ

[lorenzoponsb]

Thanks!!! and please, change the DOC: https://github.com/Azure/kafka-sink-azure-kusto?tab=readme-ov-file#311-miscellaneous-dead-letter-queue