Kafka exporter does not support zstd compression

[ikstewa]

Component(s)

exporter/kafka

What happened?

Description

Documentation stats zstd is supported however an older version of sarama is being used which does not support zstd.

Steps to Reproduce

Set zstd compression on a kafka exporter:

  kafka:
    brokers:
      - kafka:9092
    topic: "apo_metrics_otlp_json"
    encoding: otlp_json
    producer:
      compression: zstd

Expected Result

zstd compression is supported

Actual Result

Collector fails to start up with the following error:

2023-08-31 16:10:18 Error: failed to build pipelines: failed to create "kafka" exporter for data type "metrics": kafka: invalid configuration (zstd compression requires Version >= V2_1_0_0)
2023-08-31 16:10:18 2023/08/31 23:10:18 collector server run finished with error: failed to build pipelines: failed to create "kafka" exporter for data type "metrics": kafka: invalid configuration (zstd compression requires Version >= V2_1_0_0)

Collector version

otel/opentelemetry-collector-contrib:0.83.0

Environment information

Environment

docker

OpenTelemetry Collector configuration

No response

Log output

2023-08-31 16:10:18 Error: failed to build pipelines: failed to create "kafka" exporter for data type "metrics": kafka: invalid configuration (zstd compression requires Version >= V2_1_0_0)
2023-08-31 16:10:18 2023/08/31 23:10:18 collector server run finished with error: failed to build pipelines: failed to create "kafka" exporter for data type "metrics": kafka: invalid configuration (zstd compression requires Version >= V2_1_0_0)

Additional context

No response

[github-actions]

Pinging code owners:

• exporter/kafka: @pavolloffay @MovieStoreGuy

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[ikstewa]

Confirmed the issue was due to missing protocol_version field. Setting to a value > 2.1.0 fixed the issue.