[exporter/kafka] Impelement partitioning for OTLP metrics

[SHaaD94]

Description: Add resource attributes based partitioning for OTLP metrics

In our backend we really need an ability to distribute metrics based on resource attributes.
For this I added additional flag to the configuration.
Some code from traces partitioning by traceId reused.

Judging by issues, this feature is anticipated by several more people.

Link to tracking Issue: 31675

Additionally this feature was menioned in these issues: 29433, 30666

Testing:
Added tests for hashing utility.
Added tests for marshalling and asserting correct keys and the number of messages.
Tested locally with host metrics and chained OTLP metrics receiver.

Documentation:
Changelog entry
Flag is added to the doc of KafkaExporter

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: SHaaD94 / name: Evgeniy Zuikin (a78e71a, 9b4bea7, 925ed8b, 740ec99, ccd1fef, 82c9e64, e536f98, d6745b2, b602a8b, fb81f66, 1f43023, ca9f993, 593702b, deca155, a11531b, e8a2ce6, 4390dfe, 088c4ae, 65bcb4a, 7802c17, c773a31, 25c61a5, 25a8d77, dcc32bb, 1380a2a, e56a50e, a813b9f, 5123395, f14f588, c74f6c7, d161c6b, 6e34560, 4e5ee2f, b4e20e9, 965631f, e09d644, 5c5502d, 9df70f1, db57263, d4ae3df, 22dd715, 13d6284, 9474705, b66537a, 2628a01, 6203927, 244ca03, 8784607, 3ba71b2)

[SHaaD94]

Hello, @dmitryax. Any chance you could have a look at this PR?
Thank you!

[SHaaD94]

I bumped go version to 1.21.9 to fix (it is present in the main branch as well)

Vulnerability #1: GO-2024-2687
    HTTP/2 CONTINUATION flood in net/http
  More info: https://pkg.go.dev/vuln/GO-2024-2687
  Standard library
    Found in: net/http@go1.21.8
    Fixed in: net/http@go1.21.9
    Example traces found:
      #1: config.go:10:2: kafkaexporter.init calls sarama.init, which eventually calls http.CanonicalHeaderKey
      #2: jaeger_marshaler.go:11:2: kafkaexporter.init calls model.init, which eventually calls http.Header.Del
      #3: jaeger_marshaler.go:11:2: kafkaexporter.init calls model.init, which eventually calls http.Header.Get
      #4: kafka_exporter.go:151:25: kafkaexporter.kafkaLogsProducer.Close calls sarama.syncProducer.Close, which eventually calls http.bodyEOFSignal.Read
      #5: pdata_marshaler.go:70:32: kafkaexporter.pdataMetricsMarshaler.Marshal calls pdatautil.MapHash, which eventually calls http.bodyLocked.Read
      #6: kafka_exporter.go:40:20: kafkaexporter.kafkaErrors.Error calls fmt.Sprintf, which eventually calls http.http2ConnectionError.Error
...

Please let me know if it is a wrong way to fix it, thanks

[dmitryax]

This control is super confusing. It's unclear what Key is doing and why it's needed. I believe the flag can be simply sent to the newPdataMetricsMarshaler constructor. And there is no need to expose the KeyableMetricsMarshaler API here.

I understand that you are doing it consistently with what's already done for tracing. We can keep it as is, but should be refactored later for both metrics and traces. Feel free to submit an issue if you agree.

[SHaaD94]

If you don't mind, I would prefer to leave it as is for now and refactor it in next PR. I am planing to raise it shortly for logs partitioning by resources.

[dmitryax]

Yes, that's what I suggesting