[prometheus components] Translate the "target" info metric to/from resource attributes

[dashpole]

The spec was updated in open-telemetry/opentelemetry-specification#2381 to require the prometheus receiver to convert the "target" info metric to resource attributes, and to recommend converting resource attributes to the "target" info metric when exporting.

See https://github.com/OpenObservability/OpenMetrics/blob/main/specification/OpenMetrics.md#supporting-target-metadata-in-both-push-based-and-pull-based-systems for a description of the "target" info metric.

[gillg]

Hi @dashpole I have a little question around your plans to prepare the migration on my side.
Do you plan to deprecate resource_to_telemetry_conversion on prometheus exporter, or this config will be in charge to "enable" the target_info metric ?

And same question for prometheusremotewrite eporter, by the way are resource attributes already handled or not at all ?

[gillg]

Additional question, does attributes will be happened if the metric target_info already exists ?

Exemple 1:
Prometheus receiver where target_info already present > resource enrichement on all metrics > prometheus exporter

Exemple 2:
OTLP receiver with a metric target present > resource enrichement on all metrics > prometheus exporter

Does target_info at prometheus expoter side will have original labels + new resource attributes ?

[dashpole]

Do you plan to deprecate resource_to_telemetry_conversion on prometheus exporter, or this config will be in charge to "enable" the target_info metric ?

I think that would be the correct thing to do, but we should discuss with the wider group.

are resource attributes already handled or not at all ?

Resource attributes are dropped in the PRW exporter.

does attributes will be happened if the metric target_info already exists ?

In example 1, the incoming target_info will be translated into resource attributes. Then the attributes are modified, then the resource is translated into target info. The only difference between the initial target_info metric and the final one will be the modifications made in the processor, and the addition of job+instance labels (which are added to all metrics when scraped).

For example 2, that is a good call-out. I don't think we handle that edge case correctly right nowRight now, we would expose duplicate metrics, but we should probably make a non-empty resource override the original target metric.

Does target_info at prometheus expoter side will have original labels + new resource attributes ?

Yes.

[Aneurysm9]

Resource attributes are dropped in the PRW exporter.

PRW also supports resource_to_telemetry, so it should likely share a fate with that capability from the prometheus exporter.

[gillg]

@Aneurysm9 good catch, the readme is not up to date :)

[dashpole]

We still need to implement resource parsing behavior in the receiver

[gouthamve]

Now that the parsing behaviour is implemented, the only thing remaining is prometheusexporter. @dashpole How do we want to proceed with deprecating resource_to_telemetry_conversion?

Can I help with the effort somehow?

[dashpole]

It might be good to discuss that with the wg (now that we are hopefully done with sparse histograms). I don't think there is much urgency to get rid of it, since it is an optional, off-by-default feature.

I think its best to start with adding target_info in the prometheus exporter, and go from there.