Include the raw measurement values as part of a histogram data point.

[samin36]

Use-case
We have a use case where we need to accumulate the metrics reported by multiple parallel, unique instances of a short-lived process. We cannot use the OTLP exporter to directly export to the Collector, as the most recent data point will replace the current one (instead of accumulating). To circumvent the issue, we have a custom exporter that reports the data to another long-running process, which takes care of the accumulation and ultimately exports to the Collector. This works for all instrument types except Histogram; the data reported by the short-lived process only contains a summary of the data: min, max, counts, and sum. We need the actual, raw measurement values in the long-running process for the accumulation.

Proposed Solution
We maintain a copy of the client (Opentelemetry-cpp) library, so we can modify the Histogram data point to include the raw measurement values and fix our problem. But, there may be others out there with the same use-case as ours, and for that reason, I am opening this issue to see if this can be part of the official specification. Thoughts?

[jack-berg]

The request to be able to export raw measurements is a duplicate of #617. I think its unlikely we would extend the histogram OTLP definitions to include raw measurements, since the whole point of metrics is to aggregate many raw measurements to reduce the footprint of the exported data.

However, I have to say I don't understand your use case. You mention that you need the raw measurements since the histogram type only contains a summary of the data, but the same is also true for sum data aggregated for counters / up down counters. OTLP histograms can be re-aggregated just the same as sum data.

[samin36]

Thank you for addressing the issue! As for the use case, as I mentioned earlier, we want to accumulate the metrics reported by parallel instances of a short-lived process. Last we checked, the Collector doesn't seem to support aggregating metrics from multiple sources. To work around this, we have a separate, long-running process that will be export the metrics on behalf of the short-lived processes. Here is a diagram:

To be concrete, suppose the short-lived process has a histogram metric called test_hist. In the long-running process, we will also have the same histogram metric called test_hist. In the short-lived process, I record some values for the histogram using Record and when the time is up, the periodic exporting metric reader will use a custom exporter to export the metric data to the long-running process. Upon receiving the data, the long-running process will parse the information and use it to update its copy of the test_hist Histogram. But, it can't as it needs to call Record, which only accepts the raw value. Just to complete our use case, let's assume the long-running process can somehow call Record with the right values, and it does so successfully. Now, the short-lived process is gone, and when another instance of it is launched, the long-running process will update the same test_hist metric (created at the very beginning when process was launched) with the values from the new short-lived process. Thus, it accumulates over multiple short-lived processes.

[jack-berg]

Upon receiving the data, the long-running process will parse the information and use it to update its copy of the test_hist Histogram. But, it can't as it needs to call Record, which only accepts the raw value.

You're right - the long running process won't be able to accept the aggregated histograms and use them to call Record on the metrics API. The metrics API record operation is for recording raw measurements at the source, where as you're trying to re-aggregate aggregated metrics. Its entirely possible to do, but you'll need to use your own tooling to do so. This type of activity is common in metric backends, and that's kind of what the long running processor is doing. Ultimately, the long running process should be able to emit raw OTLP records.

FWIW, I believe it should be possible to develop a custom processor (or receiver) to accomplish this in the collector. Though as you note, I don't know how you would accomplish it using any of the out-of-the-box tools. Its conceptually similar to the tailsamplingprocessor in that it accumulates state, but the tailsamplingprocessor is accumulating spans where you need to accumulate / aggregate metrics.

[reyang]

The ask for raw measurement is a duplicate of #617.

The actual problem described here seems to be a different thing which is covered by #3797 (comment).