You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This timestamp is useful in a certain situation. My concrete use case is to expose GitHub API rate limit as a metric. This API rate limit is a value that decreases over time, and periodically it resets to the maximum value. We can get this API rate limit value as a HTTP response header when calling GitHub API.
Imagine a situation where there are two (physical) servers making this GitHub API call, and for the sake of illustration, let's assume that one server issues more requests and the other one does less.
Server1 (frequently issue requests)
Server2 (somehow less requests)
Observed API limit
100
4900
(Last time a server made a GH API call)
3 minutes ago
50 minutes ago
From human's point of view, because server1 made an API call more recently than server2, we can tell that server1's observed API limit is the current value. However, without exposing the sample timestamp, Prometheus cannot distinguish which one is the most recent value. This is because when Prometheus collects sample from these two servers, when there's no timestamp specified, it treats these samples as "the samples collected now".
By exposing the sample timestamp, Prometheus should be able to treat these two samples correctly, and it can tell that server1' s value is the most recent value.
Request
With #967, client_python should learn timestamps for Gauge metrics in the multiprocessing mode. This issue is asking for an option to expose those timestamps via Prometheus format.
If you need sample timestamps, then implement Prometheus exposition yourself. It is very easy (I did this multiple times in Python with very little code).
Generic code and servers should never set timestamps, and as Prometheus documentation says this should not be supported in libraries by default.
The Gauge aggregation from multiple processes, is a different store. It is required there internally to have timestamp, to be able to find what was the most recent Set of the gauge. But this value is purely due to Python multi processing.
Spin off from #967 and #847
Background
The Prometheus exposition format (https://github.com/prometheus/docs/blob/main/content/docs/instrumenting/exposition_formats.md#comments-help-text-and-type-information) allows exporters to specify an optional timestamp for each sample. If this is unset, the collector uses the timestamp that the sample is collected.
This timestamp is useful in a certain situation. My concrete use case is to expose GitHub API rate limit as a metric. This API rate limit is a value that decreases over time, and periodically it resets to the maximum value. We can get this API rate limit value as a HTTP response header when calling GitHub API.
Imagine a situation where there are two (physical) servers making this GitHub API call, and for the sake of illustration, let's assume that one server issues more requests and the other one does less.
From human's point of view, because server1 made an API call more recently than server2, we can tell that server1's observed API limit is the current value. However, without exposing the sample timestamp, Prometheus cannot distinguish which one is the most recent value. This is because when Prometheus collects sample from these two servers, when there's no timestamp specified, it treats these samples as "the samples collected now".
By exposing the sample timestamp, Prometheus should be able to treat these two samples correctly, and it can tell that server1' s value is the most recent value.
Request
With #967, client_python should learn timestamps for Gauge metrics in the multiprocessing mode. This issue is asking for an option to expose those timestamps via Prometheus format.
Code pointers
Sample already can take a timestamp (
client_python/prometheus_client/samples.py
Line 49 in 249490e
client_python/prometheus_client/exposition.py
Lines 191 to 194 in 249490e
For the multiprocessing mode, changing the metrics aggregation logic to propagate the timestamps (
client_python/prometheus_client/multiprocess.py
Line 146 in 249490e
For the non-multiprocessing mode, need to change this _child_samples (
client_python/prometheus_client/metrics.py
Lines 433 to 434 in 249490e
The text was updated successfully, but these errors were encountered: