Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

example/prometheus: histogram has only +Inf buckets #2419

Closed
jpkrohling opened this issue Nov 26, 2021 · 1 comment · Fixed by #2493
Closed

example/prometheus: histogram has only +Inf buckets #2419

jpkrohling opened this issue Nov 26, 2021 · 1 comment · Fixed by #2493
Labels
area:metrics Part of OpenTelemetry Metrics bug Something isn't working

Comments

@jpkrohling
Copy link
Member

Description

The Prometheus example is a bit confusing, with three histogram measurements resulting in only two buckets (2x +Inf), each with a value of 1:

# HELP ex_com_two 
# TYPE ex_com_two histogram
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="+Inf"} 1
ex_com_two_sum{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0"} 2
ex_com_two_count{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0"} 1
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="+Inf"} 1
ex_com_two_sum{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0"} 2
ex_com_two_count{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0"} 1

Reading the code, I would expect multiple buckets, some of them perhaps empty. Turns out, config.DefaultHistogramBoundaries is an empty list of values:

config := prometheus.Config{}
c := controller.New(
processor.NewFactory(
selector.NewWithHistogramDistribution(
histogram.WithExplicitBoundaries(config.DefaultHistogramBoundaries),
),
aggregation.CumulativeTemporalitySelector(),
processor.WithMemory(true),
),
)

Changing the example so that the first param to the factory is selector.NewWithHistogramDistribution() results in the following, which is already closer to my expectation:

# HELP ex_com_two 
# TYPE ex_com_two histogram
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="0.005"} 0
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="0.01"} 0
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="0.025"} 0
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="0.05"} 0
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="0.1"} 0
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="0.25"} 0
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="0.5"} 0
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="1"} 0
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="2.5"} 1
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="5"} 1
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="10"} 1
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="+Inf"} 1
ex_com_two_sum{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0"} 2
ex_com_two_count{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0"} 1
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="0.005"} 0
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="0.01"} 0
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="0.025"} 0
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="0.05"} 0
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="0.1"} 0
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="0.25"} 0
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="0.5"} 0
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="1"} 0
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="2.5"} 1
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="5"} 1
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="10"} 1
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="+Inf"} 2
ex_com_two_sum{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0"} 14
ex_com_two_count{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0"} 2

Perhaps the example could show how to specify explicit buckets. For instance, by changing the original example to have the following config:

	config := prometheus.Config{
		DefaultHistogramBoundaries: []float64{1, 2, 5, 10, 20, 50},
	}

This would result in the following buckets:

# HELP ex_com_two 
# TYPE ex_com_two histogram
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="1"} 0
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="2"} 0
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="5"} 1
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="10"} 1
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="20"} 1
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="50"} 1
ex_com_two_bucket{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="+Inf"} 1
ex_com_two_sum{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0"} 2
ex_com_two_count{ex_com_lemons="13",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0"} 1
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="1"} 0
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="2"} 0
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="5"} 1
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="10"} 1
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="20"} 2
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="50"} 2
ex_com_two_bucket{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0",le="+Inf"} 2
ex_com_two_sum{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0"} 14
ex_com_two_count{A="1",B="2",C="3",ex_com_lemons="10",service_name="unknown_service:prometheus",telemetry_sdk_language="go",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.2.0"} 2

Environment

  • OS: Fedora 35
  • Architecture: amd64
  • Go Version: 1.17
  • opentelemetry-go version: 1ea6ee3 (main as of this writing)

Steps To Reproduce

  1. cd example/prometheus
  2. go run .
  3. open localhost:2222
@jpkrohling jpkrohling added the bug Something isn't working label Nov 26, 2021
@jpkrohling jpkrohling changed the title examples/prometheus: histogram has only +Inf buckets example/prometheus: histogram has only +Inf buckets Nov 26, 2021
@MrAlias MrAlias added the area:metrics Part of OpenTelemetry Metrics label Dec 6, 2021
@NeesonD
Copy link

NeesonD commented Dec 9, 2021

@MrAlias

  1. Hi, bro. why func NewWithHistogramDistribution() can set multi histogram.Option params ? I found just the last one set in config.explicitBoundaries.
  2. How do I set up a separate bucket for each Histogram.

like this:

metricServerReqDur = prometheus.NewHistogramVec(prometheus.HistogramOpts{
		Namespace:   serverNamespace,
		Subsystem:   "requests",
		Name:        "duration_ms",
		Help:        "http server requests duration(ms).",
		ConstLabels: nil,
		Buckets:     []float64{5, 10, 25, 50, 100, 250, 500, 1000},
	}, []string{"path"})

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:metrics Part of OpenTelemetry Metrics bug Something isn't working
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants