Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jaeger Monitor not compatible with current spanmetrics setup #864

Closed
elbakerino opened this issue Apr 30, 2023 · 3 comments · Fixed by #1174
Closed

Jaeger Monitor not compatible with current spanmetrics setup #864

elbakerino opened this issue Apr 30, 2023 · 3 comments · Fixed by #1174
Labels
bug Something isn't working

Comments

@elbakerino
Copy link

Bug Report

Which version of the demo you are using?

471204eec5031dffcb241c143822d7e4905abc75

Symptom

Jaeger Monitor does not show metrics: "No data yet! Please see these instructions on how to set up your span metrics."

Reproduce

Clone the mentioned version, start all up, open Jaeger Monitor and you won't see metrics.

Additional Context

It seems that Jaeger requires the setup of the spanmetrics PROCESSOR, but the e.g. Grafana Dashboard here requires the format of the spanmetrics CONNECTOR.

See this SO post for other context.

With adding the deprecated PROCESSOR, the Monitor tab works again.

What would be the correct fix or what other options would be the "reference setup" here?

With this config, an additional PROCESSOR is added - and the Monitor tab works again, but now data is of course written twice to Prometheus.

receivers:
    otlp:
        protocols:
            grpc:

exporters:
    otlp:
        endpoint: "jaeger:4317"
        tls:
            insecure: true
    logging:
    prometheus:
        # this prometheus is used by `spanmetrics` CONNECTOR + PROCESSOR
        endpoint: "otelcol:9464"
        # these configs apply to the `spanmetrics` CONNECTOR
        resource_to_telemetry_conversion:
            enabled: true
        enable_open_metrics: true # even with `false`, the data wasn't compatible with `jaeger`

processors:
    batch:
    # deprecated spanmetrics PROCESSOR, used for compatibility with Jaeger Metrics
    spanmetrics:
        metrics_exporter: prometheus

connectors:
    # new spanmetrics CONNECTOR
    spanmetrics:

service:
    pipelines:
        traces:
            receivers: [ otlp ]
            processors: [ batch, spanmetrics ] # configuring spanmetrics PROCESSOR (deprecated)
            exporters: [ otlp, spanmetrics ] # configuring spanmetrics CONNECTOR (not compatible w/ jaeger)
        metrics:
            receivers: [ redis, otlp, spanmetrics ] # configuring spanmetrics CONNECTOR as source of "metrics" (not compatible w/ jaeger)
            processors: [ batch ]
            exporters: [ prometheus ]
@elbakerino elbakerino added the bug Something isn't working label Apr 30, 2023
@puckpuck
Copy link
Contributor

puckpuck commented May 9, 2023

When moving from processor to connector the metric names and tags also had some small changes to them. Could that be what is the issue?

@julianocosta89
Copy link
Member

Probably yes, a possible solution here would be adding a transform processor to the collector before sending the metrics to Jaeger.

I wonder if Jaeger will adapt the new release to align with the spanmetrics connector metrics name.

@pavolloffay do you happen to know more about that?

@Ecostack
Copy link

Is there any follow up on this issue? I am encountering the same issue while just testing the demo application.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants