Add Developer Metrics

[aldy505]

Problem Statement

Sentry has Developer Metrics in alpha. We should bring that into self-hosted after GA.

Solution Brainstorm

Discussion: getsentry/sentry#58584
Feature docs: https://develop.sentry.dev/delightful-developer-metrics/
SDK: https://develop.sentry.dev/sdk/metrics/

SDK guide on Python: https://develop.sentry.dev/delightful-developer-metrics/sending-metrics-sdk/

[aldy505]

IMPORTANT: See this comment's edit history for older Sentry versions.

To enable the feature, make sure you're on 24.3.0 or higher. You'd need to edit your sentry/sentry.conf.py file and add these feature flags:

SENTRY_FEATURES.update(
  {
    feature: True
    for feature in (
      # ......
      # For DDM
      # Metrics: Enable ingestion and storage of custom metrics. See custom-metrics for UI.
      "organizations:custom-metrics",
      # Enables experimental WIP custom metrics related features
      "organizations:custom-metrics-experimental",
      # Enables automatically deriving of code mappings
      "organizations:derive-code-mappings",
    )
  }
)

Add new Snuba container on your docker-compose.yml:

  snuba-generic-metrics-gauges-consumer:
    <<: *snuba_defaults
    command: rust-consumer --storage generic_metrics_gauges_raw --consumer-group snuba-gen-metrics-gauges-consumers --auto-offset-reset=latest --max-batch-time-ms 1000 --no-strict-offset-reset

Then do ./install.sh all over again.

[hubertdeng123]

Thanks for creating this issue

[DarkByteZero]

These are required Database Migrations:

docker compose run --rm snuba-api migrations migrate -g spans

[aldy505]

Hello @ayirr7, since GA for Metrics is near, do you want to re-create this PR #2757? (You can't reopen since it's already locked, you need to create a new PR)

I believe adding the gauge consumer now is a good idea, since it will ingest some data beforehand, that can be viewed later on the UI when GA's arrived -- so the Metrics UI won't be empty when it's enabled.

[marianhlavac]

To enable the feature, make sure you're on 23.11.2 or higher.

Note for anyone trying to test this on their self-hosted instance, the minimum version is probably actually higher. Had no luck on 24.1.1 (the Metrics page was not showing at all), after upgrading to 24.7.0 it worked well.

[xuweixi11]

anybody used the metric alert success? it's never triggered, my version is 24.7.0

[aldy505]

Sad news: https://sentry.zendesk.com/hc/en-us/articles/26369339769883-Metrics-Beta-Coming-to-an-End

Currently there are a few draft PR that removes the metrics functionality. Although if you're self-hosting the version 23.12.0 to 24.8.0, Metrics is available for you.

Sad to see this thing go.