Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[prometheus] Eliminate labels fingerprint #9785

Merged
merged 15 commits into from
Jun 27, 2024
Merged

[prometheus] Eliminate labels fingerprint #9785

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
9954a98
remove collector labels fingerprint
gpop63 May 4, 2024
55c01ba
remove query labels fingerprint
gpop63 May 4, 2024
88610b4
remove remote_write labels fingerprint
gpop63 May 4, 2024
2614fdd
update docs
gpop63 May 4, 2024
149bf86
bump package version
gpop63 May 4, 2024
0a39469
fix PR link
gpop63 May 4, 2024
13cdd32
Merge remote-tracking branch 'upstream/main' into tsdb_eliminate_prom…
gpop63 May 13, 2024
25446ae
bump kibana version
gpop63 May 16, 2024
f91a316
fingerprint metrics_names for remote_write
gpop63 Jun 20, 2024
83388df
bump minor version
gpop63 Jun 20, 2024
9588aa1
build readme
gpop63 Jun 20, 2024
c6ab41b
add metrics_names_fingerprint field
gpop63 Jun 20, 2024
dc98267
Merge branch 'main' into tsdb_eliminate_prometheus_fingerprint
gpop63 Jun 25, 2024
a8d7500
update changelog description
gpop63 Jun 26, 2024
4e416b2
Merge remote-tracking branch 'upstream/main' into tsdb_eliminate_prom…
gpop63 Jun 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions packages/prometheus/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.15.3"
gpop63 marked this conversation as resolved.
Show resolved Hide resolved
changes:
- description: Eliminate labels fingerprint for tsdb purposes and use keyword type instead.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, Remove instead of Eliminate would be a better word.
Also, I wonder if it is correct/important to have the phrase - and use the keyword type instead.

type: enhancement
link: https://github.com/elastic/integrations/pull/9785
- version: "1.15.2"
changes:
- description: Fix typo - Replace "darastream" with "datastream"
Expand Down
14 changes: 0 additions & 14 deletions packages/prometheus/data_stream/collector/elasticsearch/ingest_pipeline/default.yml
View file
Open in desktop

This file was deleted.

8 changes: 2 additions & 6 deletions packages/prometheus/data_stream/collector/fields/fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,10 @@
type: group
fields:
- name: labels.*
type: object
object_type: keyword
description: |
Prometheus metric labels
- name: labels_fingerprint
type: keyword
dimension: true
description: Autogenerated ID representing the fingerprint of labels object
description: |
Prometheus metric labels
- name: prometheus.metrics.*
type: object
object_type: double
Expand Down
4 changes: 0 additions & 4 deletions packages/prometheus/data_stream/query/elasticsearch/ingest_pipeline/default.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,6 @@ processors:
field: "prometheus.labels.query_name"
value: "{{_ingest._key}}"
description: Add query_name as a label to cover cases, when query returns a single value with no labels
- fingerprint:
fields: ["prometheus.labels"]
target_field: "prometheus.labels_fingerprint"
ignore_missing: true
on_failure:
- set:
field: event.kind
Expand Down
8 changes: 2 additions & 6 deletions packages/prometheus/data_stream/query/fields/fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,10 @@
type: group
fields:
- name: labels.*
type: object
object_type: keyword
description: |
Prometheus metric labels
- name: labels_fingerprint
type: keyword
dimension: true
description: Autogenerated ID representing the fingerprint of labels object and includes query name
description: |
Prometheus metric labels
- name: prometheus.query.*
type: object
object_type: double
Expand Down
4 changes: 0 additions & 4 deletions packages/prometheus/data_stream/remote_write/elasticsearch/ingest_pipeline/default.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,6 @@ processors:
field: "prometheus.labels.metrics_names"
value: ["{{_ingest._key}}"]
description: Add all keys of the 'prometheus' object to the earlier created field, it includes all metric names and key "labels"
- fingerprint:
fields: ["prometheus.labels"]
gpop63 marked this conversation as resolved.
Show resolved Hide resolved
target_field: "prometheus.labels_fingerprint"
ignore_missing: true
- remove:
field: "prometheus.labels.metrics_names"
description: Remove field as it contains a redundant information and can impact the documents size
Expand Down
8 changes: 2 additions & 6 deletions packages/prometheus/data_stream/remote_write/fields/fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,10 @@
type: group
fields:
- name: labels.*
type: object
object_type: keyword
description: |
Prometheus metric labels
- name: labels_fingerprint
type: keyword
dimension: true
description: Autogenerated ID representing the fingerprint of all labels and the list of metrics names
description: |
Prometheus metric labels
- name: metrics.*
type: object
object_type: double
Expand Down
9 changes: 3 additions & 6 deletions packages/prometheus/docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -223,8 +223,7 @@ The fields reported are:
| prometheus.\*.histogram | Prometheus histogram metric | object | |
| prometheus.\*.rate | Prometheus rated counter metric | object | gauge |
| prometheus.\*.value | Prometheus gauge metric | object | gauge |
| prometheus.labels.\* | Prometheus metric labels | object | |
| prometheus.labels_fingerprint | Autogenerated ID representing the fingerprint of labels object | keyword | |
| prometheus.labels.\* | Prometheus metric labels | keyword | |
| prometheus.metrics.\* | Prometheus metric | object | gauge |
| service.address | Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets). | keyword | |
| service.type | The type of the service data is collected from. The type can be used to group and correlate logs and metrics from one service type. Example: If logs or metrics are collected from Elasticsearch, `service.type` would be `elasticsearch`. | keyword | |
Expand Down Expand Up @@ -437,8 +436,7 @@ The fields reported are:
| prometheus.\*.histogram | Prometheus histogram metric | object | |
| prometheus.\*.rate | Prometheus rated counter metric | object | gauge |
| prometheus.\*.value | Prometheus gauge metric | object | gauge |
| prometheus.labels.\* | Prometheus metric labels | object | |
| prometheus.labels_fingerprint | Autogenerated ID representing the fingerprint of all labels and the list of metrics names | keyword | |
| prometheus.labels.\* | Prometheus metric labels | keyword | |
| prometheus.metrics.\* | Prometheus metric | object | gauge |
| service.address | Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets). | keyword | |
| service.type | The type of the service data is collected from. The type can be used to group and correlate logs and metrics from one service type. Example: If logs or metrics are collected from Elasticsearch, `service.type` would be `elasticsearch`. | keyword | |
Expand Down Expand Up @@ -660,8 +658,7 @@ The fields reported are:
| host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword | |
| host.os.version | Operating system version as a raw string. | keyword | |
| host.type | Type of host. For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword | |
| prometheus.labels.\* | Prometheus metric labels | object | |
| prometheus.labels_fingerprint | Autogenerated ID representing the fingerprint of labels object and includes query name | keyword | |
| prometheus.labels.\* | Prometheus metric labels | keyword | |
| prometheus.query.\* | Prometheus value resulted from PromQL | object | gauge |
| service.address | Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets). | keyword | |
| service.type | The type of the service data is collected from. The type can be used to group and correlate logs and metrics from one service type. Example: If logs or metrics are collected from Elasticsearch, `service.type` would be `elasticsearch`. | keyword | |
Expand Down
4 changes: 2 additions & 2 deletions packages/prometheus/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
format_version: 2.10.0
name: prometheus
title: Prometheus
version: 1.15.2
version: 1.15.3
description: Collect metrics from Prometheus servers with Elastic Agent.
type: integration
categories:
- observability
- monitoring
- containers
conditions:
kibana.version: "^8.12.1"
kibana.version: "^8.14.0"
screenshots:
- src: /img/prometheus-server-overview.png
title: Metricbeat Prometheus Overview
Expand Down