Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify documentation around selecor.relabel-config option #7266

Merged
merged 2 commits into from
Apr 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cmd/thanos/query.go
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ func registerQuery(app *extkingpin.App) {
storeSelectorRelabelConf := *extflag.RegisterPathOrContent(
cmd,
"selector.relabel-config",
"YAML with relabeling configuration that allows the Querier to select specific TSDBs by their external label. It follows native Prometheus relabel-config syntax. See format details: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config ",
"YAML file with relabeling configuration that allows selecting blocks to query based on their external labels. It follows the Thanos sharding relabel-config syntax. For format details see: https://thanos.io/tip/thanos/sharding.md/#relabelling ",
extflag.WithEnvSubstitution(),
)

Expand Down
23 changes: 12 additions & 11 deletions docs/components/compact.md
Original file line number Diff line number Diff line change
Expand Up @@ -437,18 +437,19 @@ Flags:
resolution forever
--selector.relabel-config=<content>
Alternative to 'selector.relabel-config-file'
flag (mutually exclusive). Content of
YAML file that contains relabeling
configuration that allows selecting
blocks. It follows native Prometheus
relabel-config syntax. See format details:
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
flag (mutually exclusive). Content of YAML
file with relabeling configuration that allows
selecting blocks to act on based on their
external labels. It follows thanos sharding
relabel-config syntax. For format details see:
https://thanos.io/tip/thanos/sharding.md/#relabelling
--selector.relabel-config-file=<file-path>
Path to YAML file that contains relabeling
configuration that allows selecting
blocks. It follows native Prometheus
relabel-config syntax. See format details:
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
Path to YAML file with relabeling
configuration that allows selecting blocks
to act on based on their external labels.
It follows thanos sharding relabel-config
syntax. For format details see:
https://thanos.io/tip/thanos/sharding.md/#relabelling
--tracing.config=<content>
Alternative to 'tracing.config-file' flag
(mutually exclusive). Content of YAML file
Expand Down
22 changes: 11 additions & 11 deletions docs/components/query.md
Original file line number Diff line number Diff line change
Expand Up @@ -468,18 +468,18 @@ Flags:
--selector.relabel-config=<content>
Alternative to 'selector.relabel-config-file'
flag (mutually exclusive). Content of YAML
with relabeling configuration that allows
the Querier to select specific TSDBs by their
external label. It follows native Prometheus
relabel-config syntax. See format details:
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
file with relabeling configuration that allows
selecting blocks to query based on their
external labels. It follows the Thanos sharding
relabel-config syntax. For format details see:
https://thanos.io/tip/thanos/sharding.md/#relabelling
--selector.relabel-config-file=<file-path>
Path to YAML with relabeling configuration
that allows the Querier to select
specific TSDBs by their external label.
It follows native Prometheus
relabel-config syntax. See format details:
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
Path to YAML file with relabeling
configuration that allows selecting blocks
to query based on their external labels.
It follows the Thanos sharding relabel-config
syntax. For format details see:
https://thanos.io/tip/thanos/sharding.md/#relabelling
--store=<store> ... Deprecation Warning - This flag is deprecated
and replaced with `endpoint`. Addresses of
statically configured store API servers
Expand Down
23 changes: 12 additions & 11 deletions docs/components/store.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,18 +192,19 @@ Flags:
https://thanos.io/tip/thanos/logging.md/#configuration
--selector.relabel-config=<content>
Alternative to 'selector.relabel-config-file'
flag (mutually exclusive). Content of
YAML file that contains relabeling
configuration that allows selecting
blocks. It follows native Prometheus
relabel-config syntax. See format details:
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
flag (mutually exclusive). Content of YAML
file with relabeling configuration that allows
selecting blocks to act on based on their
external labels. It follows thanos sharding
relabel-config syntax. For format details see:
https://thanos.io/tip/thanos/sharding.md/#relabelling
--selector.relabel-config-file=<file-path>
Path to YAML file that contains relabeling
configuration that allows selecting
blocks. It follows native Prometheus
relabel-config syntax. See format details:
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
Path to YAML file with relabeling
configuration that allows selecting blocks
to act on based on their external labels.
It follows thanos sharding relabel-config
syntax. For format details see:
https://thanos.io/tip/thanos/sharding.md/#relabelling
--store.enable-index-header-lazy-reader
If true, Store Gateway will lazy memory map
index-header only once the block is required by
Expand Down
23 changes: 12 additions & 11 deletions docs/components/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -297,18 +297,19 @@ Flags:
remote storage
--selector.relabel-config=<content>
Alternative to 'selector.relabel-config-file'
flag (mutually exclusive). Content of
YAML file that contains relabeling
configuration that allows selecting
blocks. It follows native Prometheus
relabel-config syntax. See format details:
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
flag (mutually exclusive). Content of YAML
file with relabeling configuration that allows
selecting blocks to act on based on their
external labels. It follows thanos sharding
relabel-config syntax. For format details see:
https://thanos.io/tip/thanos/sharding.md/#relabelling
--selector.relabel-config-file=<file-path>
Path to YAML file that contains relabeling
configuration that allows selecting
blocks. It follows native Prometheus
relabel-config syntax. See format details:
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
Path to YAML file with relabeling
configuration that allows selecting blocks
to act on based on their external labels.
It follows thanos sharding relabel-config
syntax. For format details see:
https://thanos.io/tip/thanos/sharding.md/#relabelling
--timeout=5m Timeout to download metadata from remote storage
--tracing.config=<content>
Alternative to 'tracing.config-file' flag
Expand Down
31 changes: 12 additions & 19 deletions docs/sharding.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

# Background

Currently, all components that read from object store assume that all the operations and functionality should be done based on **all** the available blocks that are present in the certain bucket's root directory.
By default, all components that read from object store assume that all the operations and functionality should be done based on **all** the available blocks that are present in the given bucket's root directory.

This is in most cases totally fine, however with time and allowance of storing blocks from multiple `Sources` into the same bucket, the number of objects in a bucket can grow drastically.
This is in most cases totally fine, however when storing blocks from many `Sources` over long durations in the same bucket the number of objects in a bucket can grow drastically.

This means that with time you might want to scale out certain components e.g:

Expand All @@ -18,35 +18,28 @@ Queries against store gateway which are touching large number of blocks (no matt

# Relabelling

Similar to [promtail](https://grafana.com/docs/loki/latest/clients/promtail/configuration/#relabel_configs) this config will follow native [Prometheus relabel-config](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config) syntax.
Similar to [promtail](https://grafana.com/docs/loki/latest/clients/promtail/configuration/#relabel_configs) this config follows native [Prometheus relabel-config](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config) syntax.

Now, thanos only support following relabel actions:
Currently, thanos only supports the following relabel actions:

* keep

* drop

* hashmod
* `external labels` for all components
* on `external labels` for all components

* `__block_id` for store gateway, see this [example](https://github.com/observatorium/configuration/blob/bf1304b0d7bce2ae3fefa80412bb358f9aa176fb/environments/openshift/manifests/observatorium-template.yaml#L1514-L1521)
* on `__block_id` for store gateway, see this [example](https://github.com/observatorium/configuration/blob/bf1304b0d7bce2ae3fefa80412bb358f9aa176fb/environments/openshift/manifests/observatorium-template.yaml#L1514-L1521)

The relabel config defines filtering process done on **every** synchronization with object storage.
The relabel config is used to filter the relevant blocks every time a Thanos component synchronizes with object storage. If a block is dropped, this means it will be ignored by the component.

We will allow potentially manipulating with several of inputs:
Currently, only external labels are allowed as sources. The block will be dropped if the output is an empty label set.

* External labels:
* `<name>`
If no relabel-config is given, all external labels will be kept.

Output:
Example usages:

* If output is empty, drop block.

By default, on empty relabel-config, all external labels are assumed.

Example usages would be:

* Drop blocks which contains external labels cluster=A
* Ignore blocks which contain the external labels `cluster=A`

```yaml
- action: drop
Expand All @@ -55,7 +48,7 @@ Example usages would be:
- cluster
```

* Keep only blocks which contains external labels cluster=A
* Keep only blocks which contain the external labels `cluster=A`

```yaml
- action: keep
Expand Down
2 changes: 1 addition & 1 deletion pkg/extkingpin/flags.go
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ func RegisterSelectorRelabelFlags(cmd FlagClause) *extflag.PathOrContent {
return extflag.RegisterPathOrContent(
cmd,
"selector.relabel-config",
"YAML file that contains relabeling configuration that allows selecting blocks. It follows native Prometheus relabel-config syntax. See format details: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config ",
"YAML file with relabeling configuration that allows selecting blocks to act on based on their external labels. It follows thanos sharding relabel-config syntax. For format details see: https://thanos.io/tip/thanos/sharding.md/#relabelling ",
extflag.WithEnvSubstitution(),
)
}
Loading