Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mark distributor request rate limits as stable #5124

Merged
merged 2 commits into from
May 31, 2023
Merged

Mark distributor request rate limits as stable #5124

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8f121a3
Mark distributor request rate limits as stable
56quarters May 31, 2023
f4dad8f
Fix PR number
56quarters May 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@
* [ENHANCEMENT] Querier and ingester: add experimental support for streaming chunks from ingesters to queriers while evaluating queries. This can be enabled with `-querier.prefer-streaming-chunks=true`. #4886 #5078 #5094
* [ENHANCEMENT] Update Docker base images from `alpine:3.17.3` to `alpine:3.18.0`. #5065
* [ENHANCEMENT] Compactor: reduced the number of "object exists" API calls issued by the compactor to the object storage when syncing block's `meta.json` files. #5063
* [ENHANCEMENT] Distributor: Push request rate limits (`-distributor.request-rate-limit` and `-distributor.request-burst-size`) and their associated YAML configuration are now stable. #5124
* [BUGFIX] Metadata API: Mimir will now return an empty object when no metadata is available, matching Prometheus. #4782
* [BUGFIX] Store-gateway: add collision detection on expanded postings and individual postings cache keys. #4770
* [BUGFIX] Ruler: Support the `type=alert|record` query parameter for the API endpoint `<prometheus-http-prefix>/api/v1/rules`. #4302
Expand Down
10 changes: 4 additions & 6 deletions cmd/mimir/config-descriptor.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2633,23 +2633,21 @@
"kind": "field",
"name": "request_rate",
"required": false,
"desc": "Per-tenant request rate limit in requests per second. 0 to disable.",
"desc": "Per-tenant push request rate limit in requests per second. 0 to disable.",
"fieldValue": null,
"fieldDefaultValue": 0,
"fieldFlag": "distributor.request-rate-limit",
"fieldType": "float",
"fieldCategory": "experimental"
"fieldType": "float"
},
{
"kind": "field",
"name": "request_burst_size",
"required": false,
"desc": "Per-tenant allowed request burst size. 0 to disable.",
"desc": "Per-tenant allowed push request burst size. 0 to disable.",
"fieldValue": null,
"fieldDefaultValue": 0,
"fieldFlag": "distributor.request-burst-size",
"fieldType": "int",
"fieldCategory": "experimental"
"fieldType": "int"
},
{
"kind": "field",
Expand Down
4 changes: 2 additions & 2 deletions cmd/mimir/help-all.txt.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1078,9 +1078,9 @@ Usage of ./cmd/mimir/mimir:
-distributor.remote-timeout duration
Timeout for downstream ingesters. (default 2s)
-distributor.request-burst-size int
[experimental] Per-tenant allowed request burst size. 0 to disable.
Per-tenant allowed push request burst size. 0 to disable.
-distributor.request-rate-limit float
[experimental] Per-tenant request rate limit in requests per second. 0 to disable.
Per-tenant push request rate limit in requests per second. 0 to disable.
-distributor.ring.consul.acl-token string
ACL Token used to interact with Consul.
-distributor.ring.consul.cas-retry-delay duration
Expand Down
4 changes: 4 additions & 0 deletions cmd/mimir/help.txt.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -345,6 +345,10 @@ Usage of ./cmd/mimir/mimir:
Per-tenant ingestion rate limit in samples per second. (default 10000)
-distributor.ingestion-tenant-shard-size int
The tenant's shard size used by shuffle-sharding. This value is the total size of the shard (ie. it is not the number of ingesters in the shard per zone, but the number of ingesters in the shard across all zones, if zone-awareness is enabled). Must be set both on ingesters and distributors. 0 disables shuffle sharding.
-distributor.request-burst-size int
Per-tenant allowed push request burst size. 0 to disable.
-distributor.request-rate-limit float
Per-tenant push request rate limit in requests per second. 0 to disable.
-distributor.ring.consul.hostname string
Hostname and port of Consul. (default "localhost:8500")
-distributor.ring.etcd.endpoints string
Expand Down
5 changes: 2 additions & 3 deletions docs/sources/mimir/references/configuration-parameters/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2530,12 +2530,11 @@ The `memberlist` block configures the Gossip memberlist.
The `limits` block configures default and per-tenant limits imposed by components.

```yaml
# (experimental) Per-tenant request rate limit in requests per second. 0 to
# disable.
# Per-tenant push request rate limit in requests per second. 0 to disable.
# CLI flag: -distributor.request-rate-limit
[request_rate: <float> | default = 0]

# (experimental) Per-tenant allowed request burst size. 0 to disable.
# Per-tenant allowed push request burst size. 0 to disable.
# CLI flag: -distributor.request-burst-size
[request_burst_size: <int> | default = 0]

Expand Down
8 changes: 4 additions & 4 deletions pkg/util/validation/limits.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,8 @@ func (e LimitError) Error() string {
// limits via flags, or per-user limits via yaml config.
type Limits struct {
// Distributor enforced limits.
RequestRate float64 `yaml:"request_rate" json:"request_rate" category:"experimental"`
RequestBurstSize int `yaml:"request_burst_size" json:"request_burst_size" category:"experimental"`
RequestRate float64 `yaml:"request_rate" json:"request_rate"`
RequestBurstSize int `yaml:"request_burst_size" json:"request_burst_size"`
IngestionRate float64 `yaml:"ingestion_rate" json:"ingestion_rate"`
IngestionBurstSize int `yaml:"ingestion_burst_size" json:"ingestion_burst_size"`
AcceptHASamples bool `yaml:"accept_ha_samples" json:"accept_ha_samples"`
Expand Down Expand Up @@ -181,8 +181,8 @@ type Limits struct {
// RegisterFlags adds the flags required to config this to the given FlagSet
func (l *Limits) RegisterFlags(f *flag.FlagSet) {
f.IntVar(&l.IngestionTenantShardSize, "distributor.ingestion-tenant-shard-size", 0, "The tenant's shard size used by shuffle-sharding. This value is the total size of the shard (ie. it is not the number of ingesters in the shard per zone, but the number of ingesters in the shard across all zones, if zone-awareness is enabled). Must be set both on ingesters and distributors. 0 disables shuffle sharding.")
f.Float64Var(&l.RequestRate, requestRateFlag, 0, "Per-tenant request rate limit in requests per second. 0 to disable.")
f.IntVar(&l.RequestBurstSize, requestBurstSizeFlag, 0, "Per-tenant allowed request burst size. 0 to disable.")
f.Float64Var(&l.RequestRate, requestRateFlag, 0, "Per-tenant push request rate limit in requests per second. 0 to disable.")
f.IntVar(&l.RequestBurstSize, requestBurstSizeFlag, 0, "Per-tenant allowed push request burst size. 0 to disable.")
f.Float64Var(&l.IngestionRate, ingestionRateFlag, 10000, "Per-tenant ingestion rate limit in samples per second.")
f.IntVar(&l.IngestionBurstSize, ingestionBurstSizeFlag, 200000, "Per-tenant allowed ingestion burst size (in number of samples).")
f.BoolVar(&l.AcceptHASamples, "distributor.ha-tracker.enable-for-all-users", false, "Flag to enable, for all tenants, handling of samples with external labels identifying replicas in an HA Prometheus setup.")
Expand Down