Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify MeterProvider configurable cardinality limits #2960

Merged
merged 37 commits into from
May 8, 2023
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
61264c7
Specify MetricReader configurable limits
Nov 18, 2022
7ba02d1
Add PR number and spec-matrix entry
Nov 18, 2022
16f6170
edits
Nov 18, 2022
82fb36d
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Nov 28, 2022
0f78e17
changelog order
Nov 28, 2022
26b2ce9
corrections, update spec matrix
Nov 28, 2022
57d35a5
Update specification/metrics/sdk.md
jmacd Nov 30, 2022
e755ee6
Update specification/metrics/sdk.md
jmacd Nov 30, 2022
00c44f6
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Dec 12, 2022
8c87107
Merge branch 'jmacd/metrics_limits' of github.com:jmacd/opentelemetry…
Dec 12, 2022
20c7555
revision
Dec 13, 2022
dcbcaf3
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Dec 16, 2022
7cdc7ab
toc
Dec 16, 2022
f12c8b5
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Jan 4, 2023
df16924
Apply suggestions from code review
jmacd Jan 6, 2023
1d7a117
Merge branch 'main' into jmacd/metrics_limits
reyang Jan 17, 2023
15bce36
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Feb 9, 2023
d116a28
Update with ideas from Jan Spec SIG
Feb 9, 2023
a92c4c7
Merge branch 'jmacd/metrics_limits' of github.com:jmacd/opentelemetry…
Feb 9, 2023
c72fae4
refine text
Feb 10, 2023
bfbf348
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Mar 3, 2023
6cd74e0
Revise based on prototype in otel-launcher-go
Mar 3, 2023
0e59093
revise spelling
Mar 3, 2023
c4a5a61
revise spelling
Mar 3, 2023
7e2bf6d
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Mar 27, 2023
81ae9e7
clarify wording; give MetricReader control over the default
Mar 27, 2023
88c1557
reword
Mar 27, 2023
fbf66c7
use aggregation_cardinality_limit
Mar 28, 2023
e53c22d
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Apr 17, 2023
f1f47f4
clarify attribute SETs h/t @pirgeo
Apr 17, 2023
a0a6596
Update specification/metrics/sdk.md
jmacd Apr 17, 2023
7099f07
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
Apr 19, 2023
7505081
update changelog
Apr 19, 2023
9569ef5
lint
Apr 19, 2023
1a98c25
Update specification/metrics/sdk.md
jmacd May 4, 2023
4aaad07
Merge branch 'main' of github.com:open-telemetry/opentelemetry-specif…
May 5, 2023
cee0c74
Merge branch 'jmacd/metrics_limits' of github.com:jmacd/opentelemetry…
May 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ release.

### Metrics

- Recommended limits applied by default at the MetricProvider level to protect
metrics pipelines against excessive data production from a single
instrumentation library or instrument.
([#2960](https://github.com/open-telemetry/opentelemetry-specification/pull/2960))

### Logs

### Resource
Expand Down Expand Up @@ -134,6 +139,10 @@ release.
"base2_exponential_bucket_histogram". Mark exponential histogram data model and
base2 exponential histogram aggregation as stable.
([#3041](https://github.com/open-telemetry/opentelemetry-specification/pull/3041))
- Recommended limits applied by default at the MetricProvider level to protect
metrics pipelines against excessive data production from a single
instrumentation library or instrument.
([#2960](https://github.com/open-telemetry/opentelemetry-specification/pull/2960))

### Logs

Expand Down
1 change: 1 addition & 0 deletions spec-compliance-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@ formats is required. Implementing more than one format is optional.
| The API provides a way to set and get a global default `MeterProvider`. | X | + | + | + | + | | | + | | + | - | |
| It is possible to create any number of `MeterProvider`s. | X | + | + | + | + | | | + | | + | + | |
| `MeterProvider` provides a way to get a `Meter`. | | + | + | + | + | | | + | | + | - | |
| `MeterProvider` provides a way to set cardinality limits. | X | | | | | | | | | | | |
| `get_meter` accepts name, `version` and `schema_url`. | | + | + | + | + | | | + | | + | - | |
| `get_meter` accepts `attributes`. | | | | | | | | + | | | | |
| When an invalid `name` is specified a working `Meter` implementation is returned as a fallback. | | + | + | + | + | | | | | + | - | |
Expand Down
67 changes: 67 additions & 0 deletions specification/metrics/sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ linkTitle: SDK
* [Observations inside asynchronous callbacks](#observations-inside-asynchronous-callbacks)
* [Resolving duplicate instrument registration conflicts](#resolving-duplicate-instrument-registration-conflicts)
- [Attribute limits](#attribute-limits)
- [Cardinality limits](#cardinality-limits)
* [Specific cardinality limits](#specific-cardinality-limits)
- [Exemplar](#exemplar)
* [ExemplarFilter](#exemplarfilter)
* [Built-in ExemplarFilters](#built-in-exemplarfilters)
Expand Down Expand Up @@ -75,6 +77,12 @@ metrics produced by any `Meter` from the `MeterProvider`. The [tracing SDK
specification](../trace/sdk.md#additional-span-interfaces) has provided some
suggestions regarding how to implement this efficiently.

A `MeterProvider` SHOULD provide a way to specify the configurable
[cardinality limits](#cardinality-limits) to be applied to `Meter`
instances. SDKs that support configurable limits MUST provide an
option to configure the default limits to be applied to all `Meter`
instances by default.

### MeterProvider Creation

The SDK SHOULD allow the creation of multiple independent `MeterProvider`s.
Expand Down Expand Up @@ -597,6 +605,65 @@ Attributes which belong to Metrics are exempt from the
time. Attribute truncation or deletion could affect identity of metric time
series and the topic requires further analysis.

## Cardinality limits

**Status**: [Experimental](../document-status.md)

MeterProviders SHOULD support being configured with limits to be
applied to individual metric instruments. This mechanism supports
protecting metrics pipelines from excessive data production in cases
when the number of timeseries produced by application code grows
large, which can happen due to several factors.

Whether because of single attributes having many distinct values or
because of combinatorial expansion among many attributes, these limits
help protect the overall system from individual sources of excessive
metrics instrumentation.

Implementations may take unique approach to managing concurrency in
the OTel API, therefore we cannot prescribe how to evaluate when the
limit is reached. The requirements for the implementation are:

- An instrument with no configured views SHOULD NOT implement
jmacd marked this conversation as resolved.
Show resolved Hide resolved
cardinality limits.
- An instrument with one or more configured views SHOULD detect that
the limit was reached before or during the next collection event (by
any Reader).
- An instrument is NOT REQUIRED to be remove stale timeseries (e.g.,
jmacd marked this conversation as resolved.
Show resolved Hide resolved
exclude zero-valued sums from consideration) before evaluating the
cardinality limit.

When the limit is reached by an the individual metric instrument,
jmacd marked this conversation as resolved.
Show resolved Hide resolved
subsequent new timeseries created for that instrument will have their
complete attribute set replaced by a single-attribute named `overflow`
jmacd marked this conversation as resolved.
Show resolved Hide resolved
jmacd marked this conversation as resolved.
Show resolved Hide resolved
with value `true`.
jmacd marked this conversation as resolved.
Show resolved Hide resolved

The implementation is permitted to evaluate the cardinality limit at a
time when it is convenient. Implementations are NOT REQUIRED to
exactly meet the cardinality limit when it is reached, because that
implies an unwanted degree of synchronization. Implementations MAY
exceed the limit due to the effects of concurrency. Implementations
SHOULD NOT allow unchecked cardinality growth between collection
events.

Concerning the synchronous instruments, the implementation of
cardinality limits is NOT REQUIRED to ensure that metric API events
are mapped uniquely onto a single timeseries when the instrument has
reached its cardinality limit. All that is required is that every
metric event is aggregated in exactly one timeseries.

### Specific cardinality limits

The SDKs that support cardinality limits MUST support the configuration and enforcement of the following limits.

- The number of timeseries per instrument, with a RECOMMENDED default value 2000.
jmacd marked this conversation as resolved.
Show resolved Hide resolved

All SDKs MUST NOT support the configuration or enforcement of any limit not listed above.
MrAlias marked this conversation as resolved.
Show resolved Hide resolved

SDKs SHOULD NOT support the configuration of unlimited limits. There is no unset
value for this configurable limit. Users that wish to configure
"unlimited" cardinality should instead configure a very large limit.

## Exemplar

**Status**: [Feature-freeze](../document-status.md)
Expand Down