diff --git a/accepted/2021/System.Diagnostics/Metrics-Design.md b/accepted/2021/System.Diagnostics/Metrics-Design.md new file mode 100644 index 000000000..2f760102b --- /dev/null +++ b/accepted/2021/System.Diagnostics/Metrics-Design.md @@ -0,0 +1,484 @@ +# Metrics APIs Design + +## Overview + +This document is discussing the .NET Metrics APIs design which implements the [OpenTelemetry specification](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/api.md). + +The OpenTelemetry Metrics APIs support reporting measurements about the execution of a computer program at run time. The Metrics APIs are designed explicitly for processing raw measurements, generally with the intent to produce continuous summaries of those measurements, efficiently and simultaneously. + +The OpenTelemetry architecture is splitting the measurement reporting part to a component called `APIs` and that is the part we are discussing here and proposing APIs for it. The other part is called `SDK` which will monitor, aggregate the Metrics measurements, and export the aggregated measurement to some backend. The SDK part will be implemented in the [OpenTelemetry .NET repo](https://github.com/open-telemetry/opentelemetry-dotnet) and will not be part of the .NET. The `SDK` will depend on the APIs we are going to expose in .NET. + +The proposed APIs will be used by the application or library authors to record metrics measurements. The APIs also include a listener which will allow listening to the recorded measurements events which can be used by the OpenTelemetry SDK (or other potential consumers) to aggregate the measurements and export them. + +The proposed APIs are intended to be supported on Full Framework 4.6 and up, .NET Core supported versions, and .NET 5.0 and up. The proposed APIs will be part of the System.Diagnostics.DiagnosticSource package. + +```MD040 +Naming in this proposal is picked up from the OpenTelemetry specification to avoid having .NET APIs +deviating from the standard even if the standard names are not the best option for the .NET ecosystem. +``` + +## Metric Terminologies + +### Instrument + +The Instrument is the type that will be used by the app and library authors to report measurements (e.g. Counter, ObservableGauge...etc.). The instrument will have a name that should be validated as described in the OpenTelemetry specs. The Instrument can optionally have a description and unit of measurements. Instruments will be created with a numerical type parameter (e.g. `Counter`). The Instrument is going to support all CLS-compliant numerical types which are supported on the Full Framework (Byte, Int16, Int32, Int64, Single, Double, and Decimal). + +There are two types of instruments: + +- The first type we'll call it just `Instrument` for simplicity. These instruments are called inside a request, meaning they have an associated distributed Context (with Span, Baggage, etc.). OpenTelemetry specs call this type of instrument a synchronous Instrument but we are trying to avoid confusion with the async feature of the .NET. The proposal here proposes two instrument classes of that type: `Counter` and `Histogram`. +- The second type is called `ObservableInstrument` which reports measurements by a callback, and lacks Context. OpenTelemetry specs call this type of instrument an asynchronous Instrument but we are trying to avoid confusion with the async feature of the .NET. The proposal here is proposing three instrument classes of that type: `ObservableCounter`, `ObservableGauge`, and `ObservableUpDownCounter`. + +### Meter + +Meter is the factory type responsible for creating Instruments. Meter will have a name and optional version strings. + +### MeterListener + +The listener is the type that allows listening to the measurements reported by instruments (e.g. Counter, ObservableCounter, etc.). The listener is an important class that will be used by OpenTelemetry to implement the Metrics SDK. + +### Tags + +Tag is the term used to refer to a key-value attribute associated with a metric event. Each tag categorizes the metric event, allowing events to be filtered and grouped for analysis. +Current OpenTelemetry specs call it Attributes. We have used the tags name in the tracing APIs and we'll stick with this name for the sake of consistency with tracing. + +## APIs Proposal + +### Meter Class + +```csharp +namespace System.Diagnostics.Metrics +{ + public class Meter : IDisposable + { + + /// + /// The constructor allows creating the Meter class with the name and optionally the version. + /// The name should be validated as described by the OpenTelemetry specs. + /// + public Meter(string name) { throw null; } + public Meter(string name, string? version) { throw null; } + + /// + /// Getter properties to retrieve the Meter name and version + /// + public string Name { get; } + public string? Version { get; } + + /// + /// Factory methods to create Counter and Histogram instruments. + /// + public Counter CreateCounter( + string name, + string? unit = null, + string? description = null) where T : struct { throw null; } + + public Histogram CreateHistogram( + string name, + string? unit = null, + string? description = null) where T : struct { throw null; } + + /// + /// Factory methods to create an ObservableCounter instrument. + /// + + public ObservableCounter CreateObservableCounter( + string name, + Func observeValue, + string? unit = null, + string? description = null) where T : struct { throw null; } + + public ObservableCounter CreateObservableCounter( + string name, + Func> observeValue, + string? unit = null, + string? description = null,) where T : struct { throw null; } + + public ObservableCounter CreateObservableCounter( + string name, + Func>> observeValues, + string? unit = null, + string? description = null) where T : struct { throw null; } + + /// + /// Factory methods to create ObservableGauge instrument. + /// + public ObservableGauge CreateObservableGauge( + string name, + Func observeValue, + string? unit = null, + string? description = null,) where T : struct { throw null; } + + public ObservableGauge CreateObservableGauge( + string name, + Func> observeValue, + string? unit = null, + string? description = null) where T : struct { throw null; } + + public ObservableGauge CreateObservableGauge( + string name, + Func>> observeValues, + string? unit = null, + string? description = null) where T : struct { throw null; } + + /// + /// Factory methods to create ObservableUpDownCounter instrument. + /// + public ObservableUpDownCounter CreateObservableUpDownCounter( + string name, + Func observeValue, + string? unit = null, + string? description = null,) where T : struct { throw null; } + + public ObservableUpDownCounter CreateObservableUpDownCounter( + string name, + Func> observeValue, + string? unit = null, + string? description = null,) where T : struct { throw null; } + + public ObservableUpDownCounter CreateObservableUpDownCounter( + string name, + Func>> observeValues, + string? unit = null, + string? description = null) where T : struct { throw null; } + + public void Dispose() { throw null; } + } +} +``` + +### Instrument Base Class + +```csharp +namespace System.Diagnostics.Metrics +{ + /// + /// Is the base class which contains all common properties between different types of instruments. + /// It contains the protected constructor and the Publish method allows activating the instrument + /// to start recording measurements. + /// + + public abstract class Instrument + { + /// + /// Protected constructor to initialize the common instrument properties. + /// + protected Instrument(Meter meter, string name, string? unit, string? description) { throw null; } + + /// + /// Publish is to allow activating the instrument to start recording measurements and to allow + /// listeners to start listening to such measurements. + /// + protected void Publish() { throw null; } + + /// + /// Getters to retrieve the properties that the instrument is created with. + /// + public Meter Meter { get; } + public string Name { get; } + public string? Unit { get; } + public string? Description { get; } + + /// + /// A property tells if a listener is listening to this instrument measurement recording. + /// + public bool Enabled => throw null; + + /// + /// A property tells if the instrument is a regular instrument or an observable instrument. + /// + public virtual bool IsObservable => throw null; + } +} +``` + +### Instrument Base Class + +```csharp +namespace System.Diagnostics.Metrics +{ + /// + /// Instrument is the base class from which all instruments that report measurements in the context of the request will inherit from. + /// Mainly it will support the CLS compliant numerical types. + /// + public abstract class Instrument : Instrument where T : struct + { + /// + /// Protected constructor to create the instrument with the common properties. + /// + protected Instrument(Meter meter, string name, string? unit, string? description) : + base(meter, name, unit, description) { throw null; } + + /// + /// Record measurement overloads allowing passing different numbers of tags. + /// + + protected void RecordMeasurement(T measurement) { throw null; } + + protected void RecordMeasurement( + T measurement, + KeyValuePair tag) { throw null; } + + protected void RecordMeasurement( + T measurement, + KeyValuePair tag1, + KeyValuePair tag2) { throw null; } + + protected void RecordMeasurement( + T measurement, + KeyValuePair tag1, + KeyValuePair tag2, + KeyValuePair tag3) { throw null; } + + protected void RecordMeasurement( + T measurement, + ReadOnlySpan> tags) { throw null; } + } +} +``` + +### Observable Instrument Base class + +```csharp +namespace System.Diagnostics.Metrics +{ + /// + /// ObservableInstrument is the base class from which all observable instruments will inherit from. + /// It will only support the CLS compliant numerical types. + /// + public abstract class ObservableInstrument : Instrument where T : struct + { + /// + /// Protected constructor to create the instrument with the common properties. + /// + protected ObservableInstrument( + Meter meter, + string name, + string? unit, + string? description) : base(meter, name, unit, description) { throw null; } + + /// + /// Observe() fetches the current measurements being tracked by this instrument. + /// + protected abstract IEnumerable> Observe(); + + public override bool IsObservable => true; + } +} +``` + +### Measurement class + +```csharp + +namespace System.Diagnostics.Metrics +{ + /// + /// A measurement stores one observed value and its associated tags. This type is used by Observable instruments' Observe() method when reporting current measurements with associated tags. + /// with the associated tags. + /// + public readonly struct Measurement where T : struct + { + /// + /// Construct the Measurement using the value and the list of tags. + /// We'll always copy the input list as this is not perf hot path. + /// + public Measurement(T value) { throw null; } + public Measurement(T value, IEnumerable> tags) { throw null; } + public Measurement(T value, params KeyValuePair[] tags) { throw null; } + public Measurement(T value, ReadOnlySpan> tags) { throw null; } + + public ReadOnlySpan> Tags { get { throw null; } } + public T Value { get; } + } +} +``` + +### Instruments Concrete Classes + +```csharp +namespace System.Diagnostics.Metrics +{ + /// + /// The counter is an Instrument that supports non-negative increments. + /// e.g. Number of completed requests. + /// + public sealed class Counter : Instrument where T : struct + { + public void Add(T delta) { throw null; } + public void Add(T delta, + KeyValuePair tag) { throw null; } + public void Add(T delta, + KeyValuePair tag1, + KeyValuePair tag2) { throw null; } + public void Add(T delta, + KeyValuePair tag1, + KeyValuePair tag2, + KeyValuePair tag3) { throw null; } + public void Add(T delta, + ReadOnlySpan> tags) { throw null; } + public void Add(T delta, + params KeyValuePair[] tags) { throw null; } + } + + /// + /// The histogram is an Instrument that can be used to report arbitrary values + /// that are likely to be statistically meaningful. It is intended for statistics such as the request duration. + /// e.g. the request duration. + /// + public sealed class Histogram : Instrument where T : struct + { + public void Record(T value) { throw null; } + public void Record( + T value, + KeyValuePair tag) { throw null; } + public void Record( + T value, + KeyValuePair tag1, + KeyValuePair tag2) { throw null; } + public void Record( + T value, + KeyValuePair tag1, + KeyValuePair tag2, + KeyValuePair tag3) { throw null; } + public void Record(T value, + ReadOnlySpan> tags) { throw null; } + public void Record( + T value, + params KeyValuePair[] tags) { throw null; } + } +} + +``` + +### Observable Instruments + +```csharp +namespace System.Diagnostics.Metrics +{ + /// + /// ObservableCounter is an observable Instrument that reports monotonically increasing value(s) + /// when the instrument is being observed. + /// e.g. CPU time (for different processes, threads, user mode or kernel mode). + /// + public sealed class ObservableCounter : ObservableInstrument where T : struct + { + protected override IEnumerable> Observe() { throw null; } + } + + /// + /// ObservableUpDownCounter is an observable Instrument that reports additive value(s) + /// when the instrument is being observed. + /// e.g. the process heap size + /// + public sealed class ObservableUpDownCounter : ObservableInstrument where T : struct + { + protected override IEnumerable> Observe() { throw null; } + } + + /// + /// ObservableGauge is an observable Instrument that reports non-additive value(s) + /// when the instrument is being observed. + /// e.g. the current room temperature + /// + public sealed class ObservableGauge : ObservableInstrument where T : struct + { + protected override IEnumerable> Observe() { throw null; } + } +} +``` + +### MeterListener + +```csharp +namespace System.Diagnostics.Metrics +{ + /// + /// A delegate to represent the callbacks signatures used in the listener. + /// + public delegate void MeasurementCallback( + Instrument instrument, + T measurement, + ReadOnlySpan> tags, + object? state) where T : struct; + + + /// + /// The listener class can be used to listen to kinds of instruments. + /// recorded measurements. + /// + public sealed class MeterListener : IDisposable + { + /// + /// Simple constructor + /// + public MeterListener() { throw null; } + + /// + /// Callbacks to get notification when an instrument is published + /// + public Action? InstrumentPublished { get; set; } + + /// + /// Callbacks to get notification when stopping the measurement on some instrument + /// this can happen when the Meter or the Listener is disposed of. Or calling Stop() + /// on the listener. + /// + public Action? MeasurementsCompleted { get; set; } + + /// + /// Start listening to a specific instrument measurement recording. + /// + public void EnableMeasurementEvents(Instrument instrument, object? state = null) { throw null; } + + /// + /// Stop listening to a specific instrument measurement recording. + /// returns the associated state. + /// + public object? DisableMeasurementEvents(Instrument instrument) { throw null; } + + /// + /// Set a callback for a specific numeric type to get the measurement recording notification + /// from all instruments which enabled listened to and was created with the same specified + /// numeric type. If a measurement of type T is recorded and a callback of type T is registered, that callback is used. If there is no callback for type T but there is a callback for type object, the measured value is boxed and reported via the object typed callback. If there is neither type T callback nor object callback then the measurement will not be reported. + /// + public void SetMeasurementEventCallback(MeasurementCallback? measurementCallback) where T : struct { throw null; } + + public void Start() { throw null; } + + /// + /// Call all Observable instruments to get the recorded measurements reported to the + /// callbacks enabled by SetMeasurementEventCallback + /// + public void RecordObservableInstruments() { throw null; } + + public void Dispose() { throw null; } + } +} +``` + +### Library Measurement Recording Example + +```csharp + Meter meter = new Meter("io.opentelemetry.contrib.mongodb", "v1.0"); + Counter counter = meter.CreateCounter("Requests"); + counter.Add(1); + counter.Add(1, KeyValuePair.Create("request", "read")); +``` + +### Listening Example + +```csharp + InstrumentListener listener = new InstrumentListener(); + listener.InstrumentPublished = (instrument, meterListener) => + { + if (instrument.Name == "Requests" && instrument.Meter.Name == "io.opentelemetry.contrib.mongodb") + { + meterListener.EnableMeasurementEvents(instrument, null); + } + }; + listener.SetMeasurementEventCallback((instrument, measurement, tags, state) => + { + Console.WriteLine($"Instrument: {instrument.Name} has recorded the measurement {measurement}"); + }); + listener.Start(); +```