-
Notifications
You must be signed in to change notification settings - Fork 168
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add instrumentation for simple convenience chat calls (#107)
This PR builds foundation for OpenAI SDK tracing and metrics instrumentation (using Otel-compatible .NET primitives). It's limited to convenience `ChatClient` methods without streaming. The PR implements instrumentation according to [OpenTelemetry GenAI semantic conventions](https://github.com/open-telemetry/semantic-conventions/tree/main/docs/gen-ai). The intention is to add instrumentation to other methods and client types and evolve it along with OTel GenAI semantic conventions. TODO (in this PR): - [x] add samples/docs - [x] add experimental feature-flag required to enable instrumentation - we don't know when OTel semantic conventions will be stable and expect breaking changes. TODO (in next PRs): - [ ] add instrumentation to streaming calls and protocol methods - [ ] track prompts and completions in events - ...
- Loading branch information
Showing
15 changed files
with
966 additions
and
18 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| ## Observability with OpenTelemetry | ||
|
|
||
| > Note: | ||
| > OpenAI .NET SDK instrumentation is in development and is not complete. See [Available sources and meters](#available-sources-and-meters) section for the list of covered operations. | ||
| OpenAI .NET library is instrumented with distributed tracing and metrics using .NET [tracing](https://learn.microsoft.com/dotnet/core/diagnostics/distributed-tracing) | ||
| and [metrics](https://learn.microsoft.com/dotnet/core/diagnostics/metrics-instrumentation) API and supports [OpenTelemetry](https://learn.microsoft.com/dotnet/core/diagnostics/observability-with-otel). | ||
|
|
||
| OpenAI .NET instrumentation follows [OpenTelemetry Semantic Conventions for Generative AI systems](https://github.com/open-telemetry/semantic-conventions/tree/main/docs/gen-ai). | ||
|
|
||
| ### How to enable | ||
|
|
||
| The instrumentation is **experimental** - volume and semantics of the telemetry items may change. | ||
|
|
||
| To enable the instrumentation: | ||
|
|
||
| 1. Set instrumentation feature-flag using one of the following options: | ||
|
|
||
| - set the `OPENAI_EXPERIMENTAL_ENABLE_OPEN_TELEMETRY` environment variable to `"true"` | ||
| - set the `OpenAI.Experimental.EnableOpenTelemetry` context switch to true in your application code when application | ||
| is starting and before initializing any OpenAI clients. For example: | ||
|
|
||
| ```csharp | ||
| AppContext.SetSwitch("OpenAI.Experimental.EnableOpenTelemetry", true); | ||
| ``` | ||
|
|
||
| 2. Enable OpenAI telemetry: | ||
|
|
||
| ```csharp | ||
| builder.Services.AddOpenTelemetry() | ||
| .WithTracing(b => | ||
| { | ||
| b.AddSource("OpenAI.*") | ||
| ... | ||
| .AddOtlpExporter(); | ||
| }) | ||
| .WithMetrics(b => | ||
| { | ||
| b.AddMeter("OpenAI.*") | ||
| ... | ||
| .AddOtlpExporter(); | ||
| }); | ||
| ``` | ||
|
|
||
| Distributed tracing is enabled with `AddSource("OpenAI.*")` which tells OpenTelemetry to listen to all [ActivitySources](https://learn.microsoft.com/dotnet/api/system.diagnostics.activitysource) with names starting with `OpenAI.*`. | ||
|
|
||
| Similarly, metrics are configured with `AddMeter("OpenAI.*")` which enables all OpenAI-related [Meters](https://learn.microsoft.com/dotnet/api/system.diagnostics.metrics.meter). | ||
|
|
||
| Consider enabling [HTTP client instrumentation](https://www.nuget.org/packages/OpenTelemetry.Instrumentation.Http) to see all HTTP client | ||
| calls made by your application including those done by the OpenAI SDK. | ||
| Check out [OpenTelemetry documentation](https://opentelemetry.io/docs/languages/net/getting-started/) for more details. | ||
|
|
||
| ### Available sources and meters | ||
|
|
||
| The following sources and meters are available: | ||
|
|
||
| - `OpenAI.ChatClient` - records traces and metrics for `ChatClient` operations (except streaming and protocol methods which are not instrumented yet) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| using System; | ||
|
|
||
| namespace OpenAI; | ||
|
|
||
| internal static class AppContextSwitchHelper | ||
| { | ||
| /// <summary> | ||
| /// Determines if either an AppContext switch or its corresponding Environment Variable is set | ||
| /// </summary> | ||
| /// <param name="appContexSwitchName">Name of the AppContext switch.</param> | ||
| /// <param name="environmentVariableName">Name of the Environment variable.</param> | ||
| /// <returns>If the AppContext switch has been set, returns the value of the switch. | ||
| /// If the AppContext switch has not been set, returns the value of the environment variable. | ||
| /// False if neither is set. | ||
| /// </returns> | ||
| public static bool GetConfigValue(string appContexSwitchName, string environmentVariableName) | ||
| { | ||
| // First check for the AppContext switch, giving it priority over the environment variable. | ||
| if (AppContext.TryGetSwitch(appContexSwitchName, out bool value)) | ||
| { | ||
| return value; | ||
| } | ||
| // AppContext switch wasn't used. Check the environment variable. | ||
| string envVar = Environment.GetEnvironmentVariable(environmentVariableName); | ||
| if (envVar != null && (envVar.Equals("true", StringComparison.OrdinalIgnoreCase) || envVar.Equals("1"))) | ||
| { | ||
| return true; | ||
| } | ||
|
|
||
| // Default to false. | ||
| return false; | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| namespace OpenAI.Telemetry; | ||
|
|
||
| internal class OpenTelemetryConstants | ||
| { | ||
| // follow OpenTelemetry GenAI semantic conventions: | ||
| // https://github.com/open-telemetry/semantic-conventions/tree/v1.27.0/docs/gen-ai | ||
|
|
||
| public const string ErrorTypeKey = "error.type"; | ||
| public const string ServerAddressKey = "server.address"; | ||
| public const string ServerPortKey = "server.port"; | ||
|
|
||
| public const string GenAiClientOperationDurationMetricName = "gen_ai.client.operation.duration"; | ||
| public const string GenAiClientTokenUsageMetricName = "gen_ai.client.token.usage"; | ||
|
|
||
| public const string GenAiOperationNameKey = "gen_ai.operation.name"; | ||
|
|
||
| public const string GenAiRequestMaxTokensKey = "gen_ai.request.max_tokens"; | ||
| public const string GenAiRequestModelKey = "gen_ai.request.model"; | ||
| public const string GenAiRequestTemperatureKey = "gen_ai.request.temperature"; | ||
| public const string GenAiRequestTopPKey = "gen_ai.request.top_p"; | ||
|
|
||
| public const string GenAiResponseIdKey = "gen_ai.response.id"; | ||
| public const string GenAiResponseFinishReasonKey = "gen_ai.response.finish_reasons"; | ||
| public const string GenAiResponseModelKey = "gen_ai.response.model"; | ||
|
|
||
| public const string GenAiSystemKey = "gen_ai.system"; | ||
| public const string GenAiSystemValue = "openai"; | ||
|
|
||
| public const string GenAiTokenTypeKey = "gen_ai.token.type"; | ||
|
|
||
| public const string GenAiUsageInputTokensKey = "gen_ai.usage.input_tokens"; | ||
| public const string GenAiUsageOutputTokensKey = "gen_ai.usage.output_tokens"; | ||
| } |
Oops, something went wrong.