Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add instrumentation for simple convenience chat calls #107

Merged
merged 11 commits into from
Aug 9, 2024

Conversation

lmolkova
Copy link
Contributor

@lmolkova lmolkova commented Jul 2, 2024

This PR builds foundation for OpenAI SDK tracing and metrics instrumentation (using Otel-compatible .NET primitives).

It's limited to convenience ChatClient methods without streaming. The PR implements instrumentation according to OpenTelemetry GenAI semantic conventions.

The intention is to add instrumentation to other methods and client types and evolve it along with OTel GenAI semantic conventions.

TODO (in this PR):

  • add samples/docs
  • add experimental feature-flag required to enable instrumentation - we don't know when OTel semantic conventions will be stable and expect breaking changes.

TODO (in next PRs):

  • add instrumentation to streaming calls and protocol methods
  • track prompts and completions in events
  • ...

@lmolkova lmolkova marked this pull request as ready for review July 5, 2024 23:14
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
src/Custom/Chat/ChatClient.cs Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
src/Custom/Chat/ChatClient.cs Outdated Show resolved Hide resolved
src/Utility/Instrumentation/Constants.cs Outdated Show resolved Hide resolved
// follows OpenTelemetry GenAI semantic conventions:
// https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai

public const string ErrorTypeKey = "error.type";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we have otel APIs that operate on UTF8 strings?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OTel allows unicode attribute names and uses unicode strings on the wire format.

src/Utility/Instrumentation/InstrumentationFactory.cs Outdated Show resolved Hide resolved
src/Utility/Instrumentation/InstrumentationScope.cs Outdated Show resolved Hide resolved
src/Utility/Instrumentation/InstrumentationScope.cs Outdated Show resolved Hide resolved
src/Utility/Instrumentation/InstrumentationScope.cs Outdated Show resolved Hide resolved
@lmolkova lmolkova force-pushed the openai-chat-basic-tracing branch from d5e6a06 to 4b25142 Compare August 5, 2024 21:29
@lmolkova lmolkova force-pushed the openai-chat-basic-tracing branch from 381b7d3 to 245624a Compare August 9, 2024 20:52
@joseharriaga joseharriaga merged commit d5b5c60 into openai:main Aug 9, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants