Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add wrapAISDKModel method for Vercel's AI SDK #896

Merged
merged 5 commits into from
Jul 30, 2024

Conversation

jacoblee93
Copy link
Collaborator

@jacoblee93 jacoblee93 commented Jul 27, 2024

Improve support for tracing Vercel AI SDK calls with a new wrapAISDKModel method

Because AI SDK calls nest the text/object stream within the response, this PR adds an optional parameter to traceable that specifies a key in the response object to wrap. It also adds support for specially tapping ReadableStream so that traceable does not change them to plain async iterators.

Here's an example:

import { anthropic } from "@ai-sdk/anthropic";
import { streamObject } from "ai";
import { wrapAISDKModel } from "langsmith/wrappers/vercel";

const modelWithTracing = wrapAISDKModel(anthropic("claude-3-haiku-20240307"));
const { partialObjectStream } = await streamObject({
  model: modelWithTracing,
  prompt: "Write a vegetarian lasagna recipe for 4 people.",
  schema: z.object({
    ingredients: z.array(z.string()),
  }),
});
for await (const chunk of partialObjectStream) {
  console.log(chunk);
}

TODO: improve support in the UI for properly rendering token counts and aggregating chunks. The data is returned but is not in one of our currently supported formats.

Example traces:

Generate text: https://smith.langchain.com/public/fbd12847-9485-43cf-a0a3-82c0b3318594/r
Generate object: https://smith.langchain.com/public/c25944c7-2428-44d0-991b-ef3b5e47cab5/r
Stream text: https://smith.langchain.com/public/2fb34c85-fec5-4361-a487-ffcf67718750/r
Stream object: https://smith.langchain.com/public/27f5c9ba-006d-4a2f-b569-2e8da09dae23/r

CC @dqbd

@jacoblee93 jacoblee93 requested a review from dqbd July 27, 2024 21:29
@jacoblee93
Copy link
Collaborator Author

Will follow up with docs!

js/src/wrappers/vercel.ts Outdated Show resolved Hide resolved
@jacoblee93 jacoblee93 merged commit d020931 into main Jul 30, 2024
11 checks passed
@jacoblee93 jacoblee93 deleted the jacob/vercel_ai_sdk_tracing branch July 30, 2024 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants