Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GCP VertexAI adapter #164

Merged
merged 1 commit into from
Nov 25, 2024
Merged

Add GCP VertexAI adapter #164

merged 1 commit into from
Nov 25, 2024

Conversation

akihikokuroda
Copy link
Contributor

@akihikokuroda akihikokuroda commented Nov 13, 2024

Which issue(s) does this pull-request address?

Ref: #141

Description

So far _stream in chat.ts has been tried. The other 3 _genereate and _stream methods have not been tried yet.

Here are some links that I've been looking:

https://www.npmjs.com/package/@google-cloud/vertexai?activeTab=readme#get-nonstreamed-text-responses
https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference
https://github.com/googleapis/nodejs-vertexai/blob/main/src/models/generative_models.ts

Checklist

  • I have read the contributor guide
  • Linting passes: yarn lint or yarn lint:fix
  • Formatting is applied: yarn format or yarn format:fix
  • Unit tests pass: yarn test:unit
  • E2E tests pass: yarn test:e2e
  • Tests are included
  • Documentation is changed or added
  • Commit messages and PR title follow conventional commits

@akihikokuroda akihikokuroda requested a review from a team as a code owner November 13, 2024 01:06
@akihikokuroda akihikokuroda force-pushed the VertexAI branch 2 times, most recently from b160567 to 581580c Compare November 13, 2024 18:13
Copy link
Contributor

@Tomas2D Tomas2D left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work so far!

src/adapters/vertexai/chat.ts Outdated Show resolved Hide resolved
src/adapters/vertexai/chat.ts Outdated Show resolved Hide resolved
src/adapters/vertexai/chat.ts Outdated Show resolved Hide resolved
src/adapters/vertexai/chat.ts Outdated Show resolved Hide resolved
src/adapters/vertexai/chat.ts Outdated Show resolved Hide resolved
src/adapters/vertexai/llm.ts Outdated Show resolved Hide resolved
src/adapters/vertexai/llm.ts Show resolved Hide resolved
tests/e2e/adapters/vertexai/chat.test.ts Outdated Show resolved Hide resolved
tests/e2e/adapters/vertexai/llm.test.ts Outdated Show resolved Hide resolved
tests/e2e/adapters/vertexai/llm.test.ts Outdated Show resolved Hide resolved
@akihikokuroda akihikokuroda force-pushed the VertexAI branch 3 times, most recently from c7718b8 to 62deeb3 Compare November 14, 2024 17:43
package.json Show resolved Hide resolved
src/adapters/vertexai/chat.ts Outdated Show resolved Hide resolved
src/adapters/vertexai/llm.ts Outdated Show resolved Hide resolved
src/adapters/vertexai/llm.ts Outdated Show resolved Hide resolved
src/adapters/vertexai/llm.ts Outdated Show resolved Hide resolved
src/adapters/vertexai/llm.ts Outdated Show resolved Hide resolved
src/adapters/vertexai/llm.ts Outdated Show resolved Hide resolved
tests/e2e/adapters/vertexai/chat.test.ts Outdated Show resolved Hide resolved
tests/e2e/adapters/vertexai/llm.test.ts Show resolved Hide resolved
@akihikokuroda akihikokuroda force-pushed the VertexAI branch 2 times, most recently from 38f6b6e to 98ca4a9 Compare November 15, 2024 16:15
@akihikokuroda akihikokuroda changed the title [WIP] Add GCP VertexAI adapter Add GCP VertexAI adapter Nov 15, 2024
package.json Outdated Show resolved Hide resolved
src/adapters/vertexai/llm.test.ts Outdated Show resolved Hide resolved
Copy link
Contributor

@Tomas2D Tomas2D left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job! I just noticed that you did not add support for Constraint Decoding. From the documentation it seems that VertexAI supports JSON Schema.

Please look at how it is done in BAM Adapter (

protected _prepareParameters(overrides?: GenerateOptions): typeof this.parameters {
const guided = overrides?.guided ? {} : (this.parameters.decoding ?? {});
const guidedOverride = overrides?.guided;
if (guidedOverride?.choice) {
guided.choice = { ...guided.choice, choices: guidedOverride.choice };
} else if (guidedOverride?.grammar) {
guided.grammar = guidedOverride.grammar;
} else if (guidedOverride?.json) {
guided.json_schema = isString(guidedOverride.json)
? JSON.parse(guidedOverride.json)
: guidedOverride.json;
} else if (guidedOverride?.regex) {
guided.regex = guidedOverride.regex;
} else if (!isEmpty(guidedOverride ?? {})) {
throw new NotImplementedError(
`Following types ${Object.keys(overrides!.guided!).join(",")}" for the constraint decoding are not supported!`,
);
}
return {
...this.parameters,
decoding: guided,
};
}
).

src/adapters/vertexai/utils.ts Outdated Show resolved Hide resolved
src/adapters/vertexai/utils.ts Outdated Show resolved Hide resolved
src/adapters/vertexai/llm.ts Outdated Show resolved Hide resolved
src/adapters/vertexai/llm.ts Outdated Show resolved Hide resolved
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
Copy link
Contributor

@Tomas2D Tomas2D left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job! 🚀

@Tomas2D Tomas2D merged commit d2eb5b3 into i-am-bee:main Nov 25, 2024
2 checks passed
@akihikokuroda akihikokuroda deleted the VertexAI branch November 25, 2024 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants