-
Notifications
You must be signed in to change notification settings - Fork 139
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add GCP VertexAI adapter #164
Conversation
b160567
to
581580c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good work so far!
c7718b8
to
62deeb3
Compare
38f6b6e
to
98ca4a9
Compare
98ca4a9
to
d5e966e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great job! I just noticed that you did not add support for Constraint Decoding. From the documentation it seems that VertexAI supports JSON Schema.
Please look at how it is done in BAM Adapter (
bee-agent-framework/src/adapters/ibm-vllm/llm.ts
Lines 225 to 249 in d82be6e
protected _prepareParameters(overrides?: GenerateOptions): typeof this.parameters { | |
const guided = overrides?.guided ? {} : (this.parameters.decoding ?? {}); | |
const guidedOverride = overrides?.guided; | |
if (guidedOverride?.choice) { | |
guided.choice = { ...guided.choice, choices: guidedOverride.choice }; | |
} else if (guidedOverride?.grammar) { | |
guided.grammar = guidedOverride.grammar; | |
} else if (guidedOverride?.json) { | |
guided.json_schema = isString(guidedOverride.json) | |
? JSON.parse(guidedOverride.json) | |
: guidedOverride.json; | |
} else if (guidedOverride?.regex) { | |
guided.regex = guidedOverride.regex; | |
} else if (!isEmpty(guidedOverride ?? {})) { | |
throw new NotImplementedError( | |
`Following types ${Object.keys(overrides!.guided!).join(",")}" for the constraint decoding are not supported!`, | |
); | |
} | |
return { | |
...this.parameters, | |
decoding: guided, | |
}; | |
} |
d5e966e
to
a1a13c3
Compare
a1a13c3
to
9a674d4
Compare
9a674d4
to
ba5c8a3
Compare
ba5c8a3
to
32bfb5a
Compare
32bfb5a
to
a2ab9bb
Compare
Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>
a2ab9bb
to
decc353
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great job! 🚀
Which issue(s) does this pull-request address?
Ref: #141
Description
So far _stream in chat.ts has been tried. The other 3 _genereate and _stream methods have not been tried yet.
Here are some links that I've been looking:
https://www.npmjs.com/package/@google-cloud/vertexai?activeTab=readme#get-nonstreamed-text-responses
https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference
https://github.com/googleapis/nodejs-vertexai/blob/main/src/models/generative_models.ts
Checklist
yarn lint
oryarn lint:fix
yarn format
oryarn format:fix
yarn test:unit
yarn test:e2e