Add support for Azure OpenAI #106
Closed
+182
−27
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If `aoai_endpoint` is set, then use Azure OpenAI endpoint rather than the direct OpenAI endpoint. Requires `aoai_deployment` to be specified as well. You should set the model too, so that aichat knows the correct number of remaining tokens. Also adds support for `max_tokens` since the AOAI default is very low. The Azure OpenAI requires chats to be encoded as completions, so logic to do this is added as well.
|
@sigoden please could you approve the workflow so I can see if this builds correctly? |
|
After pr #150, aichat supports multiple clients. It would be better to add an independent azure-openai client. @kw217 @FalacerSelene |
|
see #166 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds support for the Azure OpenAI service, which is an alternate API for accessing the OpenAI models. In the initial-config wizard it adds a question
Use OpenAI via Azure?(default no), and if the answer is yes it asks for the endpoint and deployment name. It stores these in the config file under new keys. It also adds support formax_tokens(on both APIs), since the Azuremax_tokensdefault is very low.Details:
If
aoai_endpointis set, then use Azure OpenAI endpoint rather than the direct OpenAI endpoint. Requiresaoai_deploymentto be specified as well. You should set the model too, so that aichat knows the correct number of remaining tokens.Also adds support for
max_tokenssince the AOAI default is very low.The Azure OpenAI API currently requires chats to be encoded as completions, so logic to do this is added as well.
Tested with gpt-35-turbo (version 0301).