Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AOAI: fix CompletionsUsage model #22439

Conversation

trrwilson
Copy link
Member

@trrwilson trrwilson commented Feb 3, 2023

<<Skipping template for topic-to-topic PR>>

This addresses an error/inconsistency in the model definition for CompletionsUsage in Azure.AI.OpenAI's Inference area.

Specifically, a live service interaction demonstrates that a JSON key we want to process is named completion_tokens:

{"completion_tokens":75,"prompt_tokens":9,"total_tokens":84}

The model, meanwhile, expresses completion_token (no terminating 's') and the emitted projection code thus fails to deserialize the data.

In lieu of correcting a typo in the model comment, I replaced it with my own verbiage that I think adds a small amount of value.

@openapi-workflow-bot
Copy link

Hi, @trrwilson Thanks for your PR. I am workflow bot for review process. Here are some small tips.

  • Please ensure to do self-check against checklists in first PR comment.
  • PR assignee is the person auto-assigned and responsible for your current PR reviewing and merging.
  • For specs comparison cross API versions, Use API Specs Comparison Report Generator
  • If there is CI failure(s), to fix CI error(s) is mandatory for PR merging; or you need to provide justification in PR comment for explanation. How to fix?

  • Any feedback about review process or workflow bot, pls contact swagger and tools team. vscswagger@microsoft.com

    @openapi-pipeline-app
    Copy link

    openapi-pipeline-app bot commented Feb 3, 2023

    Swagger Validation Report

    ️️✔️BreakingChange succeeded [Detail] [Expand]
    There are no breaking changes.
    ️️✔️Breaking Change(Cross-Version) succeeded [Detail] [Expand]
    There are no breaking changes.
    ️️✔️LintDiff succeeded [Detail] [Expand]
    Validation passes for LintDiff.
    ️️✔️Avocado succeeded [Detail] [Expand]
    Validation passes for Avocado.
    ️❌SwaggerAPIView: 0 Errors, 1 Warnings failed [Detail]
    Rule Message
    ⚠️ Failed to generate swagger APIView. The readme file format is invalid and the tag is not defined. Use the provided readme template for guidance readme template. For more details, please check the detail log. "How to fix":"Check the readme file and make sure the readme file format is valid and the tag is defined. Use the provided readme template"
    ️❌CadlAPIView: 0 Errors, 1 Warnings failed [Detail]
    Rule Message
    ⚠️ Failed to generate cadl APIView. Please check the detail log and make sure cadl compiler version is the latest. "How to fix":"Check the detailed log and update cadl version to latest."
    ️️✔️ModelValidation succeeded [Detail] [Expand]
    Validation passes for ModelValidation.
    ️️✔️SemanticValidation succeeded [Detail] [Expand]
    Validation passes for SemanticValidation.
    ️️✔️PrettierCheck succeeded [Detail] [Expand]
    Validation passes for PrettierCheck.
    ️️✔️SpellCheck succeeded [Detail] [Expand]
    Validation passes for SpellCheck.
    ️❌CadlValidation: 5 Errors, 93 Warnings failed [Detail]

    Only 30 items are listed, please refer to log for more details.

    Rule Message
    MissingExamplesDirectory "details":"The 'examples' directory is missing in the cadl folder specification/cognitiveservices/OpenAI.Authoring,
    please ensure the 'examples' is added in the PR."
    InConsistentSwagger "details":"The generated swagger file 2022-06-01-preview/openapi.json from cadl specification/cognitiveservices/OpenAI.Authoring is not the same as the '/mnt/vss/_work/1/azure-rest-api-specs/specification/cognitiveservices/data-plane/AzureOpenAI/authoring/preview/2022-06-01-preview/azureopenai.json' in PR,
    please make sure the swagger is consistent with the generated swagger. You can find the difference in the pipeline log."
    MissingExamplesDirectory "details":"The 'examples' directory is missing in the cadl folder specification/cognitiveservices/OpenAI.Inference,
    please ensure the 'examples' is added in the PR."
    InConsistentSwagger "details":"The generated swagger file 2022-06-01-preview/openapi.json from cadl specification/cognitiveservices/OpenAI.Inference is not the same as the '/mnt/vss/_work/1/azure-rest-api-specs/specification/cognitiveservices/data-plane/AzureOpenAI/authoring/preview/2022-06-01-preview/azureopenai.json' in PR,
    please make sure the swagger is consistent with the generated swagger. You can find the difference in the pipeline log."
    SwaggerNotExistInPR "details":"The generated swagger file specification/cognitiveservices/OpenAI.Inference/@azure-tools/cadl-autorest/2022-12-01/openapi.json from cadl specification/cognitiveservices/OpenAI.Inference is not in the PR,
    please make sure it presents into the PR."
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/model.models.cadl#L58
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/model.models.cadl#L74
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/model.models.cadl#L88
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L21
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L25
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L37
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L46
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/file.models.cadl#L23
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/file.models.cadl#L27
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L49
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L52
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L91
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L67
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L74
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L109
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L120
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L131
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L137
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L149
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L163
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L170
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/finetune.models.cadl#L177
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/model.models.cadl#L21
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/model.models.cadl#L25
    ⚠️ @azure-tools/cadl-azure-core/casing-style The names of Property types must use camelCase
    Location: cognitiveservices/OpenAI.Authoring/models/model.models.cadl#L38
    ️️✔️PR Summary succeeded [Detail] [Expand]
    Validation passes for Summary.
    Posted by Swagger Pipeline | How to fix these errors?

    @openapi-pipeline-app
    Copy link

    openapi-pipeline-app bot commented Feb 3, 2023

    Swagger Generation Artifacts

    ️️✔️ApiDocPreview succeeded [Detail] [Expand]
    ️️✔️SDK Breaking Change Tracking succeeded [Detail] [Expand]

    Breaking Changes Tracking

    ️❌ azure-sdk-for-net-track2 failed [Detail]
    • Code Generator Failed [Logs] Generate from cb84cb2190da6ed99e5c875b0a68dd22b6e1a9e1. SDK Automation 14.0.0
      command	pwsh ./eng/scripts/Automation-Sdk-Init.ps1 ../azure-sdk-for-net_tmp/initInput.json ../azure-sdk-for-net_tmp/initOutput.json
      command	pwsh ./eng/scripts/Invoke-GenerateAndBuildV2.ps1 ../azure-sdk-for-net_tmp/generateInput.json ../azure-sdk-for-net_tmp/generateOutput.json
      cmderr	[Invoke-GenerateAndBuildV2.ps1] �[31;1mException: �[0m/mnt/vss/_work/1/s/azure-sdk-for-net/eng/scripts/Invoke-GenerateAndBuildV2.ps1:133
      cmderr	[Invoke-GenerateAndBuildV2.ps1] �[36;1mLine |
      cmderr	[Invoke-GenerateAndBuildV2.ps1] �[36;1m 133 | �[0m             �[36;1mthrow "Not provide service name or namespace."�[0m
      cmderr	[Invoke-GenerateAndBuildV2.ps1] �[36;1m     | �[31;1m             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      cmderr	[Invoke-GenerateAndBuildV2.ps1] �[31;1m�[36;1m     | �[31;1mNot provide service name or namespace.
      cmderr	[Invoke-GenerateAndBuildV2.ps1] �[0m
      error	Script return with result [failed] code [1] signal [null] cwd [azure-sdk-for-net]: pwsh ./eng/scripts/Invoke-GenerateAndBuildV2.ps1
      warn	Skip package processing as generation is failed
    Posted by Swagger Pipeline | How to fix these errors?

    @openapi-pipeline-app
    Copy link

    Swagger pipeline started successfully. If there is ApiView generated, it will be updated in this comment.

    @ghost ghost added the Cognitive Services label Feb 3, 2023
    @trrwilson
    Copy link
    Member Author

    @jpalvarezl @m-nash FYI -- came across this in the course of ad hoc testing. I'll add less ad hoc test coverage for it once it makes its way over.

    @AzureRestAPISpecReview AzureRestAPISpecReview added the Cadl Authored with Cadl label Feb 3, 2023
    @jpalvarezl jpalvarezl merged commit 31312c5 into Azure:feature/cognitiveservices/openai-cadl Feb 3, 2023
    lmazuel added a commit that referenced this pull request Feb 3, 2023
    * Initial commit.
    
    * Commit OpenAI inference cadl files.
    
    * use non-deprecated service decorator
    
    * Fixes to main
    
    * Reverted changes
    
    * use latest cadl to sync with vs code properly
    
    * cadl compilation works for authoring (but it is correct?)
    
    * Cleaned up routes and model for Inference REST CADL spec
    
    * Added cadl-output to .gitignore file and removed error response
    
    * Implemented polymorphic return for endpoint
    
    * enable Files_Import as non-standard operation, use standard operation names
    
    * Authoring CADL now emits openAPI schema almost identical to current Authoring schema
    
    * Empty model used for polymorphic expectation in embedding endpoint
    
    * Renamed models to match convention
    
    * Renamed completions related models
    
    * Adjusted comment for embeddings body types
    
    * Split up models.cadl into completions and embeddings files
    
    * Renamed models
    
    * Refactored model names
    
    * Removed redundants bits of model names
    
    * added headers
    
    * Added header to correct model
    
    * break up monolithic routes and model cadl files
    
    * add namespace to all models
    
    * Removed gitIgnore file and changed files to not have domain related namespaces
    
    * Finished embeddings endpoint and verified against source json
    
    * Verified round trip for swagger
    
    * Almost done with security types
    
    * Configured correctly OAth2
    
    * Add Auth and OAuth2 to service (thanks Jose!)
    
    * Add auth and OAuth to openapi json
    
    * Review comments
    
    * Review comments
    
    * camel casing
    
    * add versionedDependency
    
    * better versioning
    
    * Added the apiview emitter to OpenAI.Inference endpoints
    
    * PR feedback for Inference endpoints
    
    * Feature/cognitiveservices/openai cadl (#21574)
    
    * Added openai specific list API op template.
    
    * Simplified OAI authoring cadl
    
    * update package versions, correct camelCase warning in common
    
    * remove commented out models, changed AcceptedResponse to CreatedResponse
    
    * Reverted completion to use snake_casing and removed the Response and Request namespaces
    
    * reverted to snake_casing and removed Request namespace for embeddings
    
    * Added missing namespace declaration in embeddings
    
    * Updated dependencies and enabled csharp emitter for Inference
    
    * use string literals for now as enum value types (TypeDiscriminatorKnownValues) not supported in csharp emitter
    
    * Add TODO to TypeDiscriminatorKnownValues def
    
    * More specific route operation names, use snake casing for all model fields
    
    * Added more options to csharp emitter
    
    * Simplify OpenAI inferrence models and use core (#21609)
    
    * Simplify OpenAI inferrence models and use core
    
    * Simplified and corrected Cadl type usage for OpenAI.Inference
    
    * Fix camel case issues.
    
    Co-authored-by: tjprescott <trpresco@microsoft.com>
    
    * Upgrade Cadl file to 0.38 for OpenAI.Authoring service  (#22104)
    
    * Upgrade Cadl file to 0.38
    
    * Upgrade the package.json file and relevant cadl decorator
    
    * Embedding in Azure do not support integer (#22091)
    
    * Fix auth in OpenAI CADL (#22191)
    
    * Change aliases to models, renamed model Embedding to EmbeddingItem to avoid conflicts, and Completion types
    
    * Changed prompt into string[][]
    
    * update package name (#22312)
    
    * Archboard feedback (#22315)
    
    * Archboard feedback
    
    * Disable stream for now
    
    * Rename to options
    
    * Fix operation name (#22319)
    
    * Feature/openai add missing usage (#22337)
    
    * add new API version and usage field on completions and embeddings
    
    * typo
    
    * Added docs for Usage classes in completions and embeddings
    
    * Added versioning mark for new usage model
    
    * Added disambiguation for names and removed unnecessary versioning decorator
    
    * Removed unused version bump
    
    * Removed unused version from enum
    
    * Removed wrong annotation
    
    ---------
    
    Co-authored-by: Jose Alvarez <josealvar@microsoft.com>
    
    * Update main.cadl (#22362)
    
    * Embeddings model and field documentation and fix to models (#22368)
    
    * Moved usage to the response object of embeddings and added some basic documentation to models and fields
    
    * Fixed typo
    
    * Update specification/cognitiveservices/OpenAI.Inference/models/embeddings.create.cadl
    
    ---------
    
    Co-authored-by: Laurent Mazuel <laurent.mazuel@gmail.com>
    
    * Removing Model from model name and removing unnecessary aliases (#22390)
    
    * updated deps and added visibility decorator to deploymentId (#22417)
    
    * Made CompletionsOptions.model optional again
    
    * Removing ...Models namespace in the C# classes (#22449)
    
    * AOAI: fix CompletionsUsage model (#22439)
    
    ---------
    
    Co-authored-by: tjprescott <trpresco@microsoft.com>
    Co-authored-by: Glenn Harper <glharper@microsoft.com>
    Co-authored-by: Jose Alvarez <josealvar@microsoft.com>
    Co-authored-by: Glenn Harper <64209257+glharper@users.noreply.github.com>
    Co-authored-by: Johan Stenberg (MSFT) <johan.stenberg@microsoft.com>
    Co-authored-by: Mary Gao <yanmeigao1210@gmail.com>
    Co-authored-by: m-nash <64171366+m-nash@users.noreply.github.com>
    Co-authored-by: Jose Alvarez <jpalvarezl@users.noreply.github.com>
    Co-authored-by: Travis Wilson <35748617+trrwilson@users.noreply.github.com>
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    Cadl Authored with Cadl Cognitive Services
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    3 participants