Spark Configuration DataPlane Api's Swagger Changes #15085

Bharath550-ms · 2021-07-06T04:48:09Z

Spark Configuration DataPlane Api's Swagger Changes

MSFT employees can try out our new experience at OpenAPI Hub - one location for using our validation tools and finding your workflow.

Changelog

Please ensure to add changelog with this PR by answering the following questions.

What's the purpose of the update?
- new service onboarding
- new API version
- update existing version for new feature
- update existing version to fix swagger quality issue in s360
- Other, please clarify
When you are targeting to deploy new service/feature to public regions? Please provide date, or month to public if date is not available yet.
When you expect to publish swagger? Please provide date, or month to public if date is not available yet.
If it's an update to existing version, please select SDKs of specific language and CLIs that require refresh after swagger is published.
- SDK of .NET (need service team to ensure code readiness)
- SDK of Python
- SDK of Java
- SDK of Js
- SDK of Go
- PowerShell
- CLI
- Terraform
- No, no need to refresh for updates in this PR

Contribution checklist:

I commit to follow the Breaking Change Policy of "no breaking changes"
I have reviewed the documentation for the workflow.
Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.

ARM API Review Checklist

Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.
- Adding new API(s)
- Adding a new API version
- Ensure to copy the existing version into new directory structure for first commit (including refactoring) and then push new changes including version updates in separate commits. This is required to review the changes efficiently.
- Adding a new service
Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.
If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist

If there are following updates in the PR, ensure to request an approval from Breaking Change Review Board as defined in the Breaking Change Policy.

Removing API(s) in stable version
Removing properties in stable version
Removing API version(s) in stable version
Updating API in stable or public preview version with Breaking Change Validation errors
Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.

Please follow the link to find more details on PR review process.

Spark Configuration DataPlane Api's Swagger Changes

openapi-workflow-bot · 2021-07-06T04:48:11Z

Hi, @Bharath550-ms Thanks for your PR. I am workflow bot for review process. Here are some small tips.

Please ensure to do self-check against checklists in first PR comment.

PR assignee is the person auto-assigned and responsible for your current PR reviewing and merging.

For specs comparison cross API versions, Use API Specs Comparison Report Generator

If there is CI failure(s), to fix CI error(s) is mandatory for PR merging; or you need to provide justification in PR comment for explanation. How to fix?

Any feedback about review process or workflow bot, pls contact swagger and tools team. vsswagger@microsoft.com

openapi-workflow-bot · 2021-07-06T04:48:11Z

[Call for Action] To better understand Azure service dev/test scenario, and support Azure service developer better on Swagger and REST API related tests in early phase, please help to fill in with this survey https://aka.ms/SurveyForEarlyPhase. It will take 5 to 10 minutes. If you already complete survey, please neglect this comment. Thanks.

openapi-pipeline-app · 2021-07-06T04:48:12Z

Swagger Validation Report

️❌BreakingChange: 3 Errors, 0 Warnings failed [Detail]

Compared Swaggers (Based on Oad v0.8.11)
- current:preview/2021-06-01-preview/entityTypes compared with base:preview/2021-06-01-preview/entityTypes
- current:preview/2021-06-01-preview/artifacts.json compared with base:preview/2021-06-01-preview/artifacts.json

Rule	Message
`❌` 1038 - AddedPath	The new version is adding a path that was not found in the old version. New: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L2901:5
`❌` 1038 - AddedPath	The new version is adding a path that was not found in the old version. New: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L2937:5
`❌` 1038 - AddedPath	The new version is adding a path that was not found in the old version. New: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L3080:5

️⚠️LintDiff: 6 Warnings warning [Detail]

Linted configuring files (Based on source branch, openapi-validator v1.10.0 , classic-openapi-validator v1.1.8 )
- synapse/data-plane/readme.md tag:package-artifacts-2021-06-01-preview
Linted configuring files (Based on target branch, openapi-validator v1.10.0 , classic-openapi-validator v1.1.8 )
- synapse/data-plane/readme.md tag:package-artifacts-2021-06-01-preview

The following errors/warnings are introduced by current PR:

Rule	Message
⚠️ R2001 - AvoidNestedProperties	Consider using x-ms-client-flatten to provide a better end user experience Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L3390
⚠️ R2063 - OperationIdNounConflictingModelNames	OperationId has a noun that conflicts with one of the model names in definitions section. The model name will be disambiguated to 'SparkConfigurationModel'. Consider using the plural form of 'SparkConfiguration' to avoid this. Note: If you have already shipped an SDK on top of this spec, fixing this warning may introduce a breaking change. Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L2906
⚠️ R2063 - OperationIdNounConflictingModelNames	OperationId has a noun that conflicts with one of the model names in definitions section. The model name will be disambiguated to 'SparkConfigurationModel'. Consider using the plural form of 'SparkConfiguration' to avoid this. Note: If you have already shipped an SDK on top of this spec, fixing this warning may introduce a breaking change. Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L2942
⚠️ R2063 - OperationIdNounConflictingModelNames	OperationId has a noun that conflicts with one of the model names in definitions section. The model name will be disambiguated to 'SparkConfigurationModel'. Consider using the plural form of 'SparkConfiguration' to avoid this. Note: If you have already shipped an SDK on top of this spec, fixing this warning may introduce a breaking change. Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L3000
⚠️ R2063 - OperationIdNounConflictingModelNames	OperationId has a noun that conflicts with one of the model names in definitions section. The model name will be disambiguated to 'SparkConfigurationModel'. Consider using the plural form of 'SparkConfiguration' to avoid this. Note: If you have already shipped an SDK on top of this spec, fixing this warning may introduce a breaking change. Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L3045
⚠️ R2063 - OperationIdNounConflictingModelNames	OperationId has a noun that conflicts with one of the model names in definitions section. The model name will be disambiguated to 'SparkConfigurationModel'. Consider using the plural form of 'SparkConfiguration' to avoid this. Note: If you have already shipped an SDK on top of this spec, fixing this warning may introduce a breaking change. Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L3085

The following errors/warnings exist before current PR submission:

Only 10 items are listed, please refer to log for more details.

Rule	Message
⚠️ R2001 - AvoidNestedProperties	Consider using x-ms-client-flatten to provide a better end user experience Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L3916
⚠️ R2001 - AvoidNestedProperties	Consider using x-ms-client-flatten to provide a better end user experience Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L4037
⚠️ R2001 - AvoidNestedProperties	Consider using x-ms-client-flatten to provide a better end user experience Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L4055
⚠️ R2001 - AvoidNestedProperties	Consider using x-ms-client-flatten to provide a better end user experience Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L4102
⚠️ R2001 - AvoidNestedProperties	Consider using x-ms-client-flatten to provide a better end user experience Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L4120
⚠️ R2001 - AvoidNestedProperties	Consider using x-ms-client-flatten to provide a better end user experience Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L4172
⚠️ R2001 - AvoidNestedProperties	Consider using x-ms-client-flatten to provide a better end user experience Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L4211
⚠️ R2001 - AvoidNestedProperties	Consider using x-ms-client-flatten to provide a better end user experience Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L4263
⚠️ R2001 - AvoidNestedProperties	Consider using x-ms-client-flatten to provide a better end user experience Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L4282
⚠️ R2001 - AvoidNestedProperties	Consider using x-ms-client-flatten to provide a better end user experience Location: Microsoft.Synapse/preview/2021-06-01-preview/artifacts.json#L4419

️️✔️Avocado succeeded [Detail] [Expand]

Validation passes for Avocado.

️️✔️ModelValidation succeeded [Detail] [Expand]

Validation passes for ModelValidation.

️️✔️SemanticValidation succeeded [Detail] [Expand]

Validation passes for SemanticValidation.

️️✔️Cross-Version Breaking Changes succeeded [Detail] [Expand]

There are no breaking changes.

️️✔️CredScan succeeded [Detail] [Expand]

There is no credential detected.

️⚠️[Staging] SDK Track2 Validation: 2 Warnings warning [Detail]

The following tags are being changed in this PR

synapse/data-plane/readme.md#package-artifacts-2021-06-01-preview

The following errors/warnings are introduced by current PR:

Rule	Message
⚠️ Modeler/MissingType	"readme":"synapse/data-plane/readme.md", "tag":"package-artifacts-2021-06-01-preview", "details":"The schema 'NotebookCellOutputItem-text' has no type or format information whatsoever. Location:\n file:///home/vsts/work/1/azure-rest-api-specs/specification/synapse/data-plane/Microsoft.Synapse/preview/2021-06-01-preview/entityTypes/Notebook.json#/components/schemas/NotebookCellOutputItem-text"
⚠️ Modeler/MissingType	"readme":"synapse/data-plane/readme.md", "tag":"package-artifacts-2021-06-01-preview", "details":"The schema 'SynapseSparkJobActivityTypeProperties-argsItem' has no type or format information whatsoever. Location:\n file:///home/vsts/work/1/azure-rest-api-specs/specification/synapse/data-plane/Microsoft.Synapse/preview/2021-06-01-preview/entityTypes/Pipeline.json#/components/schemas/SynapseSparkJobActivityTypeProperties-argsItem"
💬 AutorestCore/Exception	"readme":"synapse/data-plane/readme.md", "tag":"package-artifacts-2021-06-01-preview", "details":"> Installing AutoRest extension '@microsoft.azure/openapi-validator' (1.8.0)"
💬 AutorestCore/Exception	"readme":"synapse/data-plane/readme.md", "tag":"package-artifacts-2021-06-01-preview", "details":"> Installed AutoRest extension '@microsoft.azure/openapi-validator' (1.8.0->1.8.0)"

The following errors/warnings exist before current PR submission:

|:speech_balloon: AutorestCore/Exception|"readme":"synapse/data-plane/readme.md",
"tag":"package-artifacts-2021-06-01-preview",
"details":"> Loading AutoRest extension '@autorest/modelerfour' (4.15.456->4.15.456)"|

️️✔️[Staging] PrettierCheck succeeded [Detail] [Expand]

Validation passes for PrettierCheck.

️️✔️[Staging] SpellCheck succeeded [Detail] [Expand]

Validation passes for SpellCheck.

️️✔️[Staging] Lint(RPaaS) succeeded [Detail] [Expand]

Validation passes for Lint(RPaaS).

_{Posted by Swagger Pipeline | How to fix these errors?}

openapi-pipeline-app · 2021-07-06T04:48:12Z

Swagger pipeline restarted successfully, please wait for status update in this comment.

ghost · 2021-07-06T04:48:13Z

Thank you for your contribution Bharath550-ms! We will review the pull request and get back to you soon.

ghost · 2021-07-06T04:48:21Z

All CLA requirements met.

openapi-workflow-bot · 2021-07-06T04:48:39Z

Hi, @Bharath550-ms your PR are labelled with WaitForARMFeedback. A notification email will be sent out shortly afterwards to notify ARM review board(armapireview@microsoft.com).

Spark Configuration DataPlane Api's Swagger Changes

…Bharath550-ms/azure-rest-api-specs into SparkConfigurationDataplaneApis

Updated readme file

idear1203 · 2021-07-06T06:33:14Z

specification/synapse/data-plane/Microsoft.Synapse/preview/2019-06-01-preview/artifacts.json

+            "name": "If-None-Match",
+            "in": "header",
+            "required": false,
+            "type": "string",


"type": "string",

Add a line below:
"x-ms-client-name": "IfNoneMatch", #Resolved

specification/synapse/data-plane/Microsoft.Synapse/preview/2019-06-01-preview/artifacts.json

idear1203 · 2021-07-06T06:52:55Z

I have two questions:

Is the API deployment work at server side already done?
Is this a GA feature or a preview feature? Is there any chance we might need to modify the Swagger spec again for this feature?

Fix CI Errors

openapi-workflow-bot · 2021-07-06T07:45:53Z

NewApiVersionRequired reason:

A service’s API is a contract with customers and is represented by using the api-version query parameter. Changes such as adding an optional property to a request/response or introducing a new operation is a change to the service’s contract and therefore requires a new api-version value. This is critically important for documentation, client libraries, and customer support.

EXAMPLE: if a customer calls a service in the public cloud using api-version=2020-07-27, the new property or operation may exist but if they call the service in a government cloud, air-gapped cloud, or Azure Stack Hub cloud using the same api-version, the property or operation may not exist. Because there is no clear relationship between the service api-version and the new property/operation, customers can’t trust the documentation and Azure customer have difficulty helping customers diagnose issues. In addition, each client library version documents the service version it supports. When an optional property or new operation is added to a service and its Swagger, new client libraries must be produced to expose this functionality to customers. Without updating the api-version, it is unclear to customers which version of a client library supports these new features.

Updated readme file

Fixed CI Errors

Fixed comments

Added changes for new preview version

* Spark Configuration DataPlane Api's Swagger Changes Spark Configuration DataPlane Api's Swagger Changes * Spark Configuration DataPlane Api's Swagger Changes Spark Configuration DataPlane Api's Swagger Changes * Updated readme file Updated readme file * Fix CI Errors Fix CI Errors * Update readme.md Updated readme file * Fixed CI Errors Fixed CI Errors * Fixed comments Fixed comments * Added changes for new preview version Added changes for new preview version

Spark Configuration DataPlane Api's Swagger Changes

7092b48

Spark Configuration DataPlane Api's Swagger Changes

Bharath550-ms requested a review from idear1203 as a code owner July 6, 2021 04:48

openapi-assignment-bot bot assigned lmazuel Jul 6, 2021

ghost added the customer-reported Issues that are reported by GitHub users external to the Azure organization. label Jul 6, 2021

ghost added the Synapses label Jul 6, 2021

openapi-workflow-bot bot added the WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required label Jul 6, 2021

Bharath550-ms added 3 commits July 6, 2021 11:18

Spark Configuration DataPlane Api's Swagger Changes

887739e

Spark Configuration DataPlane Api's Swagger Changes

Merge branch 'SparkConfigurationDataplaneApis' of https://github.com/…

e59f215

…Bharath550-ms/azure-rest-api-specs into SparkConfigurationDataplaneApis

Updated readme file

ee683b1

Updated readme file

idear1203 reviewed Jul 6, 2021

View reviewed changes

specification/synapse/data-plane/Microsoft.Synapse/preview/2019-06-01-preview/artifacts.json Outdated Show resolved Hide resolved

Fix CI Errors

44c9d4b

Fix CI Errors

openapi-pipeline-app bot added the NewApiVersionRequired label Jul 6, 2021

Bharath550-ms added 4 commits July 6, 2021 13:19

Update readme.md

adb3b9e

Updated readme file

Fixed CI Errors

c87e74f

Fixed CI Errors

Fixed comments

82c0fd7

Fixed comments

Added changes for new preview version

6a2d3bb

Added changes for new preview version

JeffreyRichter added the Approved-BreakingChange DO NOT USE! OBSOLETE label. See https://github.com/Azure/azure-sdk-tools/issues/6374 label Jul 9, 2021

lmazuel merged commit c7ce6ad into Azure:master Jul 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Spark Configuration DataPlane Api's Swagger Changes #15085

Spark Configuration DataPlane Api's Swagger Changes #15085

Bharath550-ms commented Jul 6, 2021 •

edited

Loading

openapi-workflow-bot bot commented Jul 6, 2021

openapi-workflow-bot bot commented Jul 6, 2021

openapi-pipeline-app bot commented Jul 6, 2021 •

edited

Loading

openapi-pipeline-app bot commented Jul 6, 2021 •

edited

Loading

ghost commented Jul 6, 2021

ghost commented Jul 6, 2021 •

edited by ghost

Loading

openapi-workflow-bot bot commented Jul 6, 2021

idear1203 Jul 6, 2021 •

edited

Loading

Bharath550-ms Jul 6, 2021

idear1203 commented Jul 6, 2021

openapi-workflow-bot bot commented Jul 6, 2021

Spark Configuration DataPlane Api's Swagger Changes #15085

Spark Configuration DataPlane Api's Swagger Changes #15085

Conversation

Bharath550-ms commented Jul 6, 2021 • edited Loading

Changelog

Contribution checklist:

ARM API Review Checklist

Breaking Change Review Checklist

openapi-workflow-bot bot commented Jul 6, 2021

openapi-workflow-bot bot commented Jul 6, 2021

openapi-pipeline-app bot commented Jul 6, 2021 • edited Loading

Swagger Validation Report

openapi-pipeline-app bot commented Jul 6, 2021 • edited Loading

ghost commented Jul 6, 2021

ghost commented Jul 6, 2021 • edited by ghost Loading

openapi-workflow-bot bot commented Jul 6, 2021

idear1203 Jul 6, 2021 • edited Loading

Choose a reason for hiding this comment

Bharath550-ms Jul 6, 2021

Choose a reason for hiding this comment

idear1203 commented Jul 6, 2021

openapi-workflow-bot bot commented Jul 6, 2021

Bharath550-ms commented Jul 6, 2021 •

edited

Loading

openapi-pipeline-app bot commented Jul 6, 2021 •

edited

Loading

openapi-pipeline-app bot commented Jul 6, 2021 •

edited

Loading

ghost commented Jul 6, 2021 •

edited by ghost

Loading

idear1203 Jul 6, 2021 •

edited

Loading