Merge branch 'main' into feature/aoai-2024-07-01-preview-tsp

.github/PULL_REQUEST_TEMPLATE/control_plane_template.md

@@ -17,6 +17,7 @@ What's the purpose of this PR? Check the specific option that applies. This is *
   - [ ] New API version for an existing resource provider. (If API spec is not defined in TypeSpec, the PR should have been created in adherence to [OpenAPI specs PR creation guidance](https://aka.ms/azsdkdocs/createopenapispec)).
   - [ ] Update existing version for a new feature. (This is applicable only when you are revising a private preview API version.)
   - [ ] Update existing version to fix OpenAPI spec quality issues in S360.
+  - [ ] Convert existing [OpenAPI spec to TypeSpec spec](https://aka.ms/typespec/conversion) (do not combine this with implementing changes for a new API version).
   - [ ] Other, please clarify:
     - _edit this with your clarification_
 
@@ -51,7 +52,7 @@ If one or multiple validation error/warning suppression(s) is detected in your P
 ## Getting help
 
 - First, please carefully read through this PR description, from top to bottom. Please fill out the `Purpose of this PR` and `Due diligence checklist`.
-- If you don't have permissions to remove or add labels to the PR, request `write access` per [aka.ms/azsdk/access#request-access-to-rest-api-or-sdk-repositories](https://aka.ms/azsdk/access#request-access-to-rest-api-or-sdk-repositories")
+- If you don't have permissions to remove or add labels to the PR, request `write access` per [aka.ms/azsdk/access#request-access-to-rest-api-or-sdk-repositories](https://aka.ms/azsdk/access#request-access-to-rest-api-or-sdk-repositories)
 - To understand what you must do next to merge this PR, see the `Next Steps to Merge` comment. It will appear within few minutes of submitting this PR and will continue to be up-to-date with current PR state.
 - For guidance on fixing this PR CI check failures, see the hyperlinks provided in given failure 
   and https://aka.ms/ci-fix.

.github/PULL_REQUEST_TEMPLATE/data_plane_template.md

@@ -76,7 +76,7 @@ The [Azure API Review Board](https://aka.ms/azsdk/onboarding/restapischedule) is
 ## Getting help
 
 - First, please carefully read through this PR description, from top to bottom.
-- If you don't have permissions to remove or add labels to the PR, request `write access` per [aka.ms/azsdk/access#request-access-to-rest-api-or-sdk-repositories](https://aka.ms/azsdk/access#request-access-to-rest-api-or-sdk-repositories") 
+- If you don't have permissions to remove or add labels to the PR, request `write access` per [aka.ms/azsdk/access#request-access-to-rest-api-or-sdk-repositories](https://aka.ms/azsdk/access#request-access-to-rest-api-or-sdk-repositories) 
 - To understand what you must do next to merge this PR, see the `Next Steps to Merge` comment. It will appear within few minutes of submitting this PR and will continue to be up-to-date with current PR state.
 - For guidance on fixing this PR CI check failures, see the hyperlinks provided in given failure 
   and https://aka.ms/ci-fix.

.github/workflows/_reusable-eng-tools-test.yaml

@@ -8,6 +8,17 @@ on:
         description: Name of package under eng/tools
         required: true
         type: string
+      sparse-checkout-paths:
+        description: Paths for sparse checkout
+        type: string
+      lint:
+        description: Run 'npm run lint' if true
+        required: false
+        type: boolean
+      prettier:
+        description: Run 'npm run prettier' if true
+        required: false
+        type: boolean
 
 jobs:
   test:
@@ -24,9 +35,15 @@ jobs:
     runs-on: ${{ matrix.os }}
 
     steps:
+      - if: runner.os == 'Windows'
+        run: git config --global core.longpaths true
+        shell: pwsh
+
       - uses: actions/checkout@v4
         with:
-          sparse-checkout: eng
+          sparse-checkout: |
+            eng
+            ${{ inputs.sparse-checkout-paths }}
 
       - name: Use Node ${{ matrix.node-version }}.x
         uses: actions/setup-node@v4
@@ -44,6 +61,16 @@ jobs:
         shell: pwsh
         working-directory: ./eng/tools/${{ inputs.package }}
 
+      - run: npm run lint
+        if: inputs.lint == true
+        shell: pwsh
+        working-directory: ./eng/tools/${{ inputs.package }}
+
+      - run: npm run prettier
+        if: inputs.prettier == true
+        shell: pwsh
+        working-directory: ./eng/tools/${{ inputs.package }}        
+
       - run: npm run test:ci
         shell: pwsh
         working-directory: ./eng/tools/${{ inputs.package }}

.github/workflows/specs-model-test.yaml

@@ -0,0 +1,27 @@
+name: Specs Model - Test
+
+on:
+  push:
+    branches:
+      - main
+      - typespec-next
+  pull_request:
+    paths:
+      - package-lock.json
+      - package.json
+      - tsconfig.json
+      - .github/workflows/_reusable-eng-tools-test.yaml
+      - .github/workflows/specs-model-test.yaml
+      - eng/tools/package.json
+      - eng/tools/tsconfig.json
+      - eng/tools/specs-model/**
+  workflow_dispatch:
+
+jobs:
+  specsModel:
+    name: Specs Model
+    uses: ./.github/workflows/_reusable-eng-tools-test.yaml
+    with:
+      package: specs-model
+      lint: true
+      prettier: true

.github/workflows/tsp-client-test.yaml

@@ -0,0 +1,30 @@
+name: tsp-client - Test
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
+    paths:
+      - package-lock.json
+      - package.json
+      - tsconfig.json
+      - .github/workflows/_reusable-eng-tools-test.yaml
+      - .github/workflows/tsp-client-test.yaml
+      - eng/tools/package.json
+      - eng/tools/tsconfig.json
+      - eng/tools/tsp-client/**
+      - specification/keyvault
+      - specification/sphere
+
+jobs:
+  tsp-client:
+    uses: ./.github/workflows/_reusable-eng-tools-test.yaml
+    with:
+      package: tsp-client-tests
+      sparse-checkout-paths: |
+        specification/common-types
+        specification/keyvault
+        specification/sphere

.gitignore

@@ -116,12 +116,18 @@ warnings.txt
 .apitest
 .assets
 
+# Blanket ignores
 *.js
 *.d.ts
 *.js.map
 *.d.ts.map
 *.bak
 
 # Eng Tools
-eng/tools/typespec-validation/dist
-!eng/tools/typespec-validation/cmd/*.js
+eng/tools/**/dist
+!eng/tools/**/cmd/*.js
+!eng/tools/**/eslint.config.js
+
+# No package-lock.json files should be commited except the top-level.
+**/package-lock.json
+!/package-lock.json

arm-compute/quickstart-templates/aliases.json

@@ -101,13 +101,6 @@
             "sku": "2012-Datacenter",
             "version": "latest",
             "architecture": "x64"
-          },
-          "Win2008R2SP1": {
-            "publisher": "MicrosoftWindowsServer",
-            "offer": "WindowsServer",
-            "sku": "2008-R2-SP1",
-            "version": "latest",
-            "architecture": "x64"
           }
         }
       }

cSpell.json

@@ -398,7 +398,13 @@
             ]
         },
         {
-            "filename": "**/specification/eventgrid/data-plane/Microsoft.EventGrid/stable/2024-01-01/SystemEvents.json",
+            "filename": "**/specification/eventgrid/data-plane/Microsoft.EventGrid/stable/2024-01-01/GeneratedSystemEvents.json",
+            "words": [
+                "whatsapp"
+            ]
+        },
+        {
+            "filename": "**/specification/eventgrid/data-plane/Microsoft.EventGrid/stable/2018-01-01/GeneratedSystemEvents.json",
             "words": [
                 "whatsapp"
             ]
@@ -442,7 +448,8 @@
         {
             "filename": "**/specification/netapp/**/*.json",
             "words": [
-                "SVMs"
+                "SVMs",
+                "kibibytes"
             ]
         },
         {
@@ -1151,6 +1158,15 @@
                 "mpga"
             ]
         },
+        {
+            "filename": "**/specification/cognitiveservices/data-plane/AzureOpenAI/inference/stable/**/*.json",
+            "words": [
+                "flac",
+                "FLAC",
+                "mpga",
+                "rerank"
+            ]
+        },
         {
             "filename": "**/specification/appplatform/resource-manager/Microsoft.AppPlatform/preview/2024-01-01-preview/appplatform.json",
             "words": [
@@ -1169,6 +1185,14 @@
                 "timerange"
             ]
         },
+        {
+            "filename": "**/specification/healthdataaiservices/**/*.json",
+            "words": [
+                "deidentification",
+                "deidentify",
+                "surrogation"
+            ]
+        },
         {
             "filename": "**/specification/search/data-plane/Azure.Search/**/*.json",
             "words": [

eng/.vscode/settings.json

@@ -0,0 +1,14 @@
+{
+  "editor.defaultFormatter": "esbenp.prettier-vscode",
+  "[javascript]": {
+    "editor.defaultFormatter": "esbenp.prettier-vscode"
+  },
+  "editor.formatOnPaste": false,
+  "editor.formatOnType": false,
+  "editor.formatOnSave": true,
+  "editor.formatOnSaveMode": "file", // required to format on save
+  "prettier.requireConfig": true,
+  "cSpell.words": [
+    "tseslint"
+  ]
+}

eng/README.md

@@ -0,0 +1,48 @@
+# `eng` directory
+
+The `eng` directory contains source code for automated tooling running on this repository pull requests.
+
+For context on this directory, see [Design guidelines for spec repos validation tooling] (Microsoft-internal).
+
+## Code conventions
+
+Below are code convention we strive to follow in `eng` directory:
+
+### package.json
+
+- We align `package.json` dependencies versions across all `package.json` files.
+- We align `package.json` dependencies numbers with [microsoft/typespec package.json].
+  In few cases we allow more frequent update cadence.
+- We avoid doing package overrides. For example, we use `v8` of `eslint` instead of `v9` to avoid an override.
+  [See this comment for details][eslint override].
+- We order `package.json` keys as follows: `name private type main bin scripts engines dependencies devDependencies`.
+
+### package-lock.json
+
+- We maintain only top-level `package-lock.json` file. Running `npm install` from top-level dir removes the need
+  to ever have other files.
+- We ensure the lock file remains clean by ensuring that a PR that adds or modifies any `package.json` dependencies,
+  makes changes equivalent to following protocol:
+  - `cd <local-specs-clone-root>`
+  - `git clean -xdf` to remove all untracked files.
+  - Copy-over [`package-lock.json` from `main`] to local clone.
+  - `npm install` to reflect the added or modified dependencies.
+- In case any dependencies have been removed from any `package.json`, we do `rm package-lock.json` and `npm install`.  
+  This way we ensure the lock file remains free of unused dependencies.
+- We do `npm update` only in stand-alone PRs.
+- To avoid conflicting changes when updating `package-lock.json`, only use the latest LTS version of Node, and the bundled version of npm (but no newer).  You can use `nvm install --lts` (Linux) or `nvm install lts` (Windows).
+
+## Linting and prettier
+
+- We make eslint-based linting rules and prettier mandatory in CI for all projects.
+- We use strict rulesets as baseline.
+- We discuss any desired rule divergences from the strict ruleset
+  and apply rule modifications to the configs with explanation for our decision.
+- We align `prettier` rules with [microsoft/typespec .prettierrc.json].
+
+[`package-lock.json` from `main`]: https://github.com/Azure/azure-rest-api-specs/blob/main/package-lock.json
+[Design guidelines for spec repos validation tooling]: https://dev.azure.com/azure-sdk/internal/_wiki/wikis/internal.wiki/1153/Design-guidelines-for-spec-repos-validation-tooling
+[eslint override]: https://github.com/Azure/azure-rest-api-specs/pull/29820#pullrequestreview-2177045580
+[microsoft/typespec .prettierrc.json]: https://github.com/microsoft/typespec/blob/main/.prettierrc.json
+[microsoft/typespec package.json]: https://github.com/microsoft/typespec/blob/main/package.json
+[npm/cli #7384]: https://github.com/npm/cli/issues/7384

eng/tools/package.json

@@ -1,7 +1,9 @@
 {
   "name": "azure-rest-api-specs-eng-tools",
   "devDependencies": {
+    "@azure-tools/specs-model": "file:specs-model",
     "@azure-tools/suppressions": "file:suppressions",
+    "@azure-tools/tsp-client-tests": "file:tsp-client-tests",
     "@azure-tools/typespec-requirement": "file:typespec-requirement",
     "@azure-tools/typespec-validation": "file:typespec-validation"
   },

eng/tools/specs-model/.prettierignore

@@ -0,0 +1,4 @@
+*.json
+*.md
+*.jsonc
+dist

eng/tools/specs-model/.prettierrc.mjs

@@ -0,0 +1,18 @@
+// This config is adapted from https://github.com/microsoft/typespec/blob/main/.prettierrc.json
+// See eng/README.md for context.
+/**
+ * @see https://prettier.io/docs/en/configuration.html
+ * @type {import("prettier").Config}
+ */
+const config = {
+  arrowParens: "always",
+  trailingComma: "es5",
+  bracketSpacing: true,
+  endOfLine: "lf",
+  printWidth: 100,
+  semi: true,
+  singleQuote: false,
+  tabWidth: 2,
+};
+
+export default config;

eng/tools/specs-model/cmd/get-specs-model.js

@@ -0,0 +1,5 @@
+#!/usr/bin/env node
+
+import { main } from "../dist/index.js";
+
+await main();