Merge branch 'master' of https://github.com/Azure/azure-sdk-for-python …

…into protocol_base

* 'master' of https://github.com/Azure/azure-sdk-for-python: (229 commits)
  Sync eng/common directory with azure-sdk-tools for PR 1688 (#19272)
  update all ubuntu vmImage to 20.04 (#19227)
  add base class for feedback (#19265)
  Enable tests for integration samples (#18531)
  docs: fix a few simple typos (#19127)
  Increment package version after release of azure-eventgrid (#19197)
  Increment package version after release of azure-monitor-query (#19208)
  earliest versions of communication identity tests are set up improperly. skip 1.0.0 in regression testing (#19258)
  Run mypy in azure-keyvault-administration CI (#19246)
  [text analytics] change return type of analyze actions to list of list (#18994)
  Increment package version after release of azure-data-tables (#19192)
  suppress min number less than 14 (#19230)
  [Key Vault] Address administration feedback (#19099)
  [textanalytics] remove warning tests (#19217)
  [translation] enable samples to run in CI (#19190)
  update custom_granularity_value to be more than 300 (#19215)
  Increment package version after release of azure-security-attestation (#19212)
  Enabling Before/After Test Customization for Standard CI Pipelines (#19187)
  bump sphinx version to resolve the weird issue with the caption not getting the correct class (#19188)
  [AutoRelease] t2-agrifood-2021-06-09-15946 (#19174)
  ...

.github/CODEOWNERS

@@ -49,6 +49,9 @@
 # PRLabel: %Monitor - Exporter
 /sdk/monitor/azure-monitor-opentelemetry-exporter                    @rakshith91 @lmazuel @lzchen
 
+# PRLabel: %Monitor - Log
+/sdk/monitor/azure-monitor-query                                     @rakshith91
+
 # PRLabel: %Consumption
 /sdk/consumption/                                                    @sandeepnl
 
@@ -128,7 +131,7 @@
 /common/smoketest/                                                   @lmazuel @chlowell @annatisch @rakshith91 @shurd @southpolesteve
 
 # Management Plane
-/**/*mgmt*/     @00Kai0 @msyyc @jsntcy @changlong-liu
+/**/*mgmt*/                                                          @RAY-316 @msyyc 
 
 
 ###########

.gitignore

@@ -92,10 +92,17 @@ venv/
 code_reports
 
 # Azure Storage test credentials
+sdk/storage/azure-storage-blob/tests/_shared/settings_real.py
+sdk/storage/azure-storage-queue/tests/_shared/settings_real.py
+sdk/storage/azure-storage-file-share/tests/_shared/settings_real.py
+sdk/storage/azure-storage-file-datalake/tests/settings_real.py
+
+# The locations below are deprecated - keep to prevent any accidental secrets leakage ==========
 sdk/storage/azure-storage-blob/tests/settings_real.py
 sdk/storage/azure-storage-queue/tests/settings_real.py
 sdk/storage/azure-storage-file-share/tests/settings_real.py
-sdk/storage/azure-storage-file-datalake/tests/settings_real.py
+# ==============================================================================================
+
 *.code-workspace
 sdk/cosmos/azure-cosmos/test/test_config.py
 

SUPPORT.md

@@ -0,0 +1,29 @@
+# Support
+
+## How to file issues and get help  
+
+Customers with an [Azure support plan](https://azure.microsoft.com/support/options/) can open an [Azure support ticket](https://azure.microsoft.com/support/create-ticket/).
+**We recommend this option if your problem requires immediate attention.** 
+
+### Github issues
+We use [GitHub Issues](https://github.com/Azure/azure-sdk-for-python/issues/new/choose) to track bugs, questions, and feature requests. 
+GitHub issues are free, but **response time is not guaranteed.** See [GitHub issues support process](https://devblogs.microsoft.com/azure-sdk/github-issue-support-process/) for more details.
+
+### Community resources
+- Search for similar issues in [our GitHub repository](https://github.com/Azure/azure-sdk-for-python/issues)
+- Ask a question on [StackOverflow](https://stackoverflow.com/questions/tagged/azure-sdk-python) and tag it with azure-sdk-python
+- Share or upvote feature requests on [Feedback Page](https://feedback.azure.com/forums/34192--general-feedback).
+- Take a look at the [Azure SDK blog](https://devblogs.microsoft.com/azure-sdk/).
+- Chat with other community members on [gitter](https://gitter.im/Azure/azure-sdk-for-python?source=orgpage)
+- Ask a question on [Twitter](https://twitter.com/AzureSDK)
+- Ask a question at [Microsoft Q&A](https://docs.microsoft.com/answers/products/azure?WT.mc_id=Portal-Microsoft_Azure_Support&product=all)
+- Ask a question at [Microsoft Tech Community](https://techcommunity.microsoft.com/t5/azure/ct-p/Azure)
+
+### Security bugs
+Security issues and bugs should be reported privately, via email, to the Microsoft Security Response Center(secure@microsoft.com). 
+You should receive a response within 24 hours. 
+Further information, including the MSRC PGP key, can be found in the [Security TechCenter](https://www.microsoft.com/msrc/faqs-report-an-issue?rtc=1)
+
+## Microsoft Support Policy
+
+Please refer to [Azure SDK Support and Lifecycle information](https://azure.github.io/azure-sdk/policies_support.html)

doc/dev/mgmt/generation.md

@@ -33,7 +33,7 @@ autorest readme.md --python --use="@microsoft.azure/autorest.python@~4.0.71" --p
 
 Which means "Generate the Python code for the Swagger mentioned in this readme, using autorest for Pyton v4.0.71 or above (but not v5), do not generate async files, generate multiapi if supported (if not ignore), and assume the package was already generated and it's an update"
 
-In pratical terms, this is not necessary since the Python SDK has the necessary tooling to simplify to just specify the readme.md:
+In practical terms, this is not necessary since the Python SDK has the necessary tooling to simplify to just specify the readme.md:
 
 - Checkout the branch
 - Checkout the RestAPI specs repo

doc/eng_sys_checks.md

@@ -15,13 +15,23 @@ Example PR build:
  - `Analyze` tox envs run during the `Analyze job.
  - `Test <platform>_<pyversion>` runs PR/Nightly tox envs, depending on context.
 
+
+## Targeting a specific package at build time
+
+In both `public` and `internal` projects, all builds allow a filter to be introduced at build time to narrow the set of packages build/tested.
+
+1. Click `Run New` on your target build.
+2. Before clicking `run` against `master` or your target commit, click `Variables` and add a variable. Add variable `BuildTargetingString` with value of a valid glob string.
+   1. For example, setting filter string `azure-mgmt-*` will filter a build to only management packages. A value of `azure-keyvault-secrets` will result in only building THAT specific package.
+3. Once it's set, run the build!
+
 ## Skipping a tox test environment at queue time
 
 All build definitions allow choice at queue time as to which `tox` environments actually run during the test phase.
 
 1. Find your target service `internal` build.
-2. Click `Run New`
-3. Before clicking `run` against `master` or your target commit, click `Variables` and add a variable. The value should be a comma separated list of tox environments that you want to run in the test phase.
+2. Click `Run New`.
+3. Before clicking `run` against `master` or your target commit, click `Variables` and add a variable of name `Run.ToxCustomEnvs`. The value should be a comma separated list of tox environments that you want to run in the test phase.
 4. Once it's set, run the build!
 
 This is an example setting of that narrows the default set from `whl, sdist, depends, latestdependency, minimumdependency`.

eng/.docsettings.yml

@@ -69,6 +69,8 @@ known_content_issues:
   - ['sdk/core/azure-common/README.md', '#4554']
   - ['sdk/core/azure-servicemanagement-legacy/README.md', '#4554']
   - ['sdk/eventgrid/azure-eventgrid/README.md', '#4554']
+  - ['sdk/monitor/azure-monitor-query/README.md', '#4554']
+  - ['sdk/monitor/azure-monitor-query/samples/README.md', '#4554']
   - ['sdk/graphrbac/azure-graphrbac/README.md', '#4554']
   - ['sdk/loganalytics/azure-loganalytics/README.md', '#4554']
   - ['sdk/servicebus/azure-servicebus/README.md', '#4554']
@@ -84,6 +86,7 @@ known_content_issues:
   - ['sdk/monitor/azure-monitor-opentelemetry-exporter/README.md', '#4554']
   - ['sdk/digitaltwins/azure-digitaltwins-core/swagger/README.md', '#4554']
   - ['sdk/textanalytics/azure-ai-textanalytics/swagger/README.md', '#4554']
+  - ['sdk/translation/azure-ai-translation-document/swagger/README.md', '#4554']
   - ['sdk/purview/azure-purview-catalog/swagger/README.md',  '#4554']
   - ['sdk/purview/azure-purview-scanning/swagger/README.md',  '#4554']
   - ['sdk/agrifood/azure-agrifood-farming/swagger/README.md',  '#4554']

eng/common/TestResources/deploy-test-resources.yml

@@ -33,10 +33,12 @@ parameters:
 
 
 steps:
-  - template: /eng/common/TestResources/setup-az-modules.yml
+  - template: /eng/common/pipelines/templates/steps/cache-ps-modules.yml
+
+  - template: /eng/common/TestResources/setup-environments.yml
 
   - pwsh: |
-      eng/common/TestResources/Import-AzModules.ps1
+      eng/common/scripts/Import-AzModules.ps1
 
       $subscriptionConfiguration = @'
         ${{ parameters.SubscriptionConfiguration }}

eng/common/TestResources/remove-test-resources.yml

@@ -21,7 +21,7 @@ parameters:
 
 steps:
   - pwsh: |
-      eng/common/TestResources/Import-AzModules.ps1
+      eng/common/scripts/Import-AzModules.ps1
 
       $subscriptionConfiguration = @"
         ${{ parameters.SubscriptionConfiguration }}

eng/common/TestResources/setup-environments.yml

@@ -0,0 +1,34 @@
+# Cloud Configuration will be splat into parameters of `Add-AzEnvironment`. It
+# should be JSON in the form (not all fields are required):
+# {
+#   "Name": "<environment name>",
+#   "PublishSettingsFileUrl": "<publish settings file url>",
+#   "ServiceEndpoint": "<service endpoint>",
+#   "ManagementPortalUrl": "<management portal url>",
+#   "ActiveDirectoryEndpoint": "<active directory endpoint>",
+#   "ActiveDirectoryServiceEndpointResourceId": "<active directory service endpoint resource id>",
+#   "ResourceManagerEndpoint": "<resource manager endpoint>",
+#   "GalleryEndpoint": "<gallery endpoint>",
+#   "GraphEndpoint": "<graph endpoint>",
+#   "GraphAudience": "<graph audience>",
+#   "AzureKeyVaultDnsSuffix": "<key vault suffix>",
+#   "AzureKeyVaultServiceEndpointResourceId": "<key vault service endpoint resource id>"
+# }
+
+steps:
+  - bash: sudo chown -R runner ~/.Azure
+    displayName: (MacOS) Grant access to ~/.Azure
+    condition: contains(variables['OSVmImage'], 'mac')
+
+  - task: Powershell@2
+    displayName: Register Dogfood environment
+    inputs:
+      targetType: inline
+      pwsh: true
+      script: |
+        eng/common/scripts/Import-AzModules.ps1
+
+        $environmentSpec = @"
+          $(env-config-dogfood)
+        "@ | ConvertFrom-Json -AsHashtable;
+        Add-AzEnvironment @environmentSpec

eng/common/pipelines/templates/jobs/archetype-sdk-tests-generate.yml

@@ -35,7 +35,7 @@ parameters:
 # When that occurs, provide a name other than the default value.
 - name: GenerateJobName
   type: string
-  default: 'generate_matrix'
+  default: 'generate_job_matrix'
 
 jobs:
 - job: ${{ parameters.GenerateJobName }}
@@ -44,7 +44,6 @@ jobs:
   pool:
     name: ${{ parameters.Pool }}
     vmImage: ${{ parameters.OsVmImage }}
-  displayName: Generate Job Matrix
   ${{ if parameters.DependsOn }}:
     dependsOn: ${{ parameters.DependsOn }}
   steps: