Component | Build Status |
---|---|
Management Libraries |
- Install Visual Studio 2022 (Community or higher) and make sure you have the latest updates (https://www.visualstudio.com/).
- Need at least .NET Framework 4.6.1 and 4.7 development tools
- Install the .NET Core cross-platform development workloads in VisualStudio
- Install .NET Core 6.0.100 SDK for your specific platform. (or a higher version within the 6.0.*** band) (https://dotnet.microsoft.com/download/dotnet-core/6.0)
- Install the latest version of git (https://git-scm.com/downloads)
- Install PowerShell, version 6 or higher, if you plan to make public API changes or are working with generated code snippets.
- Install NodeJS (16.x.x) if you plan to use C# code generation.
Client Libraries are sdks used to interact with azure resources at application run time while Management Libraries are those used to manage (create/modify/delete) Azure resources.
Build Repo : To build all Client and Management libraries together. Invoke dotnet build build.proj
from the root of the repo.
To scope to a single service supply a scope property e.g. dotnet build build.proj /p:Scope=servicebus
. This will build both client and management projects in the specified service. If using msbuild you must run restore first. See below for how to build the client or management libraries independently.
Path Length : To account for the 260 characters Path Length Limit encountered on windows OS, file paths within the repo is keep below 210 characters. This gives you a runway of 49 characters as clone path for your repo. Paths longer that 260 characters will cause the build to break on windows OS and on CI. Assuming you clone to the default VisualStudio location such that the root of your clone is C:\Users\\**USERNAME**\Source\Repos\azure-sdk-for-net
your username will have to be 9 characters long to avoid build errors caused by long paths. Consider using C:\git
as you clone path.
Dependencies : To ensure that the same versions of dependencies are used for all projects in the repo, package versions are managed from a central location in eng\Packages.Data.props
. When adding package references you should first ensure that an Update reference of the package with the version exist in the Packages.Data.props then Include the reference without the version in your .csproj. Contact azuresdkengsysteam@microsoft.com if you need to change versions for packages already present in Packages.Data.props
Line Endings : If working on windows OS ensure git is installed with Checkout Windows-style, commit Unix-style
option or core.autocrlf
set to true in git config. If working on Unix based Linux or MacOS ensure git is installed with Checkout as-is, commit Unix-style
option or core.autocrlf
set to input in git config
- Open any solution, eg "SDKs\Compute\Microsoft.Azure.Management.Compute.sln"
- Build solution from Visual Studio
- Open Developer Command Prompt
- From the root directory
- Invoke
msbuild eng\mgmt.proj /p:scope=Compute
Build without any scope will build all management SDK's.
In order to build one package and run it's test
msbuild eng\mgmt.proj /t:CreateNugetPackage /p:scope=Compute
Nuget package will be created in root directory under \artifacts\packages\Debug (default configuration is Debug)
- Build project in Visual Studio.
- Test Explorer window will get populated with tests. Select test and
Run
orDebug
Run e.g. msbuild eng\mgmt.proj /t:"Runtests" /p:Scope=Compute
In the above example RunTests will build and run tests for Compute only or you can use command line CLI:
dotnet test Compute\Microsoft.Azure.Management.Compute\tests\Microsoft.Azure.Management.Tests.csproj
Now you can use the same command on non-windows as above for e.g. on Ubuntu you can do something like below:
dotnet msbuild eng\mgmt.proj /t:Build /p:scope=Compute
dotnet msbuild eng\mgmt.proj /t:RunTests /p:scope=Compute
dotnet msbuild eng\mgmt.proj /t:CreateNugetPackage /p:scope=Compute
dotnet msbuild build.proj /t:Util /p:UtilityName=InstallPsModules
If you want to enable code coverage reporting, on the command line pass /p:CollectCoverage=true
like so:
dotnet tool restore
dotnet test /p:CollectCoverage=true
On developers' machines, you can open index.html
from within the TestResults
directory in each of your test projects.
Coverage reports can also be found in Azure Pipelines on the "Code Coverage" tab after a pull request validation build completes.
All covered projects should have 70% or better test coverage.
By default, all Azure.* libraries are covered, and any project that sets the IsClientLibrary=true
MSBuild property.
To exclude a project, set ExcludeFromCodeCoverage=true
in the project's MSBuild properties before other targets are imported.
Build tools are now downloaded as part of a nuget package under root\restoredPackages\microsoft.internal.netsdkbuild.mgmt.tools
If for any reason there is an update to the build tools, you will then need to first delete directory root\restoredPackages\microsoft.internal.netsdkbuild.mgmt.tools
and re-execute your build command. This will simply get the latest version of build tools.
We have created a dotnet template to make creating new management SDK library easier than ever.
See (README file)[(https://github.com/Azure/azure-sdk-for-net/blob/main/eng/templates/README.md)].
- Open Developer Command Prompt
- Navigate to service directory e.g. "sdk\eventhub"
- Invoke
dotnet build
- or Build the service.proj in the repo root, passing the directory name of the specific service as a property. e.g.
dotnet build eng\service.proj /p:ServiceDirectory=eventhub
- Open any data-plane solution e.g. "sdk\eventhub\Microsoft.Azure.EventHubs.sln"
- Build solution from Visual Studio
- Open Developer Command Prompt
- Navigate to repository root directory
- Invoke
dotnet build eng\service.proj
This repository has been configured with support for Visual Studio Code's dev container extension to make it easier to get started working on code without needing to know about how to setup all the pre-requisites. Configuration for dev containers is contained within the .devcontainer
folder off the root directory.
To get started:
- Install and configure Docker for your platform.
- Install the Remote Development extension into Visual Studio Code.
- Clone the repository to your local workstation.
- Open Visual Studio Code at the root of the reposiory.
- Select "Reopen in Container" when prompted.
After a few moments of initial configuration Visual Studio Code will launch the container with all dependencies (.NET SDK etc) pre-installed.
- Open Developer Command Prompt
- Navigate to service directory e.g. "sdk\eventhub"
- Invoke
dotnet test --filter TestCategory!=Live
(Skips live tests) - or run test against service.proj in the repo root, passing the directory name of the specific service as a property. e.g.
dotnet test eng\service.proj /p:ServiceDirectory=eventhub --filter TestCategory!=Live
- Build.
- Test Explorer window will get populated with tests. Select test and
Run
orDebug
- Open VS 2019 Command Prompt
- Navigate to repository root directory
- Invoke
dotnet test eng\service.proj --filter TestCategory!=Live
Before running or recording live tests you need to create live test resources. Many of the client libraries make use of the Azure Core Test Framework to provide the basis for the live test infrastructure, including the ability to record Live tests so that they can be run without access to Azure resources. The Test Framework documentation provides more context around test recordings and other features.
To run live tests after creating live resources:
- Open Developer Command Prompt
- Navigate to service directory e.g. "sdk\keyvault"
- Invoke
dotnet test
Some live tests may have additional steps for setting up live testing resources. See the CONTRIBUTING.md file for the service you wish to test for additional information or instructions.
In some cases, you might want to test against the latest versions of the client libraries. i.e. version not yet published to nuget. For this scenario you should make use of the UseProjectReferenceToAzureClients
property which when set to true
will switch all package references for client libraries present in the build to project references. This result in testing against the current version of the libraries in the repo. e.g. dotnet test eng\service.proj /p:ServiceDirectory=eventhub --filter TestCategory!=Live /p:UseProjectReferenceToAzureClients=true
If you make public API changes or additions, the eng\scripts\Export-API.ps1
script has to be run to update public API listings. This generates a file in the library's directory similar to the example found in sdk\template\Azure.Template\api\Azure.Template.netstandard2.0.cs
.
Running the script for a project in sdk\tables
would look like this:
eng\scripts\Export-API.ps1 tables
If the specific client library has sample snippets in markdown format, they were most likely created with help of the eng\scripts\Update-Snippets.ps1
script.
Any changes made to the snippet markdown should be done via updating the corresponding C# snippet code and subsequently running the script.
Running the script for a project, for example in sdk\keyvault
, would look like this:
eng\scripts\Update-Snippets.ps1 keyvault
When run, the code regions in the format below (where <snippetName>
is the name of the snippet):
#region Snippet:<snippetName>
//some sample code
string snippet = "some snippet code";
// The snippet updater defines the SNIPPET directive while parsing. You can use #if SNIPPET to filter lines in or out of the snippet.
#if SNIPPET
snippet = "value that would never pass a test but looks good in a sample!";
#else
string ignored = "this code will not appear in the snippet markdown";
#endif
#endregion
will be mapped to any markdown file with a corresponding code region in the format below where the snippet names match:
```C# Snippet:<snippetName>
```
See the following example of a snippet C# file and a snippet markdown file. Note that snippet names need to be globally unique under a given service directory.
Snippets also can be integrated into XML doc comments. For example:
/// <summary>
/// Some Property.
/// </summary>
/// <example>
/// This is an example of using a snippet in XML docs.
/// <code snippet="Snippet:<snippetName>">
/// // some sample code.
/// string snippet = "some snippet code";
/// </code>
public string SomeProperty { get; set; }
For general information about samples, see the Samples Guidelines
You can run eng\scripts\Export-API.ps1
and eng\scripts\Update-Snippets.ps1
simultaneously as part of the build by setting as true either:
- The property
UpdateSourceOnBuild
- The Environment variable
AZURE_DEV_UPDATESOURCESONBUILD=true
e.g.
dotnet build eng\service.proj /p:ServiceDirectory=eventhub /p:UpdateSourceOnBuild=true
.NET is using the ApiCompat tool to enforce API compatibility between versions. Builds of GA'ed libraries will fail locally and in CI if there are breaking changes.
Each library needs to provide a ApiCompatVersion
property which is set to the last GA'ed version of the library that will be used to compare APIs with the current to ensure no breaks have been introduced. Projects with this property set will download the specified package and the ApiCompat (Microsoft.DotNet.ApiCompat) tools package as part of the restore step of the project. Then as a post build step of the project it will run ApiCompat to verify the current APIs are compatible with the last GA'ed version of the APIs. For libraries that wish to disable the APICompat check they can remove the ApiCompatVersion
property from their project. Our version bump automation will automatically add or increment the ApiCompatVersion
property to the project when it detects that the version it is changing was a GA version which usually indicates that we just shipped that GA version and so it should be the new baseline for API checks.
Since the eng/Packages.Data.props is currently maintained manually, you will need to update the version number for your library in this file when releasing a new version.
The Azure SDK for .NET releases packages daily from our CI pipeline to our NuGet package dev feed to help developers use and test new libraries before they are officially released to NuGet.
Dev Feed Package Browser:
Dev Feed Package Source:
You have multiple options for referencing the dev feed. You can either add it via the NuGet CLI or manually edit your NuGet.Config file.
You can add the dev feed using the NuGet CLI, which will add it to the NuGet.Config file.
nuget sources add -Name "Azure SDK for .NET Dev Feed" -Source "https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-net/nuget/v3/index.json"
You can then view the list of NuGet package sources with this command:
nuget sources
You can add the dev feed to your NuGet.Config file, which can be at the Solution, User, or Computer level. See NuGet.Config file locations and uses to locate your NuGet.Config file.
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="Azure SDK for .NET Dev Feed" value="https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-net/nuget/v3/index.json" />
</packageSources>
<disabledPackageSources>
<clear />
</disabledPackageSources>
</configuration>
You can place a NuGet.Config file in the root of your solution. Projects within the solution will use the feed defined in that file.
You can use the following options to find the available dev feed packages:
- Search the Azure SDK for .NET Dev Feed: https://dev.azure.com/azure-sdk/public/_packaging?_a=feed&feed=azure-sdk-for-net
- In Visual Studio, use the Package Manager UI, be sure to check "Include prerelease".
- Use the NuGet CLI, for example
nuget list azure.identity -Prerelease -Allversions
Now that you have found the package you want to use, it is time to add it to your project file.
As you can see in the example below, we want to use the Azure.Data.Tables
version 3.0.0-alpha.*
. By using the *
in the version number each dotnet restore
will pull the latest version from the dev feed.
<Project Sdk="Microsoft.NET.Sdk">
<ItemGroup>
<PackageReference Include="Azure.Data.Tables" Version="3.0.0-alpha.*" />
</ItemGroup>
</Project>
To prepare a package for release you should make use of .\eng\common\scripts\Prepare-Release.ps1
script passing it appropriate arguments for the package intended for release. This script will correctly update the package version and changelog in the repo as well as update the DevOps release work items for that release.
.\eng\common\scripts\Prepare-Release.ps1 <PackageName> [<ServiceDirectory>] [<ReleaseDate>] [-ReleaseTrackingOnly]
<PackageName>
- Should match the full exact package name for the given ecosystem (i.e. "Azure.Core", "azure-core", "@azure/core", etc).<SerivceDirectory>
- Optional: Should be the exact directory name where the package resides in the repo. This is usually the same as the service name in most cases (i.e. "sdk<service_directory>" e.g. "core"). The parameter is optional and if provided will help speed-up the number of projects we have to parse to find the matching package project.<ReleaseDate>
- Optional: provide a specific date for when you plan to release the package. If one isn't given then one will be calculated based on the normal monthly shipping schedule.<ReleaseTrackingOnly>
- Optional: Switch that if passed will only update the release tracking data in DevOps and not update any versioning info or do validation in the local repo.
In sdk\< Service Name >
, you will find projects for services that have already been implemented
- Client library projects needs to use the $(RequiredTargetFrameworks) (defined in eng/Directory.Build.Data.props) property in its TargetFramework while management library projects should use $(SdkTargetFx) (defined in AzSdk.reference.props)
- Projects of related packages are grouped together in a folder following the structure specified in Repo Structure
- Client library packages are in a folder name like Microsoft.Azure.< ServiceName >
- Management library packages are in a folder named like Microsoft.Azure.Management.< Resource Provider Name >
- Each shipping package contains a project for their generated and /or Customization code
- The folder 'Generated' contains the generated code
- The folder 'Customizations' contains additions to the generated code - this can include additions to the generated partial classes, or additional classes that augment the SDK or call the generated code
- The file generate.cmd, used to generate library code for the given package, can also be found in this project
- Create fork of Azure REST API Specs
- Create fork of Azure SDK for .NET
- Create your Swagger specification for your HTTP API. For more information see Introduction to Swagger - The World's Most Popular Framework for APIs
- Install the latest version of AutoRest and use it to generate your C# client. For more info on getting started with AutoRest, see the AutoRest repository
- Create a branch in your fork of Azure SDK for .NET and add your newly generated code to your project. If you don't have a project in the SDK yet, look at some of the existing projects and build one like the others.
- MANDATORY: Add or update tests for the newly generated code.
- Once added to the Azure SDK for .NET, build your local package using client or management library instructions shown in the above sections.
- For management libraries run
eng\scripts\Update-Mgmt-Yml.ps1
to update PR include paths ineng\pipelines\mgmt.yml
- A Pull request of your Azure SDK for .NET changes against master branch of the Azure SDK for .NET
- The pull requests will be reviewed and merged by the Azure SDK team
- If you have never created an SDK for your service before, you will need the following things to get your SDK in the repo
- Follow the standard process described above.
- Project names helps in using basic heuristics in finding projects as well it's associated test projects during CI process.
- Create a new directory using the name of your service as specified in azure-rest-api-specs/specification Repo
- Follow the the directory structure below
sdk\<service name>\<package name>\README.md
sdk\<service name>\<package name>\*src*
sdk\<service name>\<package name>\*tests*
sdk\<service name>\<package name>\*samples*
e.g.
sdk\eventgrid\Microsoft.Azure.EventGrid\src\Microsoft.Azure.EventGrid.csproj
sdk\eventgrid\Microsoft.Azure.EventGrid\tests\Microsoft.Azure.EventGrid.Tests.csproj
sdk\eventgrid\Microsoft.Azure.Management.EventGrid\src\Microsoft.Azure.Management.EventGrid.csproj
sdk\eventgrid\Microsoft.Azure.Management.EventGrid\tests\Microsoft.Azure.Management.EventGrid.Tests.csproj
Ensure that your service name is the same as it is specified in the azure-rest-api-specs/specification Repo, that your csproj files starts with Microsoft.Azure , that test files end with .Tests and that management plane project files contain .Management. If you are adding a new service directory, ensure that it is mapped to a friendly name at ServiceMapping
-
Copy .csproj from any other .csproj and update the following information in the new .csproj
Project Properties AssemblyTitle Description VersionPrefix PackageTags PackageReleaseNotes
PackageReleaseNotes are important because this information is displayed on https://www.nuget.org when your nuget package is published
- Copy existing generate.ps1 file from another service and update the
ResourceProvider
name that is applicable to your SDK. Resource provider refers to the relative path of your REST spec directory in Azure-Rest-Api-Specs repository During SDK generation, this path helps to locate the REST API spec from thehttps://github.com/Azure/azure-rest-api-specs
-
Install templates for both data-plane and management-plane (control-plan) SDKs:
# Data-plane SDK dotnet new --install sdk/template dotnet new azuresdk --name Azure.MyService --output sdk/myservice --ServiceDirectory myservice --ProjectName Azure.MyService # Management-plane SDK dotnet new --install eng/templates/Azure.ResourceManager.Template dotnet new azuremgmt --help
There are several options available for management-plane SDKs. You can see all those available with
--help
as shown above, or read about them in our documentation.This will perform most of the renames, namespace fix-ups, etc., for you automatically; though, be sure to check all files - especially the README.md file(s) - for required manual changes. If the template is already installed, this same command will upgrade it.
-
Modify
autorest.md
to point to you Swagger file or central README.md file. E.g.input-file: - https://raw.githubusercontent.com/Azure/azure-rest-api-specs/master/specification/storage/resource-manager/Microsoft.Storage/stable/2019-06-01/blob.json - https://raw.githubusercontent.com/Azure/azure-rest-api-specs/master/specification/storage/resource-manager/Microsoft.Storage/stable/2019-06-01/file.json - https://raw.githubusercontent.com/Azure/azure-rest-api-specs/master/specification/storage/resource-manager/Microsoft.Storage/stable/2019-06-01/storage.json
require: https://github.com/Azure/azure-rest-api-specs/blob/49fc16354df7211f8392c56884a3437138317d1f/specification/azsadmin/resource-manager/storage/readme.md
-
Run
dotnet build /t:GenerateCode
in src directory of the project (e.g.net\sdk\storage\Azure.Management.Storage\src
). This would run AutoRest and generate the code. (NOTE: this step requires Node 14). -
For management plan libraries add
azure-arm: true
setting toautorest.md
client constructors and options would be auto-generated. For data-plane libraries follow the next two steps. -
Add a
*ClientOptions
type that inherits fromClientOptions
and has a service version enum:namespace Azure.Management.Storage { public class StorageManagementClientOptions: ClientOptions { private const ServiceVersion Latest = ServiceVersion.V2019_06_01; internal static StorageManagementClientOptions Default { get; } = new StorageManagementClientOptions(); public StorageManagementClientOptions(ServiceVersion serviceVersion = Latest) { VersionString = serviceVersion switch { ServiceVersion.V2019_06_01 => "2019-06-01", _ => throw new ArgumentOutOfRangeException(nameof(serviceVersion)) }; } internal string VersionString { get; } public enum ServiceVersion { #pragma warning disable CA1707 // Identifiers should not contain underscores V2019_06_01 = 1 #pragma warning restore CA1707 } } }
-
Add public constructors to all the clients using a partial class.
public partial class FileSharesClient { public FileSharesClient(string subscriptionId, TokenCredential tokenCredential): this(subscriptionId, tokenCredential, StorageManagementClientOptions.Default) { } public FileSharesClient(string subscriptionId, TokenCredential tokenCredential, StorageManagementClientOptions options): this(new ClientDiagnostics(options), ManagementClientPipeline.Build(options, tokenCredential), subscriptionId, apiVersion: options.VersionString) { } }
Before a pull request will be considered by the Azure SDK team, the following requirements must be met:
- Prior to issuing the pull request:
- All code must have completed any necessary legal signoff for being publicly viewable (Patent review, JSR review, etc.)
- The changes cannot break any existing functional/unit tests that are part of the central repository.
- This includes all tests, even those not associated with the given feature area.
- Code submitted must have basic unit test coverage, and have all the unit tests pass. Testing is the full responsibility of the service team
- Functional tests are encouraged, and provide teams with a way to mitigate regressions caused by other code contributions.
- Code should be commented.
- Code should be fully code reviewed.
- Code should be able to merge without any conflicts into the dev branch being targeted.
- Code should pass all relevant static checks and coding guidelines set forth by the specific repository.
- All build warnings and code analysis warnings should be fixed prior to submission.
- As part of the pull request (aka, in the text box on GitHub as part of submitting the pull request):
- Proof of completion of the code review and test passes requirements above.
- Identity of QA responsible for feature testing (can be conducted post-merging of the pull request).
- Short description of the payload of pull request.
- After the pull request is submitted:
- Our SLA is 48 hours. When your PR is submitted someone on our team will be auto assigned the PR for review. No need to email us
- MS internal folks, please reach out to us via our Teams channel or
- Send an email to the Azure Developer Platform team adpteam@microsoft.com alias.
- Include all interested parties from your team as well.
- In the message, make sure to acknowledge that the legal signoff process is complete.
Once all of the above steps are met, the following process will be followed:
- A member of the Azure SDK team will review the pull request on GitHub.
- If the pull request meets the repository's requirements, the individual will approve the pull request, merging the code into the dev branch of the source repository.
- The owner will then respond to the email sent as part of the pull request, informing the group of the completion of the request.
- If the request does not meet any of the requirements, the pull request will not be merged, and the necessary fixes for acceptance will be communicated back to the partner team.
- If you disagree with the overall approach of the PR, comment on the general PR rather than individual lines of code.
- Leaving suggested changes is welcomed, but please never commit them for a PR you did not create.
- When you are seeking to understand something rather than request corrections, it is suggested to use language such as "I'm curious ..." as a prefix to comments.
- For comments that are just optional suggestions or are explicitly non-blocking, prefix them with "nit: " or "non-blocking: ".
- Avoid marking a PR as "Request Changes" unless you have serious concerns that should block the PR from merging.
- When to mark a PR as "Approved"
- You feel confident that the code meets a high quality bar, has adequate test coverage, is ready to merge.
- You have left comments that are uncontroversial and there is a shared understanding with the author that the comments can be addressed or resolved prior to being merged without significant discussion or significant change to the design or approach.
- When to leave comments without approval
- You do not feel confident that your review alone is sufficient to call the PR ready to merge.
- You have feedback that may require detailed discussion or may indicate a need to change the current design or approach in a non-trivial way.
- When to mark a PR as "Request Changes"
- You have significant concerns that must be addressed before this PR should be merged such as unintentional breaking changes, security issues, or potential data loss.
- If you add significant changes to a PR after it has been marked approved, please confirm with reviewers that they still approve before merging.
- Please ensure that you have obtained an approval from at least one of the code owners before merging.
- If a reviewer marks your PR as approved along with specific comments, it is expected that those comments will be addressed or resolved prior to merging.
- One exception is when a comment clearly states that the feedback is optional or just a nit
- When in doubt, reach out to the commentor to confirm that they have no concerns with you merging without addressing a comment.
Linux (Ubuntu 16.04) | MacOS 10.13 | Windows Server 2016 | |
---|---|---|---|
.NET Core 2.1 | x | x | x |
Much of the management plane SDK code is generated from metadata specs about the REST APIs. Do not submit PRs that modify generated code. Instead,
- File an issue describing the problem,
- Refer to the the AutoRest project to view and modify the generator, or
- Add additional methods, properties, and overloads to the SDK by adding classes in the 'Customizations' folder of a project
For more information on how we version see Versioning
For information about breaking changes see Breaking Change Rules