Skip to content

Latest commit

 

History

History
364 lines (263 loc) · 20.3 KB

azure-powershell-developer-guide.md

File metadata and controls

364 lines (263 loc) · 20.3 KB
Raw

Azure PowerShell Developer Guide

The Azure PowerShell Developer Guide was created to help with the development and testing of Azure PowerShell cmdlets. This guide contains information on how to set up your environment, create a new project, implement cmdlets, record and run tests, and more.

Note: Except for the way described in this page, There is a new way to generate PowerShell modules through AutoRest PowerShell generator. And related links are attached in the end.

Table of Contents

Prerequisites

The following prerequisites should be completed before contributing to the Azure PowerShell repository:

Environment Setup

GitHub Basics

If you don't have experience with Git and GitHub, some of the terminology and process can be confusing. Here is a guide to understanding the GitHub flow and here is a guide to understanding the basic Git commands.

To develop in the Azure PowerShell repository locally, you first need to create your own fork. For more information on how to fork, click here.

Once your fork of the Azure PowerShell repository has been created, you need to clone your fork to your local machine. To do so, run the following command:

git clone https://github.com/<YOUR GITHUB USERNAME>/azure-powershell.git

You now be able to create your own branches, commit changes, and push commits to your fork.

Note: we recommend adding the Azure/azure-powershell repository to your list of tracked repositories in Git. This allows you to easily pull changes from the Azure repository. To do this, run the following command:

git remote add upstream https://github.com/Azure/azure-powershell.git

Then, to pull changes from the main branch in Azure/azure-powershell into your local working branch, run the following command:

git pull upstream main

Building the Environment

Note: to build the environment locally, you need platyPS install on your machine. Please see the Prerequisites section for details on how to install this module.

After cloning the repository to your local machine, you want to ensure that you can build the environment. To do so, launch VS Developer Command Prompt (which is installed with Visual Studio) and run the following command (from the root of the repository) to do a full build:

msbuild build.proj

Alternatively, you can open any command prompt (Command Prompt, Windows PowerShell, or PowerShell 7), navigate to the root of the repository, and run:

PS C:\azure-powershell> dotnet msbuild build.proj

Generating Help

We build the dll-Help.xml files (used to display the help content for cmdlets in PowerShell) from markdown using the platyPS module. Since this help generation step can take 10-15 minutes, it is a separate part of the command line build process. Run this to generate help:

msbuild build.proj /t:GenerateHelp

Running Static Analysis

To keep consistency across our modules, we've implemented a static analysis system. This verifies various aspects (dependencies, breaking changes, etc.) for your module. Run this command to execute static analysis validation for the built modules:

msbuild build.proj /t:StaticAnalysis

Note: this can add 10-15 minutes to your build time due to help generation.

Running Tests

Launch VS Developer Command Prompt and run the following command (from the root of the repository) to run all of the tests in playback:

msbuild build.proj /t:Test

Alternatively, you can open any command prompt (Command Prompt, Windows PowerShell, or PowerShell 7), navigate to the root of the repository, and run:

PS C:\azure-powershell> dotnet msbuild build.proj /t:Test

Before Adding a New Project

.NET SDK

Before adding a new project to Azure PowerShell, you must have generated an SDK for .NET using AutoRest for your service, and it must have been merged into the repository.

For more information about on-boarding a new library in the SDK for .NET repository, click here.

Design Review

Before development, you must meet with the Azure PowerShell team to have a design review for your proposed PowerShell cmdlets. We advise that this review is held no earlier than three weeks out from code complete of the release you want to ship the cmdlets with. For a small number of cmdlet changes and/or additions, an email containing the markdown files for the proposed changes is suggested. For a large number of changes and/or additions, a meeting is required with the Azure PowerShell team.

Before submitting a design review, please be sure that you have read the documents found in the Azure PowerShell Design Guidelines folder.

Please submit a design review here: https://github.com/Azure/azure-powershell-cmdlet-review-pr

Note: You will need to be part of the GitHub Azure org to see this repository. Please go to this link to become part of the Azure org.

We recommend using the platyPS module to easily generate markdown files that contains the above information and including the files in the design submission.

Point of Contact

You must provide a contact (Microsoft alias + GitHub alias) on your team that is responsible for handling issues with your SDK and cmdlets, as well as open source contributions to your code.

Also, members of your team (who are involved with the SDKs) are advised to join the Microsoft Teams Azure SDK team, and join the adxsdkpartners alias on idweb.

Setting Up a New Project

Getting Started

When adding a new project, please follow these guidelines:

Creating the Project

Add a new folder under src with your service specific name (e.g., Compute, Sql, Websites).

We recommend copying an existing module. For example, go to src/Cdn and copy the contents of this folder. Paste these to your service folder you just created. Rename the following:

  • The folders to <SERVICE> and <SERVICE>.Test
  • The solution to <SERVICE>.sln
  • The projects (within each folder) to <SERVICE>.csproj and <SERVICE>.Test.csproj
  • The PSD1 file (in the <SERVICE> folder) to Az.<SERVICE>.psd1

Now, you'll need to edit the solution file. Open the <SERVICE>.sln in your text editor of choice. Edit these lines to use your <SERVICE> name:

  • Update the "<SERVICE>", "<SERVICE>\<SERVICE>.csproj"
  • Update the "<SERVICE>.Test", "<SERVICE>.Test\<SERVICE>.Test.csproj"
  • Note: Leave the "Accounts", "..\Accounts\Accounts\Accounts.csproj" entry as is. All modules depend on Accounts.

After the solution file is updated, save and close it. Now, open the solution file in Visual Studio. Right click on the <SERVICE> project in the Solution Explorer and select Unload project. Right click on the unloaded project and select Edit <SERVICE>.csproj. Once opened, ensure that the following things are changed:

  • Update this entry to use your service name (what you used as <SERVICE> above):
  <PropertyGroup>
    <PsModuleName>Cdn</PsModuleName>
  </PropertyGroup>
  • Remove the entry:
  <PropertyGroup>
    <RootNamespace>$(LegacyAssemblyPrefix)$(PsModuleName)</RootNamespace>
  </PropertyGroup>

Note: This is not needed since this is a new project and does not use legacy namespace conventions.

  • Update this entry to use your SDK:
  <ItemGroup>
    <PackageReference Include="Microsoft.Azure.Management.Cdn" Version="4.0.2-preview" />
  </ItemGroup>

If you have not generated your AutoRest SDK yet, remove this entry for now.

Right click on the project and select Reload project, and then build the solution by either right clicking on the solution and selecting Rebuild Solution or, from the top of Visual Studio, selecting Build > Rebuild Solution. If the build does not succeed, open the .csproj file and ensure there are no errors.

Adding Project References

There are a few existing projects that need to be added before developing any cmdlets. To add a project to the solution, right click on the solution in Solution Explorer and select Add > Existing Project. This allows you to navigate through folders to find the .csproj of the project you want to add. Once a project is added to your solution, you can add it as a reference to the <SERVICE> project by right clicking on <SERVICE> and selecting Add > Reference. This opens the Reference Manager window, and once you have selected the Projects > Solution option on the left side of the window, you are able to select which projects you want to reference in <SERVICE> by checking the box to the left of the name.

Creating Cmdlets

PowerShell Cmdlet Design Guidelines

Please check out the Cmdlet Best Practices document for more information on how to create cmdlets that follow the PowerShell guidelines.

Enable Running PowerShell when Debugging

  • Choose any project and set it as the startup project in Visual Studio
    • Right click on your project in the Solution Explorer and select Set as StartUp project
  • Right-click on the project and select Properties
  • Go to the Debug tab
  • Under Start Action, pick Start external program and type the PowerShell 6.0 directory
    • For example, C:\Program Files\PowerShell\6\pwsh.exe

Importing Modules

To import modules automatically when debug has started, follow the below steps:

  • In the Debug tab mentioned previously, go to Start Options
  • Import the Profile module, along with the module you are testing, by pasting the following in the Command line arguments box (note: you have to update the <PATH_TO_REPO> and values provided below):
    • -NoExit -Command "Import-Module <PATH_TO_REPO>/artifacts/Debug/Az.Accounts/Az.Accounts.psd1;Import-Module <PATH_TO_REPO>/artifacts/Debug/Az.<SERVICE>/Az.<SERVICE>.psd1;$DebugPreference='Continue'"
  • Note: if you do not see all of the changes you made to the cmdlets when importing your module in a PowerShell session (e.g., a cmdlet you added is not recognized as a cmdlet), you may need to delete any existing Azure PowerShell modules that you have on your machine (installed through the PowerShell Gallery) before you import your module.

Adding Help Content

All cmdlets that are created must have accompanying help that is displayed when users execute the command Get-Help <your cmdlet>.

Each cmdlet has a markdown file that contains the help content that is displayed in PowerShell; these markdown files are created (and maintained) using the platyPS module.

For complete documentation, see help-generation.md in the documentation folder.

Adding Tests

Note: As mentioned in the prerequisites section, set the PowerShell execution policy to Unrestricted for the following versions of PowerShell:

  • C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
  • C:\Windows\SysWOW64\WindowsPowerShell\v1.0\powershell.exe
  • C:\Program Files\PowerShell\{{version}}\pwsh.exe

Using Azure TestFramework

Please see our guide on Using Azure TestFramework for information on how to setup the appropriate connection string and record tests.

Scenario Tests

Adding Test Project

  • Create a new folder called ScenarioTests
  • Create a new folder called SessionRecords
  • Inside the ScenarioTests folder, create a new class called <SERVICE>TestRunner
  • In the <SERVICE>TestRunner class, it should have the similar field and constructor like shown below. The parameter values passed in are based on the real situation.
    protected readonly ITestRunner TestRunner;

    protected <SERVICE>TestRunner(ITestOutputHelper output)
    {
        TestRunner = TestManager.CreateInstance(output)
            .WithProjectSubfolderForTests("ScenarioTests")
            .WithNewPsScriptFilename($"{GetType().Name}.ps1")
            .WithCommonPsScripts(new[]
            {
                @"Common.ps1",
                @"../AzureRM.Resources.ps1",
                @"../AzureRM.Storage.ps1"
            })
            .WithNewRmModules(helper => new[]
            {
                helper.RMProfileModule,
                ...
            })
            .WithNewRecordMatcherArguments(
                userAgentsToIgnore: new Dictionary<string, string>
                {
                    ...
                },
                resourceProviders: new Dictionary<string, string>
                {
                    ...
                }
            )
            .Build();
    }

Adding Scenario Tests

  • Create a new class in <SERVICE>.Test
    • The new class must inherit from the <SERVICE>TestRunner class in this project.
    • Add [Fact] as an attribute to every test
    • Add [Trait(Category.AcceptanceType, Category.CheckIn)] as an attribute to any test that should be run during CI in Playback mode.
    • Add [Trait(Category.AcceptanceType, Category.LiveOnly)] as an attribute to any test that cannot be run in Playback mode (for example, if a test depends on a Dataplane SDK).
  • Create a ps1 file in the same folder that contains the actual tests.
  • The name of the ps1 file should exactly match with name of the class. (see sample)
    • Use Assert-AreEqual x y to verify that values are the same
    • Use Assert-AreNotEqual x y to verify that values are not the same
    • Use Assert-Throws scriptblock message to verify an exception is being thrown
    • Use Assert-ThrowsContains scriptblock substring to verify an exception is being thrown and contains a substring
    • Use Assert-Env env[] to verify environment variables
    • Use Assert-True scriptblock to verify that a script block returns true
    • Use Assert-False scriptblock to verify that a script block returns false
    • Use Assert-Null object to verify that an object is null
    • Use Assert-NotNull object to verify that an object is not null
    • Use Assert-Exists path to verify that a file exists
    • Use Assert-AreEqualArray a1 a2 to verify that arrays are the same
    • Use Assert-StartsWith s1 s2 to verify that the string s2 starts with the string s1
    • Use Assert-Match s1 s2 to verify that the string s2 matches the regular expression s1
    • Use Assert-NotMatch s1 s2 to verify that the string s2 does not match the regular expression s1

Use local files in test

CI in DevOps will happens under Debug folder. So you need to make sure that these files will be copied to that folder during the CI runs. One way to do this is config the Module.Test.csproj. There is a example:

  <ItemGroup>
    <None Update="PemFiles\*.*" CopyToOutputDirectory="PreserveNewest" />
  </ItemGroup>

Using Active Directory

  • Use the Set-TestEnvironment cmdlet from Repo-Tasks.psd1 to setup your connection string
  • Alternatively, you can set the following environment variables
    • TEST_CSM_ORGID_AUTHENTICATION is used for Resource Manager testing

Important!

  1. Be sure that you have set the ExecutionPolicy to Unrestricted on both 32-bit and 64-bit PowerShell environments, as mentioned in the prerequisites at the top
  2. When recording tests, if you are using a Prod environment, use ServicePrincipalName (SPN) and ServicePrincipalSecret. For more information on creating an SPN, click here.

AD Scenario Tests

Create this environment variables for the AD scenario tests:

  • AZURE_SERVICE_PRINCIPAL should be a service principal - an application defined in the subscription's tenant - that has management access to the subscription (or at least to a resource group in the tenant)
    • AZURE_SERVICE_PRINCIPAL=UserId=<UserGuid>;Password=<Password>;AADTenant=<TenantGuid>;SubscriptionId=<SubscriptionId>

Recording/Running Tests

  • Set up environment variables using New-TestCredential as described here
  • Run the test in Visual Studio in the Test Explorer window and make sure you got a generated JSON file that matches the test name under the SessionRecords folder

After Development

Once all of your cmdlets have been created and the appropriate tests have been added, you can open a pull request in the Azure PowerShell repository to have your cmdlets added to the next release. Please make sure to read CONTRIBUTING.md for more information on how to open a pull request, clean up commits, make sure appropriate files have been added/changed, and more.

Change Log

Whenever you make updates to a project, please make sure to update the corresponding service's ChangeLog.md file with a snippet of what you changed under the Upcoming Release header. This information is later used for the release notes that goes out with each module the next time they are released, and provides users with more information as to what has changed in the module from the previous release. For more information on updating change logs can be found in CONTRIBUTING.md

Misc

Publish to PowerShell Gallery

To publish your module to the official PowerShell gallery or the test gallery site, contact the Azure PowerShell team

AutoRest PowerShell Generator