Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify New-TestResources usage #1077

Merged
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 52 additions & 7 deletions eng/common/TestResources/New-TestResources.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,17 @@
[CmdletBinding(DefaultParameterSetName = 'Default', SupportsShouldProcess = $true, ConfirmImpact = 'Medium')]
param (
# Limit $BaseName to enough characters to be under limit plus prefixes, and https://docs.microsoft.com/azure/architecture/best-practices/resource-naming.
[Parameter(Mandatory = $true, Position = 0)]
[Parameter()]
[ValidatePattern('^[-a-zA-Z0-9\.\(\)_]{0,80}(?<=[a-zA-Z0-9\(\)])$')]
[string] $BaseName,

[ValidatePattern('^[-\w\._\(\)]+$')]
[string] $ResourceGroupName,

[Parameter(Mandatory = $true)]
[Parameter(Mandatory = $true, Position = 0)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a breaking change. You moved the first unnamed parameter position. Leave them both named.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it would actually break anyone?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think simplifying the script is worth it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't say. Do you think specifying -ServiceDirectory is too much to type especially compared to everything they had to type before, whether using the script or az CLI directly?

I suppose, at the very least, both were mandatory before so if they did expect this to be $BaseName, they'd likely get an error that "sdk/Whatever" doesn't exist.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't say. Do you think specifying -ServiceDirectory is too much to type especially compared to everything they had to type before, whether using the script or az CLI directly?

Not too much comparatively, but if we can do even better and it's pretty safe I would like to try.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you know you are introducing a breaking change, make sure you go to the sync PRs that get created on changes in this directory and trigger some of the live test pipelines against them just to make sure that they continue to function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great point @mitchdenny, I'll do that.

Copy link
Member

@weshaggard weshaggard Oct 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where are you defaulting the ServiceDirectory? Isn't it needed in order to know which arm template to deploy?

edit: Sorry I miss-read the comments it is still mandatory here so you can ignore my comment about it being required.

However we could consider figuring out a way of setting this based on where the script is executed, but that might be a little magical.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't default the ServiceDirectory only the base name. ServiceDirectory is not the only required parameter and I made it positional as well.

[string] $ServiceDirectory,

[Parameter(Mandatory = $true)]
[Parameter()]
[ValidatePattern('^[0-9a-f]{8}(-[0-9a-f]{4}){3}-[0-9a-f]{12}$')]
[string] $TestApplicationId,

Expand Down Expand Up @@ -119,6 +119,8 @@ $root = [System.IO.Path]::Combine($repositoryRoot, "sdk", $ServiceDirectory) | R
$templateFileName = 'test-resources.json'
$templateFiles = @()
$environmentVariables = @{}
# Azure SDK Developer Playground
$defaultSubscription = "faa080af-c1d8-40ad-9cce-e1a450ca5b57"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this work for partner teams?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we were giving them access. If not they can log in using Connect-AzAccount on their own.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mitchdenny @weshaggard do you happen to know if partner teams will have access to Azure SDK Dev Playground subscription?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the current permissions set I believe that its just ADP. Generally folks have contributor access for our org and then a few folks have ownership.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with defaulting to this subscription but can we please test the failure case for someone that doesn't have access to this default sub (or another test sub)? I want to make sure it is pretty clear what is going wrong and not failing in some very hard to diagnose way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it fails with pretty clear access denied message.


Write-Verbose "Checking for '$templateFileName' files under '$root'"
Get-ChildItem -Path $root -Filter $templateFileName -Recurse | ForEach-Object {
Expand All @@ -133,6 +135,26 @@ if (!$templateFiles) {
exit
}

$UserName = if ($env:USER) { $env:USER } else { "${env:USERNAME}" }

# If no base name is specified use current user name
if (!$BaseName) {
$BaseName = "$UserName$ServiceDirectory"

Log "BaseName was not set. Using default base name: '$BaseName'"
}

# Try detecting repos that support OutFile and defaulting to it
if (!$CI -and !$PSBoundParameters.ContainsKey('OutFile') -and $IsWindows)
{
# TODO: find a better way to detect the language
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chidozieononiwu is working on some common scripts one of which would give you the language. If you import common.ps1 it will import the language-settings.ps1 which will have $language defined.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mind if I file this as a followup?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that really the heuristic we want anyway? Ideally, eventually every language in any platform can do this. It's just a matter of how. When we three talked about this months ago, one idea was even just to chmod the file, which is all the CLI does on linux anyway. I'd rather we base this on some marker - perhaps a file under /eng, or something more general purpose we can read and getting repo configs, like a .json file that PS can read as an object. Then we don't have to keep updating this script and going through all the process around pushing it down.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (Test-Path "$repositoryRoot/eng/service.proj")
{
$OutFile = $true
Log "Detected .NET repository. Defaulting OutFile to true. Test environment settings would be stored into the file so you don't need to set environment variables manually."
}
}

# If no location is specified use safe default locations for the given
# environment. If no matching environment is found $Location remains an empty
# string.
Expand All @@ -147,7 +169,7 @@ if (!$Location) {
Write-Verbose "Location was not set. Using default location for environment: '$Location'"
}

# Log in if requested; otherwise, the user is expected to already be authenticated via Connect-AzAccount.
# Log in if requested; otherwise, try to login into playground subscription.
if ($ProvisionerApplicationId) {
$null = Disable-AzContextAutosave -Scope Process

Expand Down Expand Up @@ -176,6 +198,25 @@ if ($ProvisionerApplicationId) {
}
}
}
elseif (!$CI)
{
# check if user is logged in and login into
$context = Get-AzContext;
if (!$context)
{
Log "You are not logged in, connecting to 'Azure SDK Developer Playground'"
Connect-AzAccount -Subscription $defaultSubscription
}

# If no test application id is specified create a new service principal
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oooohhhh.... nice!

if (!$TestApplicationId) {
Log "TestApplicationId was not specified, creating a new service principal."
$servicePrincipal = New-AzADServicePrincipal -Role Owner
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that need to be removed at the end of the run, if it was dynamic? (or is it already and I'm just overlooking?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, in theory, you are right. But we never told anyone to remove their service principals after manual creation so I decided to postpone fixing the removing part.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any downside to removing it at the end of the script?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, tests won't run because they won't be able to sign in :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could only remove it in the Remove-TestResources script, assuming the same principal is used to run the tests (by default they are). You could save that to an environment variable or some other means to know which to remove.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed #1078


$TestApplicationId = $servicePrincipal.ApplicationId
$TestApplicationSecret = (ConvertFrom-SecureString $servicePrincipal.Secret -AsPlainText);
}
}

# Get test application OID from ID if not already provided.
if ($TestApplicationId -and !$TestApplicationOid) {
Expand Down Expand Up @@ -215,7 +256,7 @@ $ResourceGroupName = if ($ResourceGroupName) {

# Tag the resource group to be deleted after a certain number of hours if specified.
$tags = @{
Creator = if ($env:USER) { $env:USER } else { "${env:USERNAME}" }
Creator = $UserName
ServiceDirectory = $ServiceDirectory
}

Expand Down Expand Up @@ -403,7 +444,11 @@ foreach ($templateFile in $templateFiles) {

$exitActions.Invoke()

return $environmentVariables
# Suppress output locally
if ($CI)
{
return $environmentVariables
}

<#
.SYNOPSIS
Expand Down Expand Up @@ -568,4 +613,4 @@ log redaction).

.LINK
Remove-TestResources.ps1
#>
#>
21 changes: 12 additions & 9 deletions eng/common/TestResources/New-TestResources.ps1.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,16 @@ Deploys live test resources defined for a service directory to Azure.

### Default (Default)
```
New-TestResources.ps1 [-BaseName] <String> [-ResourceGroupName <String>] -ServiceDirectory <String>
-TestApplicationId <String> [-TestApplicationSecret <String>] [-TestApplicationOid <String>]
New-TestResources.ps1 [-BaseName <String>] [-ResourceGroupName <String>] [-ServiceDirectory] <String>
[-TestApplicationId <String>] [-TestApplicationSecret <String>] [-TestApplicationOid <String>]
[-DeleteAfterHours <Int32>] [-Location <String>] [-Environment <String>] [-AdditionalParameters <Hashtable>]
[-CI] [-Force] [-OutFile] [-WhatIf] [-Confirm] [<CommonParameters>]
```

### Provisioner
```
New-TestResources.ps1 [-BaseName] <String> [-ResourceGroupName <String>] -ServiceDirectory <String>
-TestApplicationId <String> [-TestApplicationSecret <String>] [-TestApplicationOid <String>]
New-TestResources.ps1 [-BaseName <String>] [-ResourceGroupName <String>] [-ServiceDirectory] <String>
[-TestApplicationId <String>] [-TestApplicationSecret <String>] [-TestApplicationOid <String>]
-TenantId <String> [-SubscriptionId <String>] -ProvisionerApplicationId <String>
-ProvisionerApplicationSecret <String> [-DeleteAfterHours <Int32>] [-Location <String>]
[-Environment <String>] [-AdditionalParameters <Hashtable>] [-CI] [-Force] [-OutFile] [-WhatIf] [-Confirm]
Expand Down Expand Up @@ -106,8 +106,8 @@ Type: String
Parameter Sets: (All)
Aliases:

Required: True
Position: 1
Required: False
Position: Named
Default value: None
Accept pipeline input: False
Accept wildcard characters: False
Expand Down Expand Up @@ -140,7 +140,7 @@ Parameter Sets: (All)
Aliases:

Required: True
Position: Named
Position: 1
Default value: None
Accept pipeline input: False
Accept wildcard characters: False
Expand All @@ -159,7 +159,7 @@ Type: String
Parameter Sets: (All)
Aliases:

Required: True
Required: False
Position: Named
Default value: None
Accept pipeline input: False
Expand Down Expand Up @@ -197,7 +197,7 @@ It is passed as to the ARM
template as 'testApplicationOid'

For more information on the relationship between AAD Applications and Service
Principals see: https://docs.microsoft.com/azure/active-directory/develop/app-objects-and-service-principals
Principals see: https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals

```yaml
Type: String
Expand Down Expand Up @@ -458,3 +458,6 @@ This cmdlet supports the common parameters: -Debug, -ErrorAction, -ErrorVariable
## NOTES

## RELATED LINKS

[Remove-TestResources.ps1]()

11 changes: 2 additions & 9 deletions eng/common/TestResources/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,17 +26,10 @@ Note that `-Subscription` is an optional parameter but recommended if your accou
is a member of multiple subscriptions.

```powershell
Connect-AzAccount -Subscription 'YOUR SUBSCRIPTION ID'
$sp = New-AzADServicePrincipal -Role Owner
eng\common\TestResources\New-TestResources.ps1 `
-BaseName 'myusername' `
-ServiceDirectory 'search' `
-TestApplicationId $sp.ApplicationId `
-TestApplicationSecret (ConvertFrom-SecureString $sp.Secret -AsPlainText)
eng\common\TestResources\New-TestResources.ps1 -ServiceDirectory 'search'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since partner teams will not have access to Playground subscription, should we leave the first line with Connect-AzAccount?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or should we prompt them to enter a subscription after the fact?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think is more user friendly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'll add the first line back

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there was a way that we can try to login and only prompt if it failed, I would vote for that.

```

If you are running this for a .NET project on Windows, the recommended method is to
add the `-OutFile` switch to the above command. This will save test environment settings
The `OutFile` switch would be set if you are running this for a .NET project on Windows. This will save test environment settings
into a test-resources.json.env file next to test-resources.json. The file is protected via DPAPI.
The environment file would be scoped to the current repository directory and avoids the need to
set environment variables or restart your IDE to recognize them.
Expand Down