Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unique id to default base name #1953

Merged
3 commits merged into from
Aug 31, 2021
Merged

Add unique id to default base name #1953

3 commits merged into from
Aug 31, 2021

Conversation

JoshLove-msft
Copy link
Member

@JoshLove-msft JoshLove-msft commented Aug 30, 2021

Now that we are automatically marking new resources for deletion, we should be able to safely ensure that the base name is unique without worrying about creating too many resources.

@JoshLove-msft JoshLove-msft changed the title Add random elements to default base name Add unique id to default base name Aug 30, 2021
@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

Copy link
Member

@heaths heaths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only concern is resource name length (there are some pretty small restrictions for some RPs), but we won't easily know until we try it. Will also vary based on $UserName length anyway, so 🤷‍♂️🚀.

@pakrym
Copy link
Contributor

pakrym commented Aug 30, 2021

I don't like this. We should fix updating RGs process instead. There are resources where there is a count limit per subscription, you would hit it very quickly while trying to debug a test-resources.json.

Doesn't seem there is a scenario where you want to keep abandoning RGs.

@JoshLove-msft
Copy link
Member Author

I don't like this. We should fix updating RGs process instead. There are resources where there is a count limit per subscription, you would hit it very quickly while trying to debug a test-resources.json.

Doesn't seem there is a scenario where you want to keep abandoning RGs.

The idea is to have a good default experience. Since we delete resources by default, we should be able to create them by default as well without having failures.

@pakrym
Copy link
Contributor

pakrym commented Aug 30, 2021

The idea is to have a good default experience. Since we delete resources by default, we should be able to create them by default as well without having failures.

Having a good default experience is something I agree with. But improving how the script behaves for an existing RG would also provide that.

@JoshLove-msft
Copy link
Member Author

The idea is to have a good default experience. Since we delete resources by default, we should be able to create them by default as well without having failures.

Having a good default experience is something I agree with. But improving how the script behaves for an existing RG would also provide that.

Here is the existing error:

New-AzADServicePrincipal: C:\Users\jolov\src\r\Azure\azure-sdk-for-net\eng\common\TestResources\New-TestResources.ps1:292
Line |
292 | … Principal = New-AzADServicePrincipal -Role Owner -Scope "/subscriptio …
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Another object with the same value for property identifierUris already exists.

@pakrym
Copy link
Contributor

pakrym commented Aug 30, 2021

I think it's a recent regression from 04e9623#diff-c87aeec4653a134d00324186ecc94dc9c68f42a760ef2db255487abcee95db89

@JoshLove-msft
Copy link
Member Author

I think it's a recent regression from 04e9623#diff-c87aeec4653a134d00324186ecc94dc9c68f42a760ef2db255487abcee95db89

@scbedd any idea how we can fix this?

@pakrym
Copy link
Contributor

pakrym commented Aug 30, 2021

We can include a random part into the principal name. That's how it worked before. Or delete and re-create a principal.

@heaths
Copy link
Member

heaths commented Aug 30, 2021

Also, it is worth pointing out that, anecdotally, people don't remove their resources by default. This has been pretty apparent from expensive Managed HSMs and their RGs being left around long after tests were re-recorded. So if the idea was a "good default experience now that we delete RGs", I wouldn't call that the norm.

Optimizing for the manual experience is, IMO, our best bet. The CI experience has been pretty stable since almost all data necessary is passed in and not calculated (except for $BaseName, basically).

@JoshLove-msft
Copy link
Member Author

Also, it is worth pointing out that, anecdotally, people don't remove their resources by default. This has been pretty apparent from expensive Managed HSMs and their RGs being left around long after tests were re-recorded. So if the idea was a "good default experience now that we delete RGs", I wouldn't call that the norm.

Optimizing for the manual experience is, IMO, our best bet. The CI experience has been pretty stable since almost all data necessary is passed in and not calculated (except for $BaseName, basically).

I meant that our bots will delete the resources if someone created a resource with default arguments.

@pakrym
Copy link
Contributor

pakrym commented Aug 30, 2021

Not quickly enough if you are making changes to your test-resources.json

@heaths
Copy link
Member

heaths commented Aug 30, 2021

Not quickly enough if you are making changes to your test-resources.json

For that you'd probably want 48hrs instead of 120hrs. 😉

@azure-sdk
Copy link
Collaborator

The following pipelines have been queued for testing:
java - template
java - template - tests
js - template
net - template
net - template - tests
python - template
python - template - tests
You can sign off on the approval gate to test the release stage of each pipeline.
See eng/common workflow

@weshaggard
Copy link
Member

This isn't a new issue but while we have a strategy for cleaning up resources we don't have a good strategy for cleaning up service principals. Creating a new one here all the time seems like it will become an issue in the future. cc @benbp

The change does seem like a regression from 04e9623#diff-c87aeec4653a134d00324186ecc94dc9c68f42a760ef2db255487abcee95db89 and seems like a reasonable fix, definitely like it better then the randomized basename, but I'd like @scbedd to take a look as well given he has some of the context for his change.

@ghost
Copy link

ghost commented Aug 31, 2021

Hello @azure-sdk!

Because this pull request has the auto-merge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

@JoshLove-msft
Copy link
Member Author

This isn't a new issue but while we have a strategy for cleaning up resources we don't have a good strategy for cleaning up service principals. Creating a new one here all the time seems like it will become an issue in the future. cc @benbp

The change does seem like a regression from 04e9623#diff-c87aeec4653a134d00324186ecc94dc9c68f42a760ef2db255487abcee95db89 and seems like a reasonable fix, definitely like it better then the randomized basename, but I'd like @scbedd to take a look as well given he has some of the context for his change.

Alright, it was already merged in the downstream repos, but I can create a new PR for any feedback you guys have.

@ghost ghost merged commit 2828d14 into Azure:main Aug 31, 2021
@benbp
Copy link
Member

benbp commented Sep 1, 2021

@weshaggard I think we can add a DeleteAfter tag to our service principals and include SP cleanup in our pipeline. SPs do support tags. We could also consider adding a reference to the oid in a tag on the resource group, but I think the cleanup logic would be more complex that way.

#1966

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants