PIM Issue and Workaround: Why arent new envts showing up in my inventory???

[Jenefer-Monroe]

The issue

In the past, as the identity running the inventory flows had the PPAdmin Role, it would get added as a Sys Admin to any new environments as they came in. And if you removed someone with the PPAdmin Role from the Sys Admin to any envt they would just get re-added.
This was just the way the product treated its privileged roles.

This behavior is changing and now these privileged role users will not get added to these SRs by default.
Instead privileged role users will be able to see all information from within product UX (ex all environments in the Admin UX or returned by the connector) but will not get the Sys Admin SR.

While this is good behavior it does conflict with the design of the starter kit which reaches in to all envts in the tenant in order to gather information.

As a result you will need to escalate the identity in all environments in order to continue using the kit.

The feature

Here is information about the feature: Manage admin roles with Microsoft Entra Privileged Identity Management

And about this this limitation for this feature:

Feature is now live!!!

The workaround until we ship with the kit

The action required is not yet available in all regions.
When it is available in your region you can use the temp solution below to fix up the permissions.

1. Create connections to both Power Platform for Admins and Power Platform for Admins V2 connectors
2. Install this solution: PIMWorkaroundSolution_1_0_0_3_managed.zip
3. Turn on the flow: PIM Workaround Flow

Role requirements

Do to complexities around Security Group protections on Environments. The following are the requirements for the Admin Role Used

1. Global Admin or Power Platform Admin roles are required. The D365 role does not work for the escalation in many cases.
2. The escalation requires these Entra roles to be assigned directly to users; association via SG is not supported.

Required Action

Grants requesting tenant admin user the system admin role within Dataverse in the environment

The workaround until new action available in your region

If you see this when you try to turn on the flow, then the action is not yet available in your region:

There are other existing steps to elevate the user and hence be added, but they are not in a form that the kit can consume yet as the connector is not yet available in all regions.. The product team is actively working to unblock us but we are not yet able to do this elevation for you.

As a result, here are your options:

• implement this logic yourself to add the identity running the kit to new environments
• not have the new environments be in the inventory until the kit can be fixed up
• manually add the identity running the kit as System Admin in the new envts as they arrive.

Once available everywhere

Once this new action is available in all regions, including the sovereign clouds, we will ship with the CoE Starter Kit directly and you will no longer need this temp solution.

[natjjensen]

Hi Jenefer and PowerCat.
It is not possible to use the workaround for envs that have a security group attached.
It fails for Microsoft Entra ID security groups and for on-prem AD security group. This is the error when an env has a security group:
"User [GUID] is not part of security group [GUID]"

[Jenefer-Monroe]

For security groups, it does work from our testing, with two known limitations

1. The escalation requires the Entra roles to be assigned directly to users, not if assigned via a group.
   
2. This SG exemption stuff only works for Global Admins and PP Admins, it does not work for D365 Admins

Can you please confirm these? If you have these and it still does not work then Im afraid you'll need to contact product support. I can help you get a repro outside the kit so that working with them is faster.

Repro for Help

If the above doesn't fix it, please let me know.
Then import this test flow, select your target environment that is failing, and run the flow. Bring this test flow to support.

testflowforPIM_20240614140015.zip

[kashanico]

@Jenefer-Monroe Can you confirm that both Global Admin and Power Platform Admin roles are required for our CoE service account for this workaround? We cannot give an account permanently active Global Admin role, this would be significant security issue for us. Global Admin roles are tightly controlled.

Our current service account has Power Platform Admin directly assigned, wondering if that will suffice? If not, I would advise against a solution that would require us to open up a large attack vector on our entire Azure tenant.

Thank you

[Jenefer-Monroe]

My bad, no they are not both required. So sorry for that confusion, I've fixed inline

[kashanico]

ah great to hear, thanks.

[orbd12]

Hey @Jenefer-Monroe, I have a permanent Power Platform Admin role activated on my account, but am running into the below error when trying to use the grant sys admin action:

  "body": {
    "code": "Forbidden",
    "message": "The caller is not authorized to perform the request.",
    "innererror": {
      "code": "InsufficientPrincipalPermissions",
      "message": "Authorization denied: Principal missing required permissions: [UserManagement.Users.Apply, All.All.ReadWrite]"
    }
  }

Apologies, for some reason I'm unable to upload screenshots.

[Jenefer-Monroe]

Is the role directly assigned? It cannot be assigned via a group.

[orbd12]

It is, sorry for not specifying.

[Jenefer-Monroe]

no problem at all.
ok unfortunately then you'll have to reach out to product support. Its a new action so they may just have a bug. Please use the test flow below for the repro you bring them.

Repro for Help

If the above doesn't fix it, please let me know.
Then import this test flow, select your target environment that is failing, and run the flow. Bring this test flow to support.

testflowforPIM_20240614140015.zip

[orbd12]

Thanks for your help @Jenefer-Monroe , got it working.

Some feedback, the action itself asks for Environment Id, which is a possible output from the list environments as admin action. However, what the action wants is the name field, as shown in the demo flow.

What was causing the issue I pasted above was that I was providing it the Environment Id output from the list environments action, not the name, as I was testing from a flow I created on my own. Maybe this could be marked more clearly in the future?

Thanks again for your help!

[Jenefer-Monroe]

sorry are you saying you got the test flow to work? Or that the action to escalate privilege's is working for you now?

[bilodeauju]

I tested the flow "PIM Workaround Flow" and it appears to be failing on all of our Teams environments with error "Action 'Grants_requesting_tenant_admin_user_the_system_admin_role' failed." Is this a known issue, or should it work for Teams environments? I have confirmed my ID has the role "Power Platform Admin" assigned directly.

[Jenefer-Monroe]

Teams environments do seem to work for me in my testing.

Can you please resubmit and see if it still fails? If so get me the exact error message and I will reach out internally to see if they have debugging ideas.

And then you should also contact product support with the repro if you repro again.

[Jenefer-Monroe]

Looks like D4T environments are failing for me now too. Posted this bug to track here in GitHub: #8580
And then also posted an ICM for the product team.

[bilodeauju]

Today the flow is working for D4T environments for me, however it is now failing on another environment that appears to be already deleted.

Here is the outputs detail for the action "List Environments as Admin" for that environment that is failing:

It lists the environment with "provisioningState": "Deleted". I verified cannot find this environment in PPAC.

The flow fails for that environment at "Grants requesting tenant admin user the system admin role" and the output shows "statusCode": 400

Here is the full output:

{
"statusCode": 400,
"headers": {
"Cache-Control": "no-store, no-cache",
"Strict-Transport-Security": "max-age=31536000; includeSubDomains",
"x-ms-islandgateway": "GA000005S,GA00000C8",
"x-ms-ppapigateway": "GA000005S,GA00000C7",
"x-ms-gateway-clusters": "prdcm001cca,prdcm001eus",
"mise-correlation-id": "247415b0-73a4-45c7-8300-e0e9c692bddd,5212ed45-87a7-421f-9f87-0313142e3100",
"api-supported-versions": "1",
"x-ms-client-request-id": "66746b45-34ce-41e0-a085-99ff1670c222",
"x-ms-service-request-id": "24eccdaf-7582-463a-9096-1548c123c72d",
"x-ms-correlation-id": "66746b45-34ce-41e0-a085-99ff1670c222",
"x-ms-activity-vector": "02.08.IN.2X.IN.13.IN.16.00",
"x-servicefabric": "NoRetry",
"Server-Timing": "x-ms-igw-upstream-headers;dur=147.4,x-ms-igw-req-overhead;dur=1.3",
"X-Content-Type-Options": "nosniff",
"Timing-Allow-Origin": "*",
"x-ms-apihub-cached-response": "true",
"x-ms-apihub-obo": "false",
"Date": "Thu, 04 Jul 2024 18:35:47 GMT",
"Content-Length": "1284",
"Content-Type": "application/json"
},
"body": {
"errors": [
{
"Subject": "Result",
"Description": "Requested instance d98f21ae-b7d2-44d1-bd28-3332966f3efc not found",
"Code": "instanceNotFound"
},
{
"Subject": "InnerException",
"Description": null,
"Code": null
},
{
"Subject": "AdditionalData",
"Description": null,
"Code": null
}
],
"information": [
{
"Subject": "Result",
"Description": "["SyncMode: Default","Requested instance d98f21ae-b7d2-44d1-bd28-3332966f3efc not found"]",
"Code": "instanceNotFound"
},
{
"Subject": "AdditionalResultDetails",
"Description": "",
"Code": null
},
{
"Subject": "RequestId",
"Description": "76469f5a-7ca6-4068-942b-83eafc205142",
"Code": null
},
{
"Subject": "CorrelationId",
"Description": "66746b45-34ce-41e0-a085-99ff1670c222",
"Code": null
},
{
"Subject": "SystemUserId",
"Description": null,
"Code": null
},
{
"Subject": "SecurityGroupId",
"Description": null,
"Code": null
},
{
"Subject": "Timestamp",
"Description": "7/4/2024 6:35:47 PM",
"Code": null
}
]
}
}

[Jenefer-Monroe]

Yes unfortunately it appears that there are some oddities in the way that the action works.
In the driver flow when we ship this natively we will catch these instances and warn. Cases like this will be intermittent (I presume) and so you will be able to ignore.

[Jenefer-Monroe]

fixed in August release https://github.com/microsoft/coe-starter-kit/releases/tag/CoEStarterKit-August2024

[patchivs]

the alternate you gave here with the flow is not working:

I am getting below error:
{
"code": "Forbidden",
"message": "The caller is not authorized to perform the request.",
"innererror": {
"code": "InsufficientPrincipalPermissions",
"message": "Authorization denied: Principal missing required permissions: [UserManagement.Users.Apply, All.All.ReadWrite]"
}
}