Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stack unable to delete ServiceLinkedRoles when upgraded to v1.4.3 #236

Closed
silkyroadsilk opened this issue Aug 15, 2023 · 1 comment
Closed
Labels
bug Something isn't working

Comments

@silkyroadsilk
Copy link

Describe the bug
When updated from version 1.4.1 to 1.4.3 the pipeline errored out in failure to delete existing Service Linked Roles.

2023-08-14 10:38:42.275 | error | toolkit | Stack Deployments Failed: Error: The stack named AWSAccelerator-AccountsStack-123456789-us-east-1 failed to deploy: UPDATE_ROLLBACK_FAILED (The following resource(s) failed to update: [DenyOnSecurityOUsF05B383A, GuardDutyServiceLinkedRoleCreateServiceLinkedRoleResourceD5FE1FBD, DenyOnMigrated7312F37B, SecurityHubServiceLinkedRoleCreateServiceLinkedRoleResource4CC7EFAA, DenyOnProduction26D683DC, DenyOnSandboxD0F93382, DenyOnDevelopmentC81CE8A0]. ): Received response status [FAILED] from custom resource. Message returned: AccessDeniedException: Resource is not in the state functionActive
AWSAccelerator-AccountsStack-1234567891234-us-east-1 |  0/32 | 10:38:23 AM | UPDATE_FAILED        | Custom::CreateServiceLinkedRole | GuardDutyServiceLinkedRole/CreateServiceLinkedRoleResource/Default (GuardDutyServiceLinkedRoleCreateServiceLinkedRoleResourceD5FE1FBD) Received response status [FAILED] from custom resource. Message returned: AccessDeniedException: Resource is not in the state functionActive
    at Object.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/json.js:61:27)
    at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/rest_json.js:61:8)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:686:14)
    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)

    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
        at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
        at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
        at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:688:12) (RequestId: 9ec79f9b-e8d9-49f3-a973-8f6d44b96d2c)
        new CustomResource (/codebuild/output/src2727/src/s3/00/source/node_modules/aws-cdk-lib/core/lib/custom-resource.js:1:823)
        \_ new ServiceLinkedRole (/codebuild/output/src2727/src/s3/00/source/packages/@aws-accelerator/constructs/lib/aws-iam/service-linked-role.ts:87:22)
        \_ AccountsStack.createServiceLinkedRole (/codebuild/output/src2727/src/s3/00/source/packages/@aws-accelerator/accelerator/lib/stacks/accelerator-stack.ts:1210:9)
        \_ AccountsStack.createGuardDutyServiceLinkedRole (/codebuild/output/src2727/src/s3/00/source/packages/@aws-accelerator/accelerator/lib/stacks/accelerator-stack.ts:901:12)
        \_ new AccountsStack (/codebuild/output/src2727/src/s3/00/source/packages/@aws-accelerator/accelerator/lib/stacks/accounts-stack.ts:258:14)
        
    \_ main (/codebuild/output/src2727/src/s3/00/source/packages/@aws-accelerator/accelerator/bin/app.ts:543:29)
        \_ processTicksAndRejections (node:internal/process/task_queues:96:5)
        \_ async /codebuild/output/src2727/src/s3/00/source/packages/@aws-accelerator/accelerator/bin/app.ts:1017:5
    AWSAccelerator-AccountsStack-1234567891234-us-east-1 |  0/32 | 10:38:23 AM | UPDATE_FAILED        | Custom::CreateServiceLinkedRole | SecurityHubServiceLinkedRole/CreateServiceLinkedRoleResource/Default (SecurityHubServiceLinkedRoleCreateServiceLinkedRoleResource4CC7EFAA) Received response status [FAILED] from custom resource. Message returned: AccessDeniedException: Resource is not in the state functionActive
        at Object.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/json.js:61:27)
        at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/rest_json.js:61:8)
        at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
        at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
        at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:686:14)
        at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
        at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
        at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
        at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
        at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:688:12) (RequestId: 4d9cd5cd-895c-433a-8444-823324098955)
        new CustomResource (/codebuild/output/src2727/src/s3/00/source/node_modules/aws-cdk-lib/core/lib/custom-resource.js:1:823)
        \_ new ServiceLinkedRole (/codebuild/output/src2727/src/s3/00/source/packages/@aws-accelerator/constructs/lib/aws-iam/service-linked-role.ts:87:22)
        \_ AccountsStack.createServiceLinkedRole (/codebuild/output/src2727/src/s3/00/source/packages/@aws-accelerator/accelerator/lib/stacks/accelerator-stack.ts:1226:11)
        \_ AccountsStack.createSecurityHubServiceLinkedRole (/codebuild/output/src2727/src/s3/00/source/packages/@aws-accelerator/accelerator/lib/stacks/accelerator-stack.ts:957:12)
        \_ new AccountsStack (/codebuild/output/src2727/src/s3/00/source/packages/@aws-accelerator/accelerator/lib/stacks/accounts-stack.ts:261:14)
        \_ main (/codebuild/output/src2727/src/s3/00/source/packages/@aws-accelerator/accelerator/bin/app.ts:543:29)
        \_ processTicksAndRejections (node:internal/process/task_queues:96:5)
        \_ async /codebuild/output/src2727/src/s3/00/source/packages/@aws-accelerator/accelerator/bin/app.ts:1017:5

To Reproduce
I have tried to re-run the AWSAccelerator-Pipeline after having upgraded landing-zone-accelerator-on-aws to version 1.4.3. In doing so the pipeline was unabled to delete the following roles AWSServiceRoleForSecurityHub', 'AWSServiceRoleForAccessAnalyzer' and 'AWSServiceRoleForAmazonGuardDuty' with the reason AccessDeniedException.

Expected behavior
I expect when the pipeline line is run, that if the roles already exist it will be able to delete the existing and replace with the new.

Additional context
I have also tried to delete a Role by hand in the AWS console and I get the following error:
IAM Access Analyzer is enabled in one or more regions in your AWS organization. Ask your administrator to delete all analyzers in all regions for your organization before attempting to delete this role.
Having seen this message I ensured that no Access Analyzers exist in any region, and tried to delete again after some time. The same error still persists even though there are no access Analyzers.

@silkyroadsilk silkyroadsilk added the bug Something isn't working label Aug 15, 2023
@silkyroadsilk
Copy link
Author

Duplicated in #237 with more context.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant