Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

r/aws_lambda_function: re-implement replace_security_groups_on_destroy #37624

Merged
merged 1 commit into from
May 23, 2024

Conversation

jar-b
Copy link
Member

@jar-b jar-b commented May 21, 2024

Description

This change implements an alternative approach for improving deletion times of security groups associated with deleted Lambda functions. Previously, the replace_security_groups_on_destroy and replacement_security_group_ids were deprecated as AWS began disallowing modification of the security groups assigned to orphaned ENI's left behind after destruction of a Lambda function.

The deprecations on these arguments will be removed with this re-implementation. Instead of attempting to modify orphaned ENI's after deletion of the function, this approach replaces the security groups assigned in the functions VPC configuration, waits for the configuration update to complete, and then deletes the function. This re-ordering allows for faster security group deletion as the lambda function ENI's left behind after deletion do not contain the configured security groups. This approach also does not violate AWS rules around mutability of Lambda function ENI's, as Lambda is handling replacement of the security groups associated with the ENI's internally. While improvements are not as significant as the previous approach, this option still offers a significant reduction in destroy time when compared to a configurations without this option enabled.

Relations

Closes #31911
Relates #31904
Relates #31520

References

Output from Acceptance Testing

% make testacc PKG=lambda TESTS="TestAccLambdaFunction_VPC_replaceSGWith"
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.22.2 test ./internal/service/lambda/... -v -count 1 -parallel 20 -run='TestAccLambdaFunction_VPC_replaceSGWith'  -timeout 360m

--- PASS: TestAccLambdaFunction_VPC_replaceSGWithDefault (426.21s)
--- PASS: TestAccLambdaFunction_VPC_replaceSGWithCustom (648.47s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/lambda     653.553s
% make testacc PKG=lambda TESTS="TestAccLambdaFunction_"
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.22.2 test ./internal/service/lambda/... -v -count 1 -parallel 20 -run='TestAccLambdaFunction_'  -timeout 360m

--- PASS: TestAccLambdaFunction_expectFilenameAndS3Attributes (7.82s)
=== CONT  TestAccLambdaFunction_loggingConfig
--- PASS: TestAccLambdaFunction_nameValidation (8.23s)
=== CONT  TestAccLambdaFunction_ephemeralStorage
--- PASS: TestAccLambdaFunction_Zip_validation (13.86s)
=== CONT  TestAccLambdaFunction_architecturesWithLayer
--- PASS: TestAccLambdaFunction_versioned (49.69s)
=== CONT  TestAccLambdaFunction_architecturesUpdate
--- PASS: TestAccLambdaFunction_S3Update_unversioned (59.60s)
=== CONT  TestAccLambdaFunction_architectures
--- PASS: TestAccLambdaFunction_skipDestroyInconsistentPlan (60.70s)
=== CONT  TestAccLambdaFunction_EnvironmentVariables_noValue
--- PASS: TestAccLambdaFunction_S3Update_basic (62.64s)
=== CONT  TestAccLambdaFunction_tags
--- PASS: TestAccLambdaFunction_nilDeadLetter (68.23s)
=== CONT  TestAccLambdaFunction_unpublishedCodeUpdate
--- PASS: TestAccLambdaFunction_basic (71.08s)
=== CONT  TestAccLambdaFunction_envVariables
--- PASS: TestAccLambdaFunction_tracing (74.88s)
=== CONT  TestAccLambdaFunction_disappears
--- PASS: TestAccLambdaFunction_EnvironmentVariables_emptyUpgrade (76.27s)
=== CONT  TestAccLambdaFunction_disablePublish
--- PASS: TestAccLambdaFunction_skipDestroy (80.78s)
=== CONT  TestAccLambdaFunction_deadLetterUpdated
--- PASS: TestAccLambdaFunction_snapStart (81.34s)
=== CONT  TestAccLambdaFunction_deadLetter
--- PASS: TestAccLambdaFunction_concurrencyCycle (98.23s)
=== CONT  TestAccLambdaFunction_VPCPublishNo_changes
--- PASS: TestAccLambdaFunction_codeSigning (105.37s)
=== CONT  TestAccLambdaFunction_s3
--- PASS: TestAccLambdaFunction_ephemeralStorage (114.15s)
=== CONT  TestAccLambdaFunction_emptyVPC
--- PASS: TestAccLambdaFunction_s3 (26.56s)
=== CONT  TestAccLambdaFunction_VPC_replaceSGWithCustom
--- PASS: TestAccLambdaFunction_loggingConfig (126.46s)
=== CONT  TestAccLambdaFunction_VPC_replaceSGWithDefault
--- PASS: TestAccLambdaFunction_LocalUpdate_nameOnly (304.98s)
=== CONT  TestAccLambdaFunction_VPC_properIAMDependencies
--- PASS: TestAccLambdaFunction_localUpdate (310.81s)
=== CONT  TestAccLambdaFunction_VPCPublishHas_changes
--- PASS: TestAccLambdaFunction_EnvironmentVariables_noValue (273.32s)
=== CONT  TestAccLambdaFunction_enablePublish
--- PASS: TestAccLambdaFunction_architectures (276.70s)
=== CONT  TestAccLambdaFunction_versionedUpdate
--- PASS: TestAccLambdaFunction_disappears (281.36s)
=== CONT  TestAccLambdaFunction_encryptedEnvVariables
--- PASS: TestAccLambdaFunction_tags (297.75s)
=== CONT  TestAccLambdaFunction_vpc
--- PASS: TestAccLambdaFunction_disablePublish (297.83s)
=== CONT  TestAccLambdaFunction_VPC_withInvocation
--- PASS: TestAccLambdaFunction_deadLetterUpdated (309.54s)
=== CONT  TestAccLambdaFunction_vpcUpdate
--- PASS: TestAccLambdaFunction_deadLetter (310.61s)
=== CONT  TestAccLambdaFunction_vpcRemoval
--- PASS: TestAccLambdaFunction_envVariables (331.82s)
=== CONT  TestAccLambdaFunction_layers
--- PASS: TestAccLambdaFunction_runtimes (438.40s)
=== CONT  TestAccLambdaFunction_layersUpdate
--- PASS: TestAccLambdaFunction_architecturesWithLayer (562.33s)
=== CONT  TestAccLambdaFunction_concurrency
--- PASS: TestAccLambdaFunction_ipv6AllowedForDualStack (701.53s)
=== CONT  TestAccLambdaFunction_KMSKeyARN_noEnvironmentVariables
--- PASS: TestAccLambdaFunction_emptyVPC (643.83s)
--- PASS: TestAccLambdaFunction_fileSystem (907.72s)
--- PASS: TestAccLambdaFunction_VPCPublishNo_changes (938.12s)
--- PASS: TestAccLambdaFunction_VPC_replaceSGWithCustom (1056.70s)
--- PASS: TestAccLambdaFunction_architecturesUpdate (1295.23s)
--- PASS: TestAccLambdaFunction_VPC_properIAMDependencies (1056.01s)
--- PASS: TestAccLambdaFunction_VPC_replaceSGWithDefault (1226.73s)
--- PASS: TestAccLambdaFunction_enablePublish (1038.48s)
--- PASS: TestAccLambdaFunction_unpublishedCodeUpdate (1479.15s)
--- PASS: TestAccLambdaFunction_encryptedEnvVariables (1228.87s)
--- PASS: TestAccLambdaFunction_VPCPublishHas_changes (1424.59s)
--- PASS: TestAccLambdaFunction_vpc (1569.97s)
--- PASS: TestAccLambdaFunction_layers (1546.31s)
--- PASS: TestAccLambdaFunction_VPC_withInvocation (1588.27s)
--- PASS: TestAccLambdaFunction_KMSKeyARN_noEnvironmentVariables (1654.83s)
--- PASS: TestAccLambdaFunction_concurrency (1782.71s)
--- PASS: TestAccLambdaFunction_layersUpdate (1924.70s)
--- PASS: TestAccLambdaFunction_versionedUpdate (2040.51s)
--- PASS: TestAccLambdaFunction_vpcUpdate (2161.88s)
--- PASS: TestAccLambdaFunction_vpcRemoval (2161.44s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/lambda     2558.481s

Copy link

Community Note

Voting for Prioritization

  • Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

  • Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
  • For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
  • Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

@github-actions github-actions bot added size/M Managed by automation to categorize the size of a PR. documentation Introduces or discusses updates to documentation. service/lambda Issues and PRs that pertain to the lambda service. labels May 21, 2024
@terraform-aws-provider terraform-aws-provider bot added the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label May 21, 2024
@jar-b jar-b force-pushed the f-lambda_replace_sg_on_destroy branch 2 times, most recently from 7e52d3d to 2299b1a Compare May 21, 2024 17:31
This change implements an alternative approach for improving deletion times of security groups associated with deleted Lambda functions. Previously, the `replace_security_groups_on_destroy` and `replacement_security_group_ids` were deprecated as AWS began disallowing modification of the security groups assigned to orphaned ENI's left behind after destruction of a Lambda function.

The deprecations on these arguments will be removed with this re-implementation. Instead of attempting to modify orphaned ENI's after deletion of the function, this approach replaces the security groups assigned in the functions VPC configuration, waits for the configuration update to complete, and then deletes the function. This re-ordering allows for faster security group deletion as the lambda function ENI's left behind after deletion do not contain the configured security groups. This approach also does not violate AWS rules around mutability of Lambda function ENI's, as Lambda is handling replacement of the security groups associated with the ENI's internally. While improvements are not as significant as the previous approach, this option still offers a significant reduction in destroy time when compared to a configurations without this option enabled.

```console
% make testacc PKG=lambda TESTS="TestAccLambdaFunction_VPC_replaceSGWith"
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.22.2 test ./internal/service/lambda/... -v -count 1 -parallel 20 -run='TestAccLambdaFunction_VPC_replaceSGWith'  -timeout 360m

--- PASS: TestAccLambdaFunction_VPC_replaceSGWithDefault (426.21s)
--- PASS: TestAccLambdaFunction_VPC_replaceSGWithCustom (648.47s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/lambda     653.553s
```

```console
% make testacc PKG=lambda TESTS="TestAccLambdaFunction_"
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.22.2 test ./internal/service/lambda/... -v -count 1 -parallel 20 -run='TestAccLambdaFunction_'  -timeout 360m

--- PASS: TestAccLambdaFunction_expectFilenameAndS3Attributes (7.82s)
=== CONT  TestAccLambdaFunction_loggingConfig
--- PASS: TestAccLambdaFunction_nameValidation (8.23s)
=== CONT  TestAccLambdaFunction_ephemeralStorage
--- PASS: TestAccLambdaFunction_Zip_validation (13.86s)
=== CONT  TestAccLambdaFunction_architecturesWithLayer
--- PASS: TestAccLambdaFunction_versioned (49.69s)
=== CONT  TestAccLambdaFunction_architecturesUpdate
--- PASS: TestAccLambdaFunction_S3Update_unversioned (59.60s)
=== CONT  TestAccLambdaFunction_architectures
--- PASS: TestAccLambdaFunction_skipDestroyInconsistentPlan (60.70s)
=== CONT  TestAccLambdaFunction_EnvironmentVariables_noValue
--- PASS: TestAccLambdaFunction_S3Update_basic (62.64s)
=== CONT  TestAccLambdaFunction_tags
--- PASS: TestAccLambdaFunction_nilDeadLetter (68.23s)
=== CONT  TestAccLambdaFunction_unpublishedCodeUpdate
--- PASS: TestAccLambdaFunction_basic (71.08s)
=== CONT  TestAccLambdaFunction_envVariables
--- PASS: TestAccLambdaFunction_tracing (74.88s)
=== CONT  TestAccLambdaFunction_disappears
--- PASS: TestAccLambdaFunction_EnvironmentVariables_emptyUpgrade (76.27s)
=== CONT  TestAccLambdaFunction_disablePublish
--- PASS: TestAccLambdaFunction_skipDestroy (80.78s)
=== CONT  TestAccLambdaFunction_deadLetterUpdated
--- PASS: TestAccLambdaFunction_snapStart (81.34s)
=== CONT  TestAccLambdaFunction_deadLetter
--- PASS: TestAccLambdaFunction_concurrencyCycle (98.23s)
=== CONT  TestAccLambdaFunction_VPCPublishNo_changes
--- PASS: TestAccLambdaFunction_codeSigning (105.37s)
=== CONT  TestAccLambdaFunction_s3
--- PASS: TestAccLambdaFunction_ephemeralStorage (114.15s)
=== CONT  TestAccLambdaFunction_emptyVPC
--- PASS: TestAccLambdaFunction_s3 (26.56s)
=== CONT  TestAccLambdaFunction_VPC_replaceSGWithCustom
--- PASS: TestAccLambdaFunction_loggingConfig (126.46s)
=== CONT  TestAccLambdaFunction_VPC_replaceSGWithDefault
--- PASS: TestAccLambdaFunction_LocalUpdate_nameOnly (304.98s)
=== CONT  TestAccLambdaFunction_VPC_properIAMDependencies
--- PASS: TestAccLambdaFunction_localUpdate (310.81s)
=== CONT  TestAccLambdaFunction_VPCPublishHas_changes
--- PASS: TestAccLambdaFunction_EnvironmentVariables_noValue (273.32s)
=== CONT  TestAccLambdaFunction_enablePublish
--- PASS: TestAccLambdaFunction_architectures (276.70s)
=== CONT  TestAccLambdaFunction_versionedUpdate
--- PASS: TestAccLambdaFunction_disappears (281.36s)
=== CONT  TestAccLambdaFunction_encryptedEnvVariables
--- PASS: TestAccLambdaFunction_tags (297.75s)
=== CONT  TestAccLambdaFunction_vpc
--- PASS: TestAccLambdaFunction_disablePublish (297.83s)
=== CONT  TestAccLambdaFunction_VPC_withInvocation
--- PASS: TestAccLambdaFunction_deadLetterUpdated (309.54s)
=== CONT  TestAccLambdaFunction_vpcUpdate
--- PASS: TestAccLambdaFunction_deadLetter (310.61s)
=== CONT  TestAccLambdaFunction_vpcRemoval
--- PASS: TestAccLambdaFunction_envVariables (331.82s)
=== CONT  TestAccLambdaFunction_layers
--- PASS: TestAccLambdaFunction_runtimes (438.40s)
=== CONT  TestAccLambdaFunction_layersUpdate
--- PASS: TestAccLambdaFunction_architecturesWithLayer (562.33s)
=== CONT  TestAccLambdaFunction_concurrency
--- PASS: TestAccLambdaFunction_ipv6AllowedForDualStack (701.53s)
=== CONT  TestAccLambdaFunction_KMSKeyARN_noEnvironmentVariables
--- PASS: TestAccLambdaFunction_emptyVPC (643.83s)
--- PASS: TestAccLambdaFunction_fileSystem (907.72s)
--- PASS: TestAccLambdaFunction_VPCPublishNo_changes (938.12s)
--- PASS: TestAccLambdaFunction_VPC_replaceSGWithCustom (1056.70s)
--- PASS: TestAccLambdaFunction_architecturesUpdate (1295.23s)
--- PASS: TestAccLambdaFunction_VPC_properIAMDependencies (1056.01s)
--- PASS: TestAccLambdaFunction_VPC_replaceSGWithDefault (1226.73s)
--- PASS: TestAccLambdaFunction_enablePublish (1038.48s)
--- PASS: TestAccLambdaFunction_unpublishedCodeUpdate (1479.15s)
--- PASS: TestAccLambdaFunction_encryptedEnvVariables (1228.87s)
--- PASS: TestAccLambdaFunction_VPCPublishHas_changes (1424.59s)
--- PASS: TestAccLambdaFunction_vpc (1569.97s)
--- PASS: TestAccLambdaFunction_layers (1546.31s)
--- PASS: TestAccLambdaFunction_VPC_withInvocation (1588.27s)
--- PASS: TestAccLambdaFunction_KMSKeyARN_noEnvironmentVariables (1654.83s)
--- PASS: TestAccLambdaFunction_concurrency (1782.71s)
--- PASS: TestAccLambdaFunction_layersUpdate (1924.70s)
--- PASS: TestAccLambdaFunction_versionedUpdate (2040.51s)
--- PASS: TestAccLambdaFunction_vpcUpdate (2161.88s)
--- PASS: TestAccLambdaFunction_vpcRemoval (2161.44s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/lambda     2558.481s
```
@jar-b jar-b force-pushed the f-lambda_replace_sg_on_destroy branch from 2299b1a to bc3a7d6 Compare May 21, 2024 17:48
@johnsonaj johnsonaj self-assigned this May 23, 2024
Copy link
Contributor

@johnsonaj johnsonaj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

make testacc TESTARGS="-run=TestAccLambdaFunction_" PKG=lambda
      
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go1.22.2 test ./internal/service/lambda/... -v -count 1 -parallel 20  -run=TestAccLambdaFunction_ -timeout 360m
    function_test.go:977: Environment variable AWS_LAMBDA_IMAGE_LATEST_ID is not set
--- SKIP: TestAccLambdaFunction_image (0.00s)
--- PASS: TestAccLambdaFunction_expectFilenameAndS3Attributes (4.02s)
--- PASS: TestAccLambdaFunction_nameValidation (4.10s)
--- PASS: TestAccLambdaFunction_nilDeadLetter (39.09s)
--- PASS: TestAccLambdaFunction_basic (62.38s)
--- PASS: TestAccLambdaFunction_codeSigning (93.68s)
--- PASS: TestAccLambdaFunction_EnvironmentVariables_emptyUpgrade (103.21s)
--- PASS: TestAccLambdaFunction_enablePublish (104.88s)
--- PASS: TestAccLambdaFunction_versioned (109.50s)
--- PASS: TestAccLambdaFunction_architectures (111.37s)
--- PASS: TestAccLambdaFunction_tracing (113.27s)
--- PASS: TestAccLambdaFunction_Zip_validation (8.38s)
--- PASS: TestAccLambdaFunction_ephemeralStorage (125.22s)
--- PASS: TestAccLambdaFunction_disablePublish (127.36s)
--- PASS: TestAccLambdaFunction_deadLetter (142.41s)
--- PASS: TestAccLambdaFunction_disappears (110.90s)
--- PASS: TestAccLambdaFunction_deadLetterUpdated (154.70s)
--- PASS: TestAccLambdaFunction_architecturesUpdate (166.02s)
--- PASS: TestAccLambdaFunction_tags (166.52s)
--- PASS: TestAccLambdaFunction_emptyVPC (74.13s)
--- PASS: TestAccLambdaFunction_versionedUpdate (178.83s)
--- PASS: TestAccLambdaFunction_skipDestroyInconsistentPlan (74.78s)
--- PASS: TestAccLambdaFunction_loggingConfig (183.65s)
--- PASS: TestAccLambdaFunction_concurrency (91.23s)
--- PASS: TestAccLambdaFunction_S3Update_unversioned (60.77s)
--- PASS: TestAccLambdaFunction_encryptedEnvVariables (190.86s)
--- PASS: TestAccLambdaFunction_concurrencyCycle (130.00s)
--- PASS: TestAccLambdaFunction_s3 (33.28s)
--- PASS: TestAccLambdaFunction_S3Update_basic (58.38s)
--- PASS: TestAccLambdaFunction_skipDestroy (255.36s)
--- PASS: TestAccLambdaFunction_unpublishedCodeUpdate (369.37s)
--- PASS: TestAccLambdaFunction_snapStart (275.05s)
--- PASS: TestAccLambdaFunction_architecturesWithLayer (583.84s)
--- PASS: TestAccLambdaFunction_fileSystem (709.12s)
--- PASS: TestAccLambdaFunction_runtimes (588.08s)
--- PASS: TestAccLambdaFunction_VPC_replaceSGWithCustom (831.64s)
--- PASS: TestAccLambdaFunction_ipv6AllowedForDualStack (1012.37s)
--- PASS: TestAccLambdaFunction_VPC_replaceSGWithDefault (1011.01s)
--- PASS: TestAccLambdaFunction_VPC_properIAMDependencies (1011.08s)
--- PASS: TestAccLambdaFunction_layers (1365.35s)
--- PASS: TestAccLambdaFunction_VPCPublishHas_changes (1733.61s)
--- PASS: TestAccLambdaFunction_layersUpdate (1773.69s)
--- PASS: TestAccLambdaFunction_vpc (1943.59s)
--- PASS: TestAccLambdaFunction_EnvironmentVariables_noValue (1978.75s)
--- PASS: TestAccLambdaFunction_KMSKeyARN_noEnvironmentVariables (1955.97s)
--- PASS: TestAccLambdaFunction_LocalUpdate_nameOnly (2206.28s)
--- PASS: TestAccLambdaFunction_localUpdate (2207.04s)
--- PASS: TestAccLambdaFunction_envVariables (2021.15s)
--- PASS: TestAccLambdaFunction_vpcUpdate (2384.40s)
--- PASS: TestAccLambdaFunction_vpcRemoval (2385.00s)
--- PASS: TestAccLambdaFunction_VPC_withInvocation (2565.87s)
--- PASS: TestAccLambdaFunction_VPCPublishNo_changes (2889.96s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/lambda	3080.097s

@jar-b jar-b merged commit b3cea5e into main May 23, 2024
50 checks passed
@jar-b jar-b deleted the f-lambda_replace_sg_on_destroy branch May 23, 2024 15:35
@github-actions github-actions bot added this to the v5.51.0 milestone May 23, 2024
@github-actions github-actions bot removed the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label May 24, 2024
Copy link

This functionality has been released in v5.51.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jun 23, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Introduces or discusses updates to documentation. service/lambda Issues and PRs that pertain to the lambda service. size/M Managed by automation to categorize the size of a PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

r/aws_lambda_function: Remove replace_security_groups_on_destroy attribute
2 participants