Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Emrserverless add image configuration #30398

Merged

Conversation

samwyma
Copy link
Contributor

@samwyma samwyma commented Apr 2, 2023

Description

Adds the ability to define a custom image for an EMR Serverless application. Also added platform_override option support for AWS ImageBuilder.

When I tried to use a public ecr image (for example public.ecr.aws/emr-serverless/hive/emr-6.9.0) for the imageUri option, I received a 500 error (also occurred when using the aws-cli as well). To get around this, I had to build a custom image off that base and put it in a temporary ECR repo. Unfortunately, this make the new acceptance test a lot more complicated and long running.

Also, sorry if my Go isn't particularly idiomatic - I'm a relative newbie with it! Happy to take on any feedback :).

Relations

Closes #24711
Closes #30230

Output from Acceptance Testing

make testacc TESTS=TestAccEMRServerlessApplication_basic PKG=emrserverless
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/emrserverless/... -v -count 1 -parallel 20 -run='TestAccEMRServerlessApplication_basic'  -timeout 180m
=== RUN   TestAccEMRServerlessApplication_basic
=== PAUSE TestAccEMRServerlessApplication_basic
=== CONT  TestAccEMRServerlessApplication_basic
--- PASS: TestAccEMRServerlessApplication_basic (81.53s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/emrserverless      100.781s

$ make testacc TESTS=TestAccEMRServerlessApplication_imageCon PKG=emrserverless
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/emrserverless/... -v -count 1 -parallel 20 -run='TestAccEMRServerlessApplication_imageCon'  -timeout 180m
=== RUN   TestAccEMRServerlessApplication_imageConfiguration
=== PAUSE TestAccEMRServerlessApplication_imageConfiguration
=== CONT  TestAccEMRServerlessApplication_imageConfiguration
--- PASS: TestAccEMRServerlessApplication_imageConfiguration (585.52s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/emrserverless      605.158s
...

@github-actions
Copy link

github-actions bot commented Apr 2, 2023

Community Note

Voting for Prioritization

  • Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

  • Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
  • For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
  • Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

@github-actions github-actions bot added size/XL Managed by automation to categorize the size of a PR. documentation Introduces or discusses updates to documentation. service/emrserverless Issues and PRs that pertain to the emrserverless service. service/imagebuilder Issues and PRs that pertain to the imagebuilder service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Apr 2, 2023
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome @samwyma 👋

It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTOR guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.

Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.

Thanks again, and welcome to the community! 😃

@samwyma
Copy link
Contributor Author

samwyma commented Apr 2, 2023

Additional test output:

$ make testacc TESTS=TestAccImageBuilderContainerRecipe_ PKG=imagebuilder                            
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/imagebuilder/... -v -count 1 -parallel 20 -run='TestAccImageBuilderContainerRecipe_'  -timeout 180m
=== RUN   TestAccImageBuilderContainerRecipe_basic
=== PAUSE TestAccImageBuilderContainerRecipe_basic
=== RUN   TestAccImageBuilderContainerRecipe_disappears
=== PAUSE TestAccImageBuilderContainerRecipe_disappears
=== RUN   TestAccImageBuilderContainerRecipe_component
=== PAUSE TestAccImageBuilderContainerRecipe_component
=== RUN   TestAccImageBuilderContainerRecipe_componentParameter
=== PAUSE TestAccImageBuilderContainerRecipe_componentParameter
=== RUN   TestAccImageBuilderContainerRecipe_description
=== PAUSE TestAccImageBuilderContainerRecipe_description
=== RUN   TestAccImageBuilderContainerRecipe_dockerfileTemplateURI
=== PAUSE TestAccImageBuilderContainerRecipe_dockerfileTemplateURI
=== RUN   TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_deviceName
=== PAUSE TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_deviceName
=== RUN   TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_deleteOnTermination
=== PAUSE TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_deleteOnTermination
=== RUN   TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_encrypted
=== PAUSE TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_encrypted
=== RUN   TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_iops
=== PAUSE TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_iops
=== RUN   TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_kmsKeyID
=== PAUSE TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_kmsKeyID
=== RUN   TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_snapshotID
=== PAUSE TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_snapshotID
=== RUN   TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_throughput
=== PAUSE TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_throughput
=== RUN   TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_volumeSize
=== PAUSE TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_volumeSize
=== RUN   TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_volumeType
=== PAUSE TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_volumeType
=== RUN   TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_noDevice
=== PAUSE TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_noDevice
=== RUN   TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_virtualName
=== PAUSE TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_virtualName
=== RUN   TestAccImageBuilderContainerRecipe_InstanceConfiguration_Image
=== PAUSE TestAccImageBuilderContainerRecipe_InstanceConfiguration_Image
=== RUN   TestAccImageBuilderContainerRecipe_kmsKeyID
=== PAUSE TestAccImageBuilderContainerRecipe_kmsKeyID
=== RUN   TestAccImageBuilderContainerRecipe_tags
=== PAUSE TestAccImageBuilderContainerRecipe_tags
=== RUN   TestAccImageBuilderContainerRecipe_workingDirectory
=== PAUSE TestAccImageBuilderContainerRecipe_workingDirectory
=== RUN   TestAccImageBuilderContainerRecipe_platformOverride
=== PAUSE TestAccImageBuilderContainerRecipe_platformOverride
=== CONT  TestAccImageBuilderContainerRecipe_basic
=== CONT  TestAccImageBuilderContainerRecipe_dockerfileTemplateURI
=== CONT  TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_deleteOnTermination
=== CONT  TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_throughput
=== CONT  TestAccImageBuilderContainerRecipe_componentParameter
=== CONT  TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_kmsKeyID
=== CONT  TestAccImageBuilderContainerRecipe_description
=== CONT  TestAccImageBuilderContainerRecipe_InstanceConfiguration_Image
=== CONT  TestAccImageBuilderContainerRecipe_platformOverride
=== CONT  TestAccImageBuilderContainerRecipe_workingDirectory
=== CONT  TestAccImageBuilderContainerRecipe_tags
=== CONT  TestAccImageBuilderContainerRecipe_kmsKeyID
=== CONT  TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_volumeType
=== CONT  TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_virtualName
=== CONT  TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_noDevice
=== CONT  TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_volumeSize
=== CONT  TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_encrypted
=== CONT  TestAccImageBuilderContainerRecipe_component
=== CONT  TestAccImageBuilderContainerRecipe_disappears
=== CONT  TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_iops
--- PASS: TestAccImageBuilderContainerRecipe_disappears (47.60s)
=== CONT  TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_snapshotID
--- PASS: TestAccImageBuilderContainerRecipe_workingDirectory (54.10s)
=== CONT  TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_deviceName
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_kmsKeyID (54.45s)
--- PASS: TestAccImageBuilderContainerRecipe_basic (54.45s)
--- PASS: TestAccImageBuilderContainerRecipe_kmsKeyID (54.46s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_noDevice (55.20s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_iops (55.37s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_virtualName (55.58s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_volumeSize (55.80s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_throughput (55.96s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_deleteOnTermination (56.00s)
--- PASS: TestAccImageBuilderContainerRecipe_description (56.02s)
--- PASS: TestAccImageBuilderContainerRecipe_componentParameter (56.57s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_volumeType (56.59s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_Image (56.97s)
--- PASS: TestAccImageBuilderContainerRecipe_platformOverride (57.34s)
--- PASS: TestAccImageBuilderContainerRecipe_dockerfileTemplateURI (57.67s)
--- PASS: TestAccImageBuilderContainerRecipe_component (57.78s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_encrypted (58.03s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_deviceName (27.51s)
--- PASS: TestAccImageBuilderContainerRecipe_tags (90.26s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_snapshotID (71.12s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/imagebuilder       138.195s

@gdavison gdavison self-assigned this Apr 5, 2023
Copy link
Contributor

@gdavison gdavison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, @samwyma! 🚀

--- PASS: TestAccEMRServerlessApplication_basic (155.22s)
--- PASS: TestAccEMRServerlessApplication_network (156.82s)
--- PASS: TestAccEMRServerlessApplication_disappears (162.08s)
--- PASS: TestAccEMRServerlessApplication_maxCapacity (197.66s)
--- PASS: TestAccEMRServerlessApplication_initialCapacity (199.12s)
--- PASS: TestAccEMRServerlessApplication_arch (199.63s)
--- PASS: TestAccEMRServerlessApplication_tags (221.45s)
--- PASS: TestAccEMRServerlessApplication_imageConfiguration (958.93s)
--- PASS: TestAccImageBuilderContainerRecipe_disappears (179.39s)
--- PASS: TestAccImageBuilderContainerRecipeDataSource_arn (189.62s)
--- PASS: TestAccImageBuilderContainerRecipesDataSource_filter (199.13s)
--- PASS: TestAccImageBuilderContainerRecipe_description (204.87s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_deviceName (207.80s)
--- PASS: TestAccImageBuilderContainerRecipe_component (212.88s)
--- PASS: TestAccImageBuilderContainerRecipe_componentParameter (216.84s)
--- PASS: TestAccImageBuilderContainerRecipe_kmsKeyID (217.91s)
--- PASS: TestAccImageBuilderContainerRecipe_workingDirectory (223.07s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_encrypted (223.52s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_volumeType (223.72s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_virtualName (223.74s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_volumeSize (223.92s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMapping_noDevice (224.19s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_Image (228.24s)
--- PASS: TestAccImageBuilderContainerRecipe_platformOverride (228.94s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_deleteOnTermination (229.37s)
--- PASS: TestAccImageBuilderContainerRecipe_dockerfileTemplateURI (229.79s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_snapshotID (245.13s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_kmsKeyID (100.84s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_iops (94.37s)
--- PASS: TestAccImageBuilderContainerRecipe_InstanceConfiguration_BlockDeviceMappingEBS_throughput (79.66s)
--- PASS: TestAccImageBuilderContainerRecipe_basic (85.88s)
--- PASS: TestAccImageBuilderContainerRecipe_tags (294.79s)

@gdavison gdavison merged commit 1ebfb59 into hashicorp:main Apr 6, 2023
@github-actions github-actions bot added this to the v4.62.0 milestone Apr 6, 2023
@github-actions
Copy link

github-actions bot commented Apr 7, 2023

This functionality has been released in v4.62.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions
Copy link

github-actions bot commented May 8, 2023

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Introduces or discusses updates to documentation. service/emrserverless Issues and PRs that pertain to the emrserverless service. service/imagebuilder Issues and PRs that pertain to the imagebuilder service. size/XL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
2 participants