[WIP] Image Builder support

[Dogers]

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #11084

Release note for CHANGELOG:

 * Add support for EC2 Image Builder

Output from acceptance testing:

$ make testacc TESTARGS='-run=TestAccXXX'

...

TODO:

• Tests for everything!
• Distribution Configuration component
• Image component
• Pipeline component
• Neaten up the data sources - currently only working off ARNs to get going
• Docs for each component

I feel some of this is pretty wordy, so some guidance on the element names would be handy :)

[teamterraform]

Notification of Recent and Upcoming Changes to Contributions

Thank you for this contribution! There have been a few recent development changes that affect this pull request. We apologize for the inconvenience, especially if there have been long review delays up until now. Please note that this is automated message from an unmonitored account. See the FAQ for additional information on the maintainer team and review prioritization.

If you are unable to complete these updates, please leave a comment for the community and maintainers so someone can potentially continue the work. The maintainers will encourage other contributors to use the existing contribution as the base for additional changes as appropriate. Otherwise, contributions that do not receive updated code or comments from the original contributor may be closed in the future so the maintainers can focus on active items.

For the most up to date information about Terraform AWS Provider development, see the Contributing Guide. Additional technical debt changes can be tracked with the technical-debt label on issues.

As part of updating a pull request with these changes, the most current unit testing and linting will run. These may report issues that were not previously reported.

Terraform 0.12 Syntax

Reference: #8950
Reference: #14417

Version 3 and later of the Terraform AWS Provider, which all existing contributions would potentially be added, only supports Terraform 0.12 and later. Certain syntax elements of Terraform 0.11 and earlier show deprecation warnings during runs with Terraform 0.12. Documentation and test configurations, such as those including deprecated string interpolations (some_attribute = "${aws_service_thing.example.id}") should be updated to the newer syntax (some_attribute = aws_service_thing.example.id). Contribution testing will automatically fail on older syntax in the near future. Please see the referenced issues for additional information.

Action Required: Terraform Plugin SDK Version 2

Reference: #14551

The Terraform AWS Provider has been upgraded to the latest version of the Terraform Plugin SDK. Generally, most changes to contributions should only involve updating Go import paths in source code files. Please see the referenced issue for additional information.

Action Required: Removal of website/aws.erb File

Reference: #14712

Any changes to the website/aws.erb file are no longer necessary and should be removed from this contribution to prevent merge issues in the near future when the file is removed from the repository. Please see the referenced issue for additional information.

Upcoming Change of Git Branch Naming

Reference: #14292

Development environments will need their upstream Git branch updated from master to main in the near future. Please see the referenced issue for additional information and scheduling.

Upcoming Change of GitHub Organization

Reference: #14715

This repository will be migrating from https://github.com/terraform-providers/terraform-provider-aws to https://github.com/hashicorp/terraform-provider-aws. No practitioner or developer action is anticipated and most GitHub functionality will automatically redirect to the new location. Go import paths including terraform-providers can remain for now. Please see the referenced issue for additional information and scheduling.

[Dogers]

True - I think the main thing is for getting the ARN of the Amazon provided base image to build on. Was thinking it'd be good to have some way to always find the current version of it rather than having to go and manually check in the console initially first?

Having just gone back to look at the recipe though, it seems you can use a wildcard in the version (or indeed an AMI ID so we could just use the aws_ami object?), so perhaps it's not needed at all then..? I've not used the image builder in the console manually yet so not sure!

[wrschneider]

Having just gone back to look at the recipe though, it seems you can use a wildcard in the version (or indeed an AMI ID so we could just use the aws_ami object?), so perhaps it's not needed at all then..? I've not used the image builder in the console manually yet so not sure!

Yes, if your goal is just to get the latest version of a parent image, it looks like that is encoded into the ARN itself.

https://docs.aws.amazon.com/imagebuilder/latest/APIReference/API_CreateImageRecipe.html
"The format for the ARN follows this example: arn:aws:imagebuilder:us-west-2:aws:image/windows-server-2016-english-full-base-x86/xxxx.x.x. You can provide the specific version that you want to use, or you can use a wildcard in all of the fields"

One recipe I created through the console with the "always use latest version" box checked, ended up with an image ARN arn:aws:imagebuilder:us-east-1:aws:image/amazon-linux-2-x86/x.x.x

This worked for me

resource "aws_imagebuilder_recipe" "recipe" {
  name = "recipe"
  parent_image = "arn:aws:imagebuilder:us-east-1:aws:image/amazon-linux-2-x86/x.x.x"
  semantic_version = "1.0.0"
  components = ["arn:aws:imagebuilder:us-east-1:aws:component/amazon-cloudwatch-agent-linux/1.0.0/1"]
}

[bflad]

Hi @Dogers 👋 First of all, wow! Nice work so far and thank you for stepping up for this large new service. Apologies on the delayed review here towards the end of our Q3 Roadmap timeframe.

The most important initial item to discuss here is, what is your availability for the next few weeks (not too many though) to work through feedback cycles? The maintainers want to offer you the opportunity to keep working on this through completion within a reasonable timeframe for getting this out to the community as promised. If you are not interested or otherwise do not have time, no pressure, and we can totally pick up from where this is currently at, ensuring that your commits are included as part of contributing. If we don't hear back by November 6th, we will move forward with this ourselves.

---

If we are to collaborate on this further, the maintainer team completed an internal service design review for this recently and many of the resources/data sources align here. However, it is difficult for us to focus on providing meaningful feedback on each component as one large change set, especially for review items that may need to be repeated everywhere. Splitting this up by resource (pairing the associated data source is fine) will also allow us to iteratively release functionality as it is ready, rather than having everything pending on potentially minor and unrelated items to those already completed. See the Contributing Guide section on New Services for more details about this request.

As a means of tracking where this currently stands for everyone watching this pull request before its split, here are the present (✅ -- not necessarily approving of what is present) versus missing (⚠️ ) items from a general point of view:

Type	Name	Code	Testing	Documentation
Data Source	aws_imagebuilder_component	✅	✅	⚠️
Data Source	aws_imagebuilder_distribution_configuration	✅	⚠️	⚠️
Data Source	aws_imagebuilder_image_pipeline	✅	⚠️	⚠️
Data Source	aws_imagebuilder_infrastructure_configuration	✅	⚠️	⚠️
Data Source	aws_imagebuilder_recipe	✅†	⚠️	⚠️
Resource	aws_imagebuilder_component	✅	⚠️	⚠️
Resource	aws_imagebuilder_distribution_configuration	✅	⚠️	⚠️
Resource	aws_imagebuilder_image_pipeline	✅	⚠️	⚠️
Resource	aws_imagebuilder_infrastructure_configuration	✅	✅	✅
Resource	aws_imagebuilder_recipe	✅	⚠️	⚠️

† file is incorrectly named

---

Thanks again, @Dogers, and please reach out if you have any questions.

[Dogers]

@bflad apologies for the silence! Absolutely happy that you've picked this up - just hope my spaghettigo didn't cause too many headaches in the tidyup :)

Presume this request can/should be closed?

[bflad]

All the commits from this pull request have been merged in some fashion. Thanks again, @Dogers!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!