New Resource: aws_ebs_snapshot_copy #3086

jwieringa · 2018-01-22T02:21:21Z

Purpose: Add aws_ebs_snapshot_copy resource

The bulk of the code is taken from resource_aws_ebs_snapshot. Tests are demonstrating the happy path is working.

Example Workflow:

Create and configure EBS volumes with Packer
Utilize Terraform to copy and encrypt snapshots to multiple regions and accounts

provider "aws" {
  region = "us-west-2"
  alias  = "uswest2"
}

provider "aws" {
  region = "us-east-1"
  alias  = "useast1"
}

resource "aws_ebs_volume" "region_test" {
  provider          = "aws.uswest2"
  availability_zone = "us-west-2a"
  size              = 1

  tags {
    Name = "testAccAwsCopySnapshotConfigWithRegions"
  }
}

resource "aws_ebs_snapshot" "region_test" {
  provider  = "aws.uswest2"
  volume_id = "${aws_ebs_volume.region_test.id}"

  tags {
    Name = "testAccAwsCopySnapshotConfigWithRegions"
  }
}

resource "aws_ebs_snapshot_copy" "region_test" {
  provider           = "aws.useast1"
  source_snapshot_id = "${aws_ebs_snapshot.region_test.id}"
  source_region      = "us-west-2"

  tags {
    Name = "testAccAwsCopySnapshotConfigWithRegions"
  }
}

Todos:

Verify acceptance tests
Test copying snapshots across regions
Test with KMS encryption
Add Documentation

jwieringa · 2018-01-22T20:06:09Z

Rebased on master and added a commit to fix acceptance tests:

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSCopySnapshot'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -run=TestAccAWSCopySnapshot -timeout 120m
=== RUN   TestAccAWSCopySnapshot_basic
--- PASS: TestAccAWSCopySnapshot_basic (69.44s)
=== RUN   TestAccAWSCopySnapshot_withDescription
--- PASS: TestAccAWSCopySnapshot_withDescription (68.02s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	137.815s

jwieringa · 2018-01-23T01:06:15Z

aws/resource_aws_copy_snapshot.go

+	req := &ec2.DescribeSnapshotsInput{
+		SnapshotIds: []*string{aws.String(id)},
+	}
+	err := conn.WaitUntilSnapshotCompleted(req)


This function call can result in an loop if incorrect configurations set. Consider a source_snapshot_id which is in us-west-2 and a config which sets source_region to us-east-1. In this scenario, the snapshot shows as pending via the AWS API for a few minutes (2m40s was observed) at which point the following is returned (I assume until the max retries is reached).

2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: <DescribeSnapshotsResponse xmlns="http://ec2.amazonaws.com/doc/2016-11-15/"> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: <requestId>d6c4ece6-eded-4a90-a5dd-204191d8bdc5</requestId> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: <snapshotSet> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: <item> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: <snapshotId>snap-0fb4b3336bc1b1493</snapshotId> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: <volumeId>vol-ffffffff</volumeId> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: <status>error</status> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: <statusMessage>Source snapshot is not found</statusMessage> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: <startTime>2018-01-23T00:05:06.000Z</startTime> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: <progress>0%</progress> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: <ownerId>721355555739</ownerId> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: <volumeSize>0</volumeSize> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: <description/> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: <encrypted>false</encrypted> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: </item> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: </snapshotSet> 2018-01-22T16:14:56.506-0800 [DEBUG] plugin.terraform-provider-aws: </DescribeSnapshotsResponse>

I suspect this could use some more work. At the ten minute mark:

Error: Error applying plan: 1 error(s) occurred: * aws_copy_snapshot.region_test: 1 error(s) occurred: * aws_copy_snapshot.region_test: ResourceNotReady: exceeded wait attempts

jwieringa · 2018-01-26T02:51:25Z

Added documentation and rebased on master.

jwieringa · 2018-01-30T21:27:41Z

Rebased on master to bring up to date with 1.8 release.

jwieringa · 2018-02-10T16:12:26Z

Rebased on master to bring up to date with 1.9 release.

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSCopySnapshot'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -run=TestAccAWSCopySnapshot -timeout 120m
=== RUN   TestAccAWSCopySnapshot_basic
--- PASS: TestAccAWSCopySnapshot_basic (104.07s)
=== RUN   TestAccAWSCopySnapshot_withDescription
--- PASS: TestAccAWSCopySnapshot_withDescription (72.90s)
=== RUN   TestAccAWSCopySnapshot_withRegions
--- PASS: TestAccAWSCopySnapshot_withRegions (122.94s)
=== RUN   TestAccAWSCopySnapshot_withKms
--- PASS: TestAccAWSCopySnapshot_withKms (94.29s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	394.522s

jwieringa · 2018-02-27T18:57:26Z

Rebased on master to bring up to date with 1.10 release.

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSCopySnapshot'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -run=TestAccAWSCopySnapshot -timeout 120m
=== RUN   TestAccAWSCopySnapshot_basic
--- PASS: TestAccAWSCopySnapshot_basic (85.71s)
=== RUN   TestAccAWSCopySnapshot_withDescription
--- PASS: TestAccAWSCopySnapshot_withDescription (106.32s)
=== RUN   TestAccAWSCopySnapshot_withRegions
--- PASS: TestAccAWSCopySnapshot_withRegions (89.16s)
=== RUN   TestAccAWSCopySnapshot_withKms
--- PASS: TestAccAWSCopySnapshot_withKms (118.07s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	399.612s

jantman · 2018-03-25T12:22:52Z

Thanks so much for working on this! I've been using a custom AMI for ECS with a non-standard Block Device Mapping. I can create the AMI itself through Terraform, but I've been having to manually copy the root device snapshot of the official ECS Optimized AMI, and then pass that snapshot ID to terraform. This will make the process much easier for me!

jwieringa · 2018-03-25T16:12:59Z

@jantman Glad it is helpful!

Rebased on master to bring up to date with 1.12.0 release.

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSCopySnapshot'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -run=TestAccAWSCopySnapshot -timeout 120m
=== RUN   TestAccAWSCopySnapshot_basic
--- PASS: TestAccAWSCopySnapshot_basic (75.40s)
=== RUN   TestAccAWSCopySnapshot_withDescription
--- PASS: TestAccAWSCopySnapshot_withDescription (88.13s)
=== RUN   TestAccAWSCopySnapshot_withRegions
--- PASS: TestAccAWSCopySnapshot_withRegions (77.39s)
=== RUN   TestAccAWSCopySnapshot_withKms
--- PASS: TestAccAWSCopySnapshot_withKms (79.17s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	320.490s

jwieringa · 2018-04-11T23:19:26Z

Hi all, it would be great to see if the new resource aws_copy_snapshot is a good candidate for the AWS provider as it would help us out a lot. Please let me know if there is anything I can do.

@radeksimko because you've helped label the PR, are you also able to help schedule a review of this addition?

Used TestAccAWSInstance_multipleRegions as an example: https://github.com/terraform-providers/terraform-provider-aws/blob/2a36215957d15aab4c7260a20742980ba70533cb/aws/resource_aws_instance_test.go#L655

jwieringa · 2018-07-10T18:28:40Z

I took another look at the original naming, aws_copy_snapshot, and I've reasoned that it was not consistent with resources like aws_ami_copy. Thus, I've modified the resource to be aws_ebs_snapshot_copy to conform to aws_<resource>_<action> which I thought matched existing resources better.

@jantman In case you've been using aws_copy_snapshot, 7e7dc07 changes the naming scheme.

I've also rebased on the master branch to resolve conflicts and updated tests so that they are passing again.

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSEbsSnapshotCopy'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -run=TestAccAWSEbsSnapshotCopy -timeout 120m
=== RUN   TestAccAWSEbsSnapshotCopy_basic
--- PASS: TestAccAWSEbsSnapshotCopy_basic (206.51s)
=== RUN   TestAccAWSEbsSnapshotCopy_withDescription
--- PASS: TestAccAWSEbsSnapshotCopy_withDescription (160.51s)
=== RUN   TestAccAWSEbsSnapshotCopy_withRegions
--- PASS: TestAccAWSEbsSnapshotCopy_withRegions (132.78s)
=== RUN   TestAccAWSEbsSnapshotCopy_withKms
--- PASS: TestAccAWSEbsSnapshotCopy_withKms (104.15s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	604.414s

I took another look at the original naming, aws_copy_snapshot, and I've reasoned that it was not consistent with resources like aws_ami_copy. Thus, I've modified the resource to be aws_ebs_snapshot_copy to conform to aws_<resource>_<action> which I thought matched existing resources better.

bflad

Great work, @jwieringa! 🚀 Sorry this wasn't reviewed and merged sooner. 😅

--- PASS: TestAccAWSEbsSnapshotCopy_withRegions (66.01s)
--- PASS: TestAccAWSEbsSnapshotCopy_withDescription (135.70s)
--- PASS: TestAccAWSEbsSnapshotCopy_basic (150.72s)
--- PASS: TestAccAWSEbsSnapshotCopy_withKms (199.33s)

bflad · 2018-10-09T16:32:57Z

website/docs/r/copy_snapshot.html.md

+---
+layout: "aws"
+page_title: "AWS: aws_ebs_snapshot_copy"
+sidebar_current: "docs-aws-resource-ebs-snapshot-copy"


The only thing missing was a link in website/aws.erb 😄 Will fix on merge

jwieringa · 2018-10-09T17:29:33Z

@bflad Thank you for the merge and handling the missing link. 🎆

bflad · 2018-10-10T19:50:26Z

This has been released in version 1.40.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

ghost · 2020-04-02T18:00:35Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

radeksimko changed the title ~~Added resource_aws_copy_snapshot~~ New Resource: aws_copy_snapshot Jan 22, 2018

radeksimko added new-resource Introduces a new resource. service/ec2 Issues and PRs that pertain to the ec2 service. labels Jan 22, 2018

jwieringa force-pushed the f-add-aws-copy-snapshot branch from 74bc52e to db64c72 Compare January 22, 2018 20:05

jwieringa commented Jan 23, 2018

View reviewed changes

jwieringa force-pushed the f-add-aws-copy-snapshot branch 2 times, most recently from 8d5caba to f252278 Compare January 26, 2018 02:50

jwieringa force-pushed the f-add-aws-copy-snapshot branch from f252278 to 8397cc3 Compare January 30, 2018 21:26

ghost added the size/XL Managed by automation to categorize the size of a PR. label Jan 30, 2018

jwieringa force-pushed the f-add-aws-copy-snapshot branch from 8397cc3 to c0307e6 Compare February 10, 2018 16:05

ghost added the size/XL Managed by automation to categorize the size of a PR. label Feb 10, 2018

jwieringa force-pushed the f-add-aws-copy-snapshot branch from c0307e6 to 937dc96 Compare February 27, 2018 18:40

ghost added the size/XL Managed by automation to categorize the size of a PR. label Feb 27, 2018

jwieringa force-pushed the f-add-aws-copy-snapshot branch from 937dc96 to a98c717 Compare March 25, 2018 16:12

ghost added the size/XL Managed by automation to categorize the size of a PR. label Mar 25, 2018

jwieringa added 4 commits July 10, 2018 10:39

Added resource_aws_copy_snapshot

e2e8042

Added aws_copy_snapshot multi region acceptance test

226f5ce

Used TestAccAWSInstance_multipleRegions as an example: https://github.com/terraform-providers/terraform-provider-aws/blob/2a36215957d15aab4c7260a20742980ba70533cb/aws/resource_aws_instance_test.go#L655

Added aws_copy_snapshot with KMS acceptance test

f414b7d

website/docs/r/copy_snapshot added documentation for new resource

0fe1b9c

jwieringa changed the title ~~New Resource: aws_copy_snapshot~~ New Resource: aws_ebs_snapshot_copy Jul 10, 2018

jwieringa force-pushed the f-add-aws-copy-snapshot branch from a98c717 to 8250874 Compare July 10, 2018 18:27

ghost added the size/XL Managed by automation to categorize the size of a PR. label Jul 10, 2018

jwieringa force-pushed the f-add-aws-copy-snapshot branch from 8250874 to 7e7dc07 Compare July 10, 2018 18:37

ghost added the size/XL Managed by automation to categorize the size of a PR. label Jul 10, 2018

bflad added this to the v1.40.0 milestone Oct 9, 2018

bflad approved these changes Oct 9, 2018

View reviewed changes

bflad merged commit 7c65edf into hashicorp:master Oct 9, 2018

bflad added a commit that referenced this pull request Oct 9, 2018

Update CHANGELOG for #3086

ff8a93c

jwieringa deleted the f-add-aws-copy-snapshot branch October 9, 2018 17:28

ghost locked and limited conversation to collaborators Apr 2, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Resource: aws_ebs_snapshot_copy #3086

New Resource: aws_ebs_snapshot_copy #3086

jwieringa commented Jan 22, 2018 •

edited

Loading

jwieringa commented Jan 22, 2018

jwieringa Jan 23, 2018 •

edited

Loading

jwieringa commented Jan 26, 2018

jwieringa commented Jan 30, 2018

jwieringa commented Feb 10, 2018

jwieringa commented Feb 27, 2018

jantman commented Mar 25, 2018

jwieringa commented Mar 25, 2018

jwieringa commented Apr 11, 2018

jwieringa commented Jul 10, 2018 •

edited

Loading

bflad left a comment

bflad Oct 9, 2018 •

edited

Loading

jwieringa commented Oct 9, 2018

bflad commented Oct 10, 2018

ghost commented Apr 2, 2020

New Resource: aws_ebs_snapshot_copy #3086

New Resource: aws_ebs_snapshot_copy #3086

Conversation

jwieringa commented Jan 22, 2018 • edited Loading

jwieringa commented Jan 22, 2018

jwieringa Jan 23, 2018 • edited Loading

Choose a reason for hiding this comment

jwieringa commented Jan 26, 2018

jwieringa commented Jan 30, 2018

jwieringa commented Feb 10, 2018

jwieringa commented Feb 27, 2018

jantman commented Mar 25, 2018

jwieringa commented Mar 25, 2018

jwieringa commented Apr 11, 2018

jwieringa commented Jul 10, 2018 • edited Loading

bflad left a comment

Choose a reason for hiding this comment

bflad Oct 9, 2018 • edited Loading

Choose a reason for hiding this comment

jwieringa commented Oct 9, 2018

bflad commented Oct 10, 2018

ghost commented Apr 2, 2020

jwieringa commented Jan 22, 2018 •

edited

Loading

jwieringa Jan 23, 2018 •

edited

Loading

jwieringa commented Jul 10, 2018 •

edited

Loading

bflad Oct 9, 2018 •

edited

Loading