EMR recreates resources when ebs_config.volumes_per_instance is greater than 1

[software-opal]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.13

Affected Resource(s)

• aws_emr_cluster

Terraform Configuration Files

resource "aws_emr_cluster" "emr_cluster" {
  name          = "my-cluster"
  release_label = "emr-5.26.0"
  applications  = ["Hadoop", "Hive", "Pig", "Hue", "Spark"]

  ebs_root_volume_size = 10

  termination_protection            = false
  keep_job_flow_alive_when_no_steps = true
  scale_down_behavior               = "TERMINATE_AT_TASK_COMPLETION"

  master_instance_group {
    instance_type = "m5.xlarge"

    ebs_config {
      size                 = 32
      type                 = "gp2"
      volumes_per_instance = 2
    }
  }

  core_instance_group {
    instance_type  = "m5.xlarge"
    instance_count = 2

    ebs_config {
      size                 = 128
      type                 = "gp2"
      volumes_per_instance = 2
    }
  }
}

Expected Behavior

Subsequent terraform apply calls should not recreate the cluster when the resource hasn't changed.

Actual Behavior

The cluster is recreated every time a terraform apply is run.

-/+ aws_emr_cluster.emr_cluster (new resource required)
...snip...
      core_instance_group.0.ebs_config.2569541827.iops:                   "0" => "0"
      core_instance_group.0.ebs_config.2569541827.size:                   "128" => "0" (forces new resource)
      core_instance_group.0.ebs_config.2569541827.type:                   "gp2" => "" (forces new resource)
      core_instance_group.0.ebs_config.2569541827.volumes_per_instance:   "1" => "0" (forces new resource)
      core_instance_group.0.ebs_config.2986691328.iops:                   "" => ""
      core_instance_group.0.ebs_config.2986691328.size:                   "" => "128" (forces new resource)
      core_instance_group.0.ebs_config.2986691328.type:                   "" => "gp2" (forces new resource)
      core_instance_group.0.ebs_config.2986691328.volumes_per_instance:   "" => "2" (forces new resource)
      core_instance_group.0.id:                                           "ig-1MCB05JHBWE0E" => <computed>
      core_instance_group.0.instance_count:                               "2" => "2"
      core_instance_group.0.instance_type:                                "m5.xlarge" => "m5.xlarge"
      core_instance_type:                                                 "m5.xlarge" => <computed>
...snip...
      master_instance_group.0.ebs_config.2636219798.iops:                 "0" => "0"
      master_instance_group.0.ebs_config.2636219798.size:                 "32" => "0" (forces new resource)
      master_instance_group.0.ebs_config.2636219798.type:                 "gp2" => "" (forces new resource)
      master_instance_group.0.ebs_config.2636219798.volumes_per_instance: "1" => "0" (forces new resource)
      master_instance_group.0.ebs_config.3054294613.iops:                 "" => ""
      master_instance_group.0.ebs_config.3054294613.size:                 "" => "32" (forces new resource)
      master_instance_group.0.ebs_config.3054294613.type:                 "" => "gp2" (forces new resource)
      master_instance_group.0.ebs_config.3054294613.volumes_per_instance: "" => "2" (forces new resource)
      master_instance_group.0.id:                                         "ig-B0L2KTB8QGNY" => <computed>
      master_instance_group.0.instance_count:                             "1" => "1"
      master_instance_group.0.instance_type:                              "m5.xlarge" => "m5.xlarge"
...snip...

Steps to Reproduce

1. terraform apply
2. terraform apply

References

• Maybe related to aws_emr_cluster "New resource required" while updating instance_count inside instance_groups #5111, EMR instance gets "forces new instance" when no changes made #5075
• Terraform docs: https://www.terraform.io/docs/providers/aws/r/emr_cluster.html#ebs_config-3
• EMR docs for describe-cluster and list-instances

[admssa]

the same for aws_emr_instance_group resource.
lifecycle ignore_changes has no effect.

[joestump]

Just spent a little time looking. Both aws_emr_instance_group and aws_emr_cluster use the flattenEBSConfig to create/set ebs_config.

Looking at the plan output I'm suspicious that scheme.NewSet and resourceAwsEMRClusterEBSConfigHash are conspiring somehow to make Terraform think the ebs_config set is "new" when it is clearly not:

core_instance_group.0.ebs_config.2569541827.iops:                   "0" => "0"
core_instance_group.0.ebs_config.2569541827.size:                   "128" => "0" (forces new resource)
core_instance_group.0.ebs_config.2569541827.type:                   "gp2" => "" (forces new resource)
core_instance_group.0.ebs_config.2569541827.volumes_per_instance:   "1" => "0" (forces new resource)
core_instance_group.0.ebs_config.2986691328.iops:                   "" => ""
core_instance_group.0.ebs_config.2986691328.size:                   "" => "128" (forces new resource)
core_instance_group.0.ebs_config.2986691328.type:                   "" => "gp2" (forces new resource)
core_instance_group.0.ebs_config.2986691328.volumes_per_instance:   "" => "2" (forces new resource)

[homiakos]

Is there a fix timeline or workaround?

[pmacdougall]

This is happening for me too using:
Terraform v0.12.25
provider.aws v2.63.0

[yizhu-wish]

It looks the # of volumes is hard-coded to 1?
https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_emr_cluster.go#L1745

[deedoz]

It looks the # of volumes is hard-coded to 1?
https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_emr_cluster.go#L1745

Agrees :
`func flattenEBSConfig(ebsBlockDevices []*emr.EbsBlockDevice) *schema.Set {

ebsConfig := make([]interface{}, 0)
for _, ebs := range ebsBlockDevices {
	ebsAttrs := make(map[string]interface{})
	if ebs.VolumeSpecification.Iops != nil {
		ebsAttrs["iops"] = int(*ebs.VolumeSpecification.Iops)
	}
	if ebs.VolumeSpecification.SizeInGB != nil {
		ebsAttrs["size"] = int(*ebs.VolumeSpecification.SizeInGB)
	}
	if ebs.VolumeSpecification.VolumeType != nil {
		ebsAttrs["type"] = *ebs.VolumeSpecification.VolumeType
	}
	ebsAttrs["volumes_per_instance"] = 1

	ebsConfig = append(ebsConfig, ebsAttrs)
}

return schema.NewSet(resourceAwsEMRClusterEBSConfigHash, ebsConfig)

}`

ebsAttrs["volumes_per_instance"] = 1

[dusty73]

It looks the # of volumes is hard-coded to 1?
https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_emr_cluster.go#L1745

Agrees :
`func flattenEBSConfig(ebsBlockDevices []*emr.EbsBlockDevice) *schema.Set {

ebsConfig := make([]interface{}, 0)
for _, ebs := range ebsBlockDevices {
	ebsAttrs := make(map[string]interface{})
	if ebs.VolumeSpecification.Iops != nil {
		ebsAttrs["iops"] = int(*ebs.VolumeSpecification.Iops)
	}
	if ebs.VolumeSpecification.SizeInGB != nil {
		ebsAttrs["size"] = int(*ebs.VolumeSpecification.SizeInGB)
	}
	if ebs.VolumeSpecification.VolumeType != nil {
		ebsAttrs["type"] = *ebs.VolumeSpecification.VolumeType
	}
	ebsAttrs["volumes_per_instance"] = 1

	ebsConfig = append(ebsConfig, ebsAttrs)
}

return schema.NewSet(resourceAwsEMRClusterEBSConfigHash, ebsConfig)

}`

ebsAttrs["volumes_per_instance"] = 1

Yes, I think it should be:

ebsAttrs["volumes_per_instance"] = len(ebsBlockDevices)

[c4po]

I created a PR to fix this bug and add test case to guard this feature.

[bflad]

The fix for this has been merged and will release with version 3.5.0 of the Terraform AWS Provider, later next week. Thanks to @c4po for the implementation. 👍

[ghost]

This has been released in version 3.5.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!