Add support for enabling cluster debugging to EMR

[gudlyf]

The option to enable debugging to an EMR cluster is currently not present in the 'aws_emr_cluster' resource. This option cannot be added after the cluster is created and must be done at time of cluster creation.

Reference: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-debugging.html
Reference: http://docs.aws.amazon.com/cli/latest/reference/emr/create-cluster.html (NOTE: --enable-debugging flag)

[bflad]

The AWS CLI option --enable-debugging is a customization in the CLI that wraps adding a step:

https://github.com/aws/aws-cli/blob/c172027fc12b9709f4082825469adc3a66a73652/awscli/customizations/emr/createcluster.py#L462-L476

Theoretically we should be able to provide the functionality and also enable other use cases by supporting a step list for the aws_emr_cluster resource:

# Not implemented - interface could change
step {
  action = "TERMINATE_CLUSTER"
  name   = "Setup Hadoop Debugging"

  hadoop_jar_step {
    jar  = "command-runner.jar"
    args = ["state-pusher-script"]
  }
}

I'll work on creating a PR for this right now. Related issues: #1593 and #1714

[bflad]

Support for configuring steps during cluster creation has been merged into master via #3673 and will release with v1.12.0 of the AWS provider, likely in about a week.

[bflad]

This has been released in version 1.12.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[dpnova]

Anyone actually managed to get this to work. Where do you specify the s3 uri as a destination for the log files?

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!