Make S3 backend work with credential_process w/out ENV variable

[OJFord]

Terraform Version

Tested each of:

v0.11.14
v0.12.5

Terraform Configuration Files

provider "aws" {
  version = ">=2.3.0"
  region  = "${var.region}"
  profile = "<name>" # tested with and without this line
}

terraform {
   required_version = ">= 0.11.8"

   backend "s3" {
     encrypt = true
     bucket  = "<bucket>"
     region  = "${var.region}"
     key     = "<key>"
   }
 }

Debug Output

2019/07/22 12:59:19 [INFO] Terraform version: 0.11.14
2019/07/22 12:59:19 [INFO] Go runtime version: go1.12.5
[...]
Initializing the backend...
2019/07/22 12:59:20 [TRACE] Preserving existing state lineage "[...]"
2019/07/22 12:59:20 [TRACE] Preserving existing state lineage "[...]"
2019/07/22 12:59:20 [INFO] Setting AWS metadata API timeout to 100ms
2019/07/22 12:59:20 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id

2019/07/22 12:59:25 [DEBUG] plugin: waiting for all plugin processes to complete...
Error configuring the backend "s3": No valid credential sources found for AWS Provider.
	Please see https://terraform.io/docs/providers/aws/index.html for more information on
	providing credentials for the AWS Provider

Please update the configuration in your Terraform files to fix this error.
If you'd like to update the configuration interactively without storing
the values in your configuration, run "terraform init".

and

2019/07/22 13:03:49 [INFO] Terraform version: 0.12.5
2019/07/22 13:03:49 [INFO] Go runtime version: go1.12.7
[...]
Initializing the backend...
2019/07/22 13:04:01 [TRACE] Meta.Backend: built configuration for "s3" backend with hash value [...]
2019/07/22 13:04:01 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2019/07/22 13:04:01 [DEBUG] New state was assigned lineage "[...]"
2019/07/22 13:04:01 [TRACE] Meta.Backend: moving from default local state only to "s3" backend
2019/07/22 13:04:01 [INFO] Setting AWS metadata API timeout to 100ms
2019/07/22 13:04:01 [INFO] Ignoring AWS metadata API endpoint at default location as it doesn't return any instance-id

Error: No valid credential sources found for AWS Provider.
	Please see https://terraform.io/docs/providers/aws/index.html for more information on
	providing credentials for the AWS Provider

Expected Behavior

Connects to S3

Actual Behavior

Error as above, but aws configure list shows correct configuration, as above with either $AWS_DEFAULT_PROFILE set, or the profile marked default in $AWS_CONFIG_FILE, and with or without profile = in the provider.

Steps to Reproduce

1. terraform init

Additional Context

# $AWS_CONFIG_FILE
[profile <name>]
credential_process=<cmd that spits out creds>
region=<region>

References

#21120 claims this support for credential_process is implemented and working in 0.11.14

[OJFord]

Oh there's some fantastically thorough testing in hashicorp/terraform-provider-aws#6913 (comment)

[OJFord]

Workaround: export AWS_SDK_LOAD_CONFIG=1

From: hashicorp/terraform-provider-aws#6913 (comment)

@lorengordon comments:

It would be nice if terraform setup the session so that it was not necessary to use AWS_SDK_LOAD_CONFIG (it is supported it the SDK, just need to pass the config option when creating the session).

[bflad]

Multiple fixes for credential ordering, automatically using the AWS shared configuration file if present, and profile configuration handling of the S3 Backend have been merged and will release with version 0.13.0-beta2 of Terraform.

[OJFord]

Splendid, thank you!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.