[Bug]: credential_process broken in 5.27.0

[ryanpodonnell1]

Terraform Core Version

Terraform v1.6.2 on darwin_amd64

AWS Provider Version

5.27.0

Affected Resource(s)

aws provider authentication using the credential process on 5.27.0

5.26.0 works as expected

Expected Behavior

should authenticate properly using the credential_process to source credentials

Actual Behavior

fails to source AWS credentials from an external process

Relevant Error/Panic Output Snippet

╷
│ Error: No valid credential sources found
│ 
│   with provider["registry.terraform.io/hashicorp/aws"],
│   on <empty> line 0:
│   (source code not available)
│ 
│ Please see https://registry.terraform.io/providers/hashicorp/aws
│ for more information about providing credentials.
│ 
│ Error: failed to refresh cached credentials, operation error STS: AssumeRole, get identity: get credentials: process provider error: error in credential_process: exit status 127
│ 
╵

Terraform Configuration Files

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "5.27.0"
    }
  }
}
provider "aws" {}

data "aws_caller_identity" "current" {}

output "caller" {
  value = data.aws_caller_identity.current.id
}

Steps to Reproduce

1. create a config file that uses the credential process (we use something internal but works fine for 5.26.0)

[profile sample]
credential_process = '<some CLI command that generates the expected JSON output (see article above)'

2. set env variables:

AWS_CONFIG_FILE=<config_file>
AWS_DEFAULT_REGION=us-west-2
AWS_PROFILE=<profile_name that points to a credential process>
AWS_SDK_LOAD_CONFIG=1

3. terraform apply --auto-approve
4. error detailing unable to find credentials

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

No

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[guillaumekh]

Confirmed, authentication w/ credential_process is currently broken.

v5.13.1: OK
v5.32.1: broken

Since the bug was introduced w/ v5.27.0, it is likely caused by the AWS SDK for Go 11/17/2023 update #34476

Should this be reported upstream? I'm not too familiar w/ Go and unsure how to further troubleshoot this. Happy to help though if someone can point me in the right direction.

[guillaumekh]

The issue is upstream. I have reproduced it w/ this simple go file:

package main

import (
	"context"
	"log"
	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/s3"
)

func main() {
	// Load the Shared AWS Configuration (~/.aws/config)
	cfg, err := config.LoadDefaultConfig(context.TODO())
	if err != nil {
		log.Fatal(err)
	}

	// Create an Amazon S3 service client
	client := s3.NewFromConfig(cfg)

	// Get the first page of results for ListObjectsV2 for a bucket
	output, err := client.ListObjectsV2(context.TODO(), &s3.ListObjectsV2Input{
		Bucket: aws.String("myBucketName"),
	})
	if err != nil {
		log.Fatal(err)
	}

	log.Println("first page results:")
	for _, object := range output.Contents {
		log.Printf("key=%s size=%d", aws.ToString(object.Key), object.Size)
	}
}

Tested w/
aws-sdk-go-v2 v1.24.1
aws-sdk-go-v2/config v1.26.3
aws-sdk-go-v2/credentials v1.16.14

I'm inclined to think this PR is the root cause: feat: ini2 parser aws/aws-sdk-go-v2#2365

[guillaumekh]

I have opened an issue upstream: aws/aws-sdk-go-v2#2455

[jmandel1027]

We're facing this issue too, we're loading our default config like above. We're rolling with:

	github.com/aws/aws-sdk-go-v2 v1.24.1
	github.com/aws/aws-sdk-go-v2/config v1.26.3
	github.com/aws/aws-sdk-go-v2/credentials v1.16.14
	github.com/aws/aws-sdk-go-v2/service/s3 v1.48.0
	github.com/aws/aws-sdk-go-v2/service/secretsmanager v1.26.2
	github.com/aws/aws-sdk-go-v2/service/sts v1.26.7
	github.com/bradleyfalzon/ghinstallation/v2 v2.9.0

[guillaumekh]

fixed upstream aws/aws-sdk-go-v2#2461

[justinretzolk]

Looks like that update was pulled in at version 5.37.0 of the AWS provider, so marking this as closed. If you experience the issue again in later versions, please open a new ticket so that we can triage it as necessary.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.