-
Notifications
You must be signed in to change notification settings - Fork 9.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
AWS_PROFILE
not respected for S3 backend when running terraform init
/terraform workspace
#20062
Comments
AWS_PROFILE
not respected for S3 backend when running terraform init
AWS_PROFILE
not respected for S3 backend when running terraform init
/terraform workspace
Hi @Stretch96, From the outputs you have posted, seems your aws cli profile is not setup to use your new user. Can you try to setup the profile using |
Hi @abiydv, the aws cli profile is set up correctly, eg: ~/.aws/credentials
~/.aws/config:
As shown in the examples, it does work, but |
@Stretch96 I've encountered this issue and the root of it is that terraform doesn't seem to support role assumption within credential profiles. The only workaround I've found is the one mentioned. |
I've found a way to make terraform crash with If you attempt to set the
Then run:
Produces the crash output:
|
I ran into the same issue. Simply specifying the profile in terraform {
backend "s3" {
# ...
profile = "develop"
}
} |
see #22377 as well |
I had the same issue. This is how I fixed it! According to the documentation, "If you're running Terraform from an EC2 instance with IAM Instance Profile using IAM Role, Terraform will just ask the metadata API endpoint for credentials." Therefore it always taking Instance Role instead of specified profile. So I set environment variable to overwrite aws metadata url.
So totally you should insert 3 environment variables:
If you need to know what happens, then try adding another environment variable for debugging:
|
Reference: #13410 Reference: #18774 Reference: #19482 Reference: #20062 Reference: #20599 Reference: #22103 Reference: #22161 Reference: #22601 Reference: #22992 Reference: #24252 Reference: #24253 Reference: #24480 Reference: #25056 Changes: ``` NOTES * backend/s3: Deprecated `lock_table`, `skip_get_ec2_platforms`, `skip_requesting_account_id` arguments have been removed * backend/s3: Credential ordering has changed from static, environment, shared credentials, EC2 metadata, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) to static, environment, shared credentials, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) * The `AWS_METADATA_TIMEOUT` environment variable no longer has any effect as we now depend on the default AWS Go SDK EC2 Metadata client timeout of one second with two retries ENHANCEMENTS * backend/s3: Always enable shared configuration file support (no longer require `AWS_SDK_LOAD_CONFIG` environment variable) * backend/s3: Automatically expand `~` prefix for home directories in `shared_credentials_file` argument * backend/s3: Add `assume_role_duration_seconds`, `assume_role_policy_arns`, `assume_role_tags`, and `assume_role_transitive_tag_keys` arguments BUG FIXES * backend/s3: Ensure configured profile is used * backend/s3: Ensure configured STS endpoint is used during AssumeRole API calls * backend/s3: Prefer AWS shared configuration over EC2 metadata credentials * backend/s3: Prefer ECS credentials over EC2 metadata credentials * backend/s3: Remove hardcoded AWS Provider messaging ``` Output from acceptance testing: ``` --- PASS: TestBackend (16.32s) --- PASS: TestBackendConfig (0.58s) --- PASS: TestBackendConfig_AssumeRole (0.02s) --- PASS: TestBackendConfig_conflictingEncryptionSchema (0.00s) --- PASS: TestBackendConfig_invalidKey (0.00s) --- PASS: TestBackendConfig_invalidSSECustomerKeyEncoding (0.00s) --- PASS: TestBackendConfig_invalidSSECustomerKeyLength (0.00s) --- PASS: TestBackendExtraPaths (13.21s) --- PASS: TestBackendLocked (28.98s) --- PASS: TestBackendPrefixInWorkspace (5.65s) --- PASS: TestBackendSSECustomerKey (17.60s) --- PASS: TestBackend_impl (0.00s) --- PASS: TestForceUnlock (17.50s) --- PASS: TestKeyEnv (50.25s) --- PASS: TestRemoteClient (4.78s) --- PASS: TestRemoteClientLocks (16.85s) --- PASS: TestRemoteClient_clientMD5 (12.08s) --- PASS: TestRemoteClient_impl (0.00s) --- PASS: TestRemoteClient_stateChecksum (17.92s) ```
* deps: Update github.com/hashicorp/aws-sdk-go-base@v0.5.0 Updated via: ``` $ go get github.com/hashicorp/aws-sdk-go-base@v0.5.0 $ go mod tidy $ go mod vendor ``` * backend/s3: Updates for Terraform v0.13.0 Reference: #13410 Reference: #18774 Reference: #19482 Reference: #20062 Reference: #20599 Reference: #22103 Reference: #22161 Reference: #22601 Reference: #22992 Reference: #24252 Reference: #24253 Reference: #24480 Reference: #25056 Changes: ``` NOTES * backend/s3: Deprecated `lock_table`, `skip_get_ec2_platforms`, `skip_requesting_account_id` arguments have been removed * backend/s3: Credential ordering has changed from static, environment, shared credentials, EC2 metadata, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) to static, environment, shared credentials, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) * The `AWS_METADATA_TIMEOUT` environment variable no longer has any effect as we now depend on the default AWS Go SDK EC2 Metadata client timeout of one second with two retries ENHANCEMENTS * backend/s3: Always enable shared configuration file support (no longer require `AWS_SDK_LOAD_CONFIG` environment variable) * backend/s3: Automatically expand `~` prefix for home directories in `shared_credentials_file` argument * backend/s3: Add `assume_role_duration_seconds`, `assume_role_policy_arns`, `assume_role_tags`, and `assume_role_transitive_tag_keys` arguments BUG FIXES * backend/s3: Ensure configured profile is used * backend/s3: Ensure configured STS endpoint is used during AssumeRole API calls * backend/s3: Prefer AWS shared configuration over EC2 metadata credentials * backend/s3: Prefer ECS credentials over EC2 metadata credentials * backend/s3: Remove hardcoded AWS Provider messaging ``` Output from acceptance testing: ``` --- PASS: TestBackend (16.32s) --- PASS: TestBackendConfig (0.58s) --- PASS: TestBackendConfig_AssumeRole (0.02s) --- PASS: TestBackendConfig_conflictingEncryptionSchema (0.00s) --- PASS: TestBackendConfig_invalidKey (0.00s) --- PASS: TestBackendConfig_invalidSSECustomerKeyEncoding (0.00s) --- PASS: TestBackendConfig_invalidSSECustomerKeyLength (0.00s) --- PASS: TestBackendExtraPaths (13.21s) --- PASS: TestBackendLocked (28.98s) --- PASS: TestBackendPrefixInWorkspace (5.65s) --- PASS: TestBackendSSECustomerKey (17.60s) --- PASS: TestBackend_impl (0.00s) --- PASS: TestForceUnlock (17.50s) --- PASS: TestKeyEnv (50.25s) --- PASS: TestRemoteClient (4.78s) --- PASS: TestRemoteClientLocks (16.85s) --- PASS: TestRemoteClient_clientMD5 (12.08s) --- PASS: TestRemoteClient_impl (0.00s) --- PASS: TestRemoteClient_stateChecksum (17.92s) ```
Multiple fixes for credential ordering, automatically using the AWS shared configuration file if present, and |
Sorry about the bad post fix etiquette, but thought many may not be able to wait til 0.13. A work around that worked for me with 0.12.26 was removing all Also, this worked with the S3 backend in one AWS account and provider resources in another (so 2 different aws creds profiles). |
* deps: Update github.com/hashicorp/aws-sdk-go-base@v0.5.0 Updated via: ``` $ go get github.com/hashicorp/aws-sdk-go-base@v0.5.0 $ go mod tidy $ go mod vendor ``` * backend/s3: Updates for Terraform v0.13.0 Reference: #13410 Reference: #18774 Reference: #19482 Reference: #20062 Reference: #20599 Reference: #22103 Reference: #22161 Reference: #22601 Reference: #22992 Reference: #24252 Reference: #24253 Reference: #24480 Reference: #25056 Changes: ``` NOTES * backend/s3: Deprecated `lock_table`, `skip_get_ec2_platforms`, `skip_requesting_account_id` arguments have been removed * backend/s3: Credential ordering has changed from static, environment, shared credentials, EC2 metadata, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) to static, environment, shared credentials, default AWS Go SDK (shared configuration, web identity, ECS, EC2 Metadata) * The `AWS_METADATA_TIMEOUT` environment variable no longer has any effect as we now depend on the default AWS Go SDK EC2 Metadata client timeout of one second with two retries ENHANCEMENTS * backend/s3: Always enable shared configuration file support (no longer require `AWS_SDK_LOAD_CONFIG` environment variable) * backend/s3: Automatically expand `~` prefix for home directories in `shared_credentials_file` argument * backend/s3: Add `assume_role_duration_seconds`, `assume_role_policy_arns`, `assume_role_tags`, and `assume_role_transitive_tag_keys` arguments BUG FIXES * backend/s3: Ensure configured profile is used * backend/s3: Ensure configured STS endpoint is used during AssumeRole API calls * backend/s3: Prefer AWS shared configuration over EC2 metadata credentials * backend/s3: Prefer ECS credentials over EC2 metadata credentials * backend/s3: Remove hardcoded AWS Provider messaging ``` Output from acceptance testing: ``` --- PASS: TestBackend (16.32s) --- PASS: TestBackendConfig (0.58s) --- PASS: TestBackendConfig_AssumeRole (0.02s) --- PASS: TestBackendConfig_conflictingEncryptionSchema (0.00s) --- PASS: TestBackendConfig_invalidKey (0.00s) --- PASS: TestBackendConfig_invalidSSECustomerKeyEncoding (0.00s) --- PASS: TestBackendConfig_invalidSSECustomerKeyLength (0.00s) --- PASS: TestBackendExtraPaths (13.21s) --- PASS: TestBackendLocked (28.98s) --- PASS: TestBackendPrefixInWorkspace (5.65s) --- PASS: TestBackendSSECustomerKey (17.60s) --- PASS: TestBackend_impl (0.00s) --- PASS: TestForceUnlock (17.50s) --- PASS: TestKeyEnv (50.25s) --- PASS: TestRemoteClient (4.78s) --- PASS: TestRemoteClientLocks (16.85s) --- PASS: TestRemoteClient_clientMD5 (12.08s) --- PASS: TestRemoteClient_impl (0.00s) --- PASS: TestRemoteClient_stateChecksum (17.92s) ```
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. |
Terraform Version
Terraform Configuration Files
Debug Output
I have created a user, which has no permissions, except the permission to assume the
develop
role, which has full permissionsExample 1
Running
terraform init
.This output is expected, as the user does not have permissions to allow access to the S3 bucket:
Example 2
Running
AWS_PROFILE=develop terraform init
Example 3
Running
AWS_SDK_LOAD_CONFIG=1 AWS_PROFILE=develop terraform init
These examples are also true with
terraform workspace
commandsAdditional Context
Unfortunately,
terraform apply
/terraform plan
can't be ran withAWS_SDK_LOAD_CONFIG
:This makes me think there is a difference in the way that credentials are loaded when using
init
vsplan
/apply
If this can't be reproduced by others, I can provide
TRACE
logs ... There's just too many redactions to go through, if this can be reproduced elsewhere ...The text was updated successfully, but these errors were encountered: