AWS_CONFIG_FILE vs. AWS_CREDENTIAL_FILE

[sopel]

Configuration and Credential Files within the official AWS CLI documentation differentiates between these two configuration files and resp. environment variable overrides AWS_CONFIG_FILE and AWS_CREDENTIAL_FILE:

• AWS_CONFIG_FILE – path to a CLI config file.
• AWS_CREDENTIAL_FILE – path to an AWS credentials file.

It elaborates on that in section Storing Credentials in Config:

The default file locations can be overridden by setting the AWS_CONFIG_FILE and AWS_CREDENTIAL_FILE environment variables to another local path. See Environment Variables for details.

The AWS CLI will also read credentials from the config file. If you want to keep all of your profile settings in a single file, you can. If there are ever credentials in both locations for a profile (say you used aws configure to update the profile's keys), the keys in the credentials file will take precendence.

However, other than AWS_CONFIG_FILE, AWS_CREDENTIAL_FILE doesn't seem to work and exhibits the following two oddities:

1. using aws configure with AWS_CREDENTIAL_FILE set still yields the credentials file to be generated in the default location (whereas AWS_CONFIG_FILE is obeyed)
2. moving the credentials file generated in 1. to the location denoted in AWS_CREDENTIAL_FILE yields 'Unable to locate credentials. You can configure credentials by running "aws configure".' again - however, after changing that location to a non existing file, boto triggers the error 'The specified config file () could not be found.' as expected, so it attempts to read the file at least

❓ Any idea what's going on here, or am I missing how this is supposed to work? Thanks!

[jamesls]

The docs are wrong, we do not support an AWS_CREDENTIAL_FILE env var. Well to be specific, we do support AWS_CREDENTIAL_FILE, but it's an older, deprecated format that is not the same thing as the shared credentials file in ~/.aws/credentials. I will get these docs updated. It looks like the page that need to be changed is http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html

[jamesls]

Docs have been updated, thanks again for letting us know.

[sopel]

Thanks @jamesls, that's an improvement - I'm afraid the docs are still a bit confusing/misleading though, insofar they suggest what I want is possible:

AWS_CONFIG_FILE – path to a CLI config file.

If the config file variable is set, aws configure will write to the specified file, and the CLI will attempt to read profiles from there instead of the default path (in a folder named .aws in your user directory).

To me, the phrase aws configure will write to the specified file naturally implies all information I submit (would also match the principle of least surprise). However, that's not the case, and Configuration and Credential Files seems to explain why:

In order to separate credentials from less sensitive options, region and output format are stored in a separate file named config in the same folder.

Assuming this is the expected behavior indeed, could you please clarify:

• Does this imply that I cannot control where the credentials are written to by aws configure?
• And if so, is this a deliberate security measure or just a missing feature at this point?
  • I realize that I can manually work around this because The AWS CLI will also read credentials from the config file, however, this defeats the usability goals of aws configure, insofar I'd like to hand over a solution to users, who are only supposed to, well, configure their credentials ;) (use case are disposable Docker containers, which is why the credentials need to be routed to persisted storage outside of the container somehow)

[jamesls]

Ah, thanks for pointing that out. I'll let our docs team know about that and get that wording updated.

The aws configure command will always write out to the shared credentials file.

A while back when we were integrating with this shared credentials file ~/.aws/credentials, I proposed a few options (#847), one which included letting the user specify if they wanted shared credentials or the AWS CLI config file (this was option number 2). The feedback we received was to use option 3, which was to just always used the shared credentials file.

That linked issue has more info if you're interested.

In the meantime, I'll get the docs updated. Thanks again.

[sandys]

not supporting AWS_CREDENTIAL_FILE is a problem when we use sudo to run aws-cli, especially with users who dont have home directories (e.g. www-data).

What is the right way to run aws-cli with "sudo" ?

[nchammas]

I agree with @sandys. It seems like a problem that you cannot point aws to a config file in a non-default location. This is handy when you need to something like sudo aws s3 sync /local/path s3://remote/path. The sudo may be required, for example, so that you can access local files. Having to set environment variables is a bit clunky given that the info you need is already available in a set of files.

[schollii]

See AWS_SHARED_CREDENTIALS_FILE

[rdonkin]

You need to set AWS_CONFIG_FILE and AWS_SHARED_CREDENTIALS_FILE environment variables, to the config file and credentials file respectively. Works fine with AWS CLI latest version.

Ref https://docs.aws.amazon.com/cli/latest/topic/config-vars.html#the-shared-credentials-file

[minlare]

I found I had to set another variable in .env to get this working

AWS_SHARED_CREDENTIALS_FILE=local/folder/credentials
AWS_CONFIG_FILE=local/folder/config
AWS_SDK_LOAD_CONFIG=1