Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor environment variable handling into new Sources::EnvSource #299

Merged
merged 5 commits into from
Feb 25, 2021

Conversation

cjlarose
Copy link
Member

@cjlarose cjlarose commented Jan 13, 2021

In principle, loading settings from environment variables isn't much different from loading them from a YAMLSource or a HashSource. This change unifies the handling of environment variables into the existing "sources" abstraction.

My own personal motivation for this change is essentially the same as #271. I want to load settings into config that I got from AWS Secrets Manager. Although AWS Secrets Manager's API technically allows you to store arbitrary string values, the AWS Console UI and the AWS Secrets Manager API encourages you to store your application's secret configuration as as single Secret resource that is a flat, JSON-encoded key-value mapping. This maps very nicely to how one might use environment variables to configure an application.

Editing secrets in the AWS console

Screen Shot 2021-01-12 at 20 04 41

Actual plaintext

Screen Shot 2021-01-12 at 20 10 37

The problem is that while config offers the ability to parse nested settings from ENV, it doesn't allow you to do the same for settings you got from another source, like AWS Secrets Manager. As a result, users are forced into an awkward workaround: export all secrets from AWS Secrets Manager into ENV, then ask config to load the ENV again. My proposed EnvSource allows users to skip the loading-into-ENV step and just hand the flat Hash over to config directly.

For example,

# fetch secrets from AWS
client = Aws::SecretsManager::Client.new
response = client.get_secret_value(secret_id: "#{ENV['ENVIRONMENT']}/my_application")
secrets = JSON.parse(response.secret_string)

# load secrets into config
secret_source = Config::Sources::EnvSource.new(secrets)
Settings.add_source!(secret_source)
Settings.reload!

merge!(hash)
end

alias :load_env! :reload_env!
Copy link
Member Author

@cjlarose cjlarose Jan 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pkuczynski Config::Options#load_env! and Config::Options#reload_env! are not documented at all, but are technically public methods.

As such, depending on whether or not we consider all visible methods to be part of the public API of this gem, this could be considered a breaking change and warrants a major version bump, so I added this under Breaking Changes in the changelog.

The change for users that use #load_env! or #reload_env! is simple: they should just replace those usages with #reload!, which is documented.

@cjlarose cjlarose force-pushed the env-source branch 4 times, most recently from 814a26f to 1750953 Compare January 13, 2021 19:48
By default, EnvSource will use "global" settings specified like
`Config.env_prefix`, `Config.env_separator`, `Config.env_separator`, and
`Config.env_parse_values`. Those configurations will be used when
parsing the ENV hash.

But when using EnvSource to load settings from some unrelated flat
string-keyed Hash source, we want to allow folks to override the settings.
@cjlarose
Copy link
Member Author

@pkuczynski let me know if you have any concerns

@cjlarose cjlarose merged commit 351c819 into rubyconfig:master Feb 25, 2021
@cjlarose cjlarose deleted the env-source branch February 25, 2021 04:58
@cjlarose
Copy link
Member Author

Ended up speaking to @pkuczynski about this in another channel. He's cool with merging and releasing.

This change was released in 3.0.0.

@cjlarose cjlarose mentioned this pull request Feb 25, 2021
ippachi pushed a commit to ippachi/config-1 that referenced this pull request Oct 3, 2021
…ubyconfig#299)

* Extract env var handling to new EnvSource class

* Allow overriding settings for parsing env sources

By default, EnvSource will use "global" settings specified like
`Config.env_prefix`, `Config.env_separator`, `Config.env_separator`, and
`Config.env_parse_values`. Those configurations will be used when
parsing the ENV hash.

But when using EnvSource to load settings from some unrelated flat
string-keyed Hash source, we want to allow folks to override the settings.

* Update CHANGELOG

* Add AWS Secrets Manager usage to README
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

1 participant