Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iterate on .env file loading #10093

Merged
merged 4 commits into from
Mar 2, 2024
Merged

Iterate on .env file loading #10093

merged 4 commits into from
Mar 2, 2024

Conversation

jtoar
Copy link
Contributor

@jtoar jtoar commented Mar 2, 2024

Follow up to #9961. Got some feedback from @cannikin and @Josh-Walker-GM. There are still some things I need to clarify, but the changes here are improvements, and we'll keep iterating before this goes out next week (currently in the v7.1 RC):

  • the name was shortened from --include-env-files to --add-env-files
  • .env files that are loaded based on NODE_ENV override instead of add
  • the logic was moved up from middleware because env loading should be done as soon as possible, and the earliest is right after we set the cwd

I'll bring up the override vs at the next meeting.

Just to document what currently happens to completeness, env files are loaded in two steps...

Layer 1

  • .env.defaults
  • .env
  • .env.${NODE_ENV}

These are loaded in an overriding manner, so that if there are conflicts, .env.${NODE_ENV} has the final say. This can't be disabled.

Layer 2

If users want, they can add more via --add-env-files, like --add-env-files stripe. The env vars loaded from files specified this way are additive—they won't override anything already set in the first layer.

I'll see about moving this functionality out of the CLI package and into CLI helpers so its shareable in another PR.

@jtoar jtoar added release:fix This PR is a fix release:feature This PR introduces a new feature labels Mar 2, 2024
@jtoar jtoar added this to the next-release milestone Mar 2, 2024
@jtoar jtoar removed the release:feature This PR introduces a new feature label Mar 2, 2024
@jtoar jtoar merged commit 1236820 into main Mar 2, 2024
40 of 41 checks passed
@jtoar jtoar deleted the ds-cli/iterate-on-add-env-files branch March 2, 2024 05:19
jtoar added a commit that referenced this pull request Mar 2, 2024
Follow up to #9961. Got some
feedback from @cannikin and @Josh-Walker-GM. There are still some things
I need to clarify, but the changes here are improvements, and we'll keep
iterating before this goes out next week (currently in the v7.1 RC):

- the name was shortened from `--include-env-files` to `--add-env-files`
- `.env` files that are loaded based on `NODE_ENV` override instead of
add
- the logic was moved up from middleware because env loading should be
done as soon as possible, and the earliest is right after we set the cwd

I'll bring up the override vs at the next meeting.

Just to document what currently happens to completeness, env files are
loaded in two steps...


#### Layer 1

- `.env.defaults`
- `.env`
- `.env.${NODE_ENV}`

These are loaded in an overriding manner, so that if there are
conflicts, `.env.${NODE_ENV}` has the final say. This can't be disabled.

#### Layer 2

If users want, they can add more via `--add-env-file`, like
`--add-env-file stripe`. The env vars loaded from files specified this
way are additive—they won't override anything already set in the first
layer.

I'll see about moving this functionality out of the CLI package and into
CLI helpers so its shareable in another PR.
@Tobbe
Copy link
Member

Tobbe commented Mar 2, 2024

Is it --add-env-file or --add-env-files? The PR description har both versions

EDIT: At my computer now and checked the actual code. It's with an "s" at the end, --add-env-files
I updated the PR description

jtoar added a commit that referenced this pull request Mar 2, 2024
Follow up to #10093. After
discussing with @orta, the `--add-env-files` flag being additive instead
of overriding wasn't a sticking point in the original implementation,
and the more feedback I get, it seems like most people expect later
files to override earlier ones, so let's switch the behavior.

In tandem, at first I tried to rename the flag back to just `--env-file`
or `--env-files`, but I'm afraid both give this cryptic error:

```
redwood-app % yarn rw --env-file prod
node: prod: not found

# Or `yarn rw --env-files prod`, which also gives this error,
# which I didn't know node would also process? Their docs only say `--env-file`...
```

The above was with my framework changes. I tried it again without my
framework changes and the error persists. As far as I can tell, `rw`
never executes and it seems like the Node.js binary itself is evaluating
the `--env-file` flag. It supports `--env-file` now, and expects it to
be a full path (so `.env.prod`). But this isn't how I thought node
processes `process.argv` at all... I thought that all flags would've
been passed to the script (here, `rw`) for processing. But maybe not,
and if so that means we can just pass options to node via `yarn rw
--my-node-option`. Which I still don't quite believe and if it is true,
then I'm not sure what to make of yet cause people have been wanting to
pass node options for a while and I always thought
`NODE_OPTIONS="--my-node-option" yarn rw ...` was the only way.

So TL;DR, that's why I've left the name here and I'd like to keep that
friction point out of the scope of this PR being considered mergeable.
jtoar added a commit that referenced this pull request Mar 2, 2024
Follow up to #10093. After
discussing with @orta, the `--add-env-files` flag being additive instead
of overriding wasn't a sticking point in the original implementation,
and the more feedback I get, it seems like most people expect later
files to override earlier ones, so let's switch the behavior.

In tandem, at first I tried to rename the flag back to just `--env-file`
or `--env-files`, but I'm afraid both give this cryptic error:

```
redwood-app % yarn rw --env-file prod
node: prod: not found

# Or `yarn rw --env-files prod`, which also gives this error,
# which I didn't know node would also process? Their docs only say `--env-file`...
```

The above was with my framework changes. I tried it again without my
framework changes and the error persists. As far as I can tell, `rw`
never executes and it seems like the Node.js binary itself is evaluating
the `--env-file` flag. It supports `--env-file` now, and expects it to
be a full path (so `.env.prod`). But this isn't how I thought node
processes `process.argv` at all... I thought that all flags would've
been passed to the script (here, `rw`) for processing. But maybe not,
and if so that means we can just pass options to node via `yarn rw
--my-node-option`. Which I still don't quite believe and if it is true,
then I'm not sure what to make of yet cause people have been wanting to
pass node options for a while and I always thought
`NODE_OPTIONS="--my-node-option" yarn rw ...` was the only way.

So TL;DR, that's why I've left the name here and I'd like to keep that
friction point out of the scope of this PR being considered mergeable.
@jtoar jtoar modified the milestones: next-release, v7.1.0 Mar 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release:fix This PR is a fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants