[Regression] dbt is looking for a target with the name given by env var DBT_TARGET in 1.8 - was not in 1.7

[jeremyyeo]

Is this a regression in a recent version of dbt-core?

• I believe this is a regression in dbt-core functionality
• I have searched the existing issues, and I could not find an existing issue for this regression

Current Behavior

If you have a setup like so:

# ~/.dbt/profiles.yml
user:
  target: default
  outputs:
    default:
      type: postgres
      host: localhost
      user: root
      password: password
      port: 5432
      dbname: postgres
      schema: public
      threads: 4

# dbt_project.yml
name: my_dbt_project
profile: user
config-version: 2
version: "1.0.0"

models:
   my_dbt_project:
      +materialized: table

-- models/foo.sql
select 1 id

And coincidentally an env var DBT_TARGET for whatever reason (assume they're using that env var all over the place in their projects - hundreds of macros, whatever - you name it, they have it - so much so that it is cumbersome to change the name of this env var in their project) ... dbt will attempt to look for that target in the profile.

$ export DBT_TARGET=someting
$ echo $DBT_TARGET
something

$ dbt run
03:40:41  Running with dbt=1.8.3
03:40:41  Encountered an error:
Runtime Error
  The profile 'user' does not have a target named 'something'. The valid target names for this profile are:
   - default

Expected/Previous Behavior

With 1.7, dbt didn't do this:

$ export DBT_TARGET=someting
$ echo $DBT_TARGET
something

$ dbt run
03:42:00  Running with dbt=1.7.17
03:42:01  Registered adapter: postgres=1.7.17
03:42:01  Unable to do partial parsing because of a version mismatch
03:42:01  Found 1 model, 0 sources, 0 exposures, 0 metrics, 402 macros, 0 groups, 0 semantic models
03:42:01  
03:42:01  Concurrency: 4 threads (target='default')
03:42:01  
03:42:01  1 of 1 START sql table model public.foo ........................................ [RUN]
03:42:01  1 of 1 OK created sql table model public.foo ................................... [SELECT 2 in 0.07s]
03:42:01  
03:42:01  Finished running 1 table model in 0 hours 0 minutes and 0.17 seconds (0.17s).
03:42:01  
03:42:01  Completed successfully
03:42:01  
03:42:01  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1

Steps To Reproduce

1. With dbt 1.8 - set env var DBT_TARGET to a value of which does not exist in your profile.
2. Run dbt and see error.

Relevant log output

No response

Environment

- OS: macOS
- Python: 3.11
- dbt (working version): 1.7.17
- dbt (regression version): 1.8.3

Which database adapter are you using with dbt?

postgres

Additional Context

This is breaking users keep on latest runs in dbt Cloud:

[dbeatty10]

Although it sounds like it will be cumbersome for them, the only way for their keep on latest version (KOLV) runs to work will be to change the name of this env var in their project.

More detail

dbt-core v1.8.0 (May 09, 2024) added this new feature:

• Global config for --target and --profile CLI flags and DBT_TARGET and DBT_PROFILE environment variables. (#7798)

So if anyone was defining and using the DBT_TARGET or DBT_PROFILE environment variables prior to v1.8, then they would see an error message like what you described.

Essentially, DBT_TARGET and DBT_PROFILE became reserved environment variables in v1.8, whereas they were not reserved prior to that.

Summary

Since we aren't going to change the behavior introduced in v1.8, I'm closing this issue. To resolve this, end users should change the name of their environment variable in the dbt Cloud UI and their code.