-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CT-2649] [Feature] Move --target
and --profile
command line flags to global config
#7798
Comments
--target
and --profile
command line flags to global config--target
and --profile
command line flags to global config
Thanks for reaching out @rexledesma ! Can you help me better understand the things you're trying to unlock that aren't possible with using dbt run --target TARGET_NAME --profile PROFILE_NAME Would these things you're thinking of be primarily aimed at users invoking dbt manually or systems invoking dynamic configurations on behalf of a user? |
This is more of just ergonomics managing configuration manually, as well as on behalf of a user. There are cases where I am developing my dbt models and the default profile and target are different from the profile + target that I want to use for local development. Having the ability to customize this via env var would be nice, so I can set this one time and not have to think about it. |
I buy it! The I think the change here might be as simple as:
Going to mark this one a GFI :) |
👍 Yep, looks like it is already included in all commands! See below. Pivot table of dbt commands and supported flags
Source code for pivot table
import re
import pandas as pd
import numpy as np
def find_instances(text):
# Matches any string of format '# dbt subcommand'
subcommand_pattern = r"# dbt .+"
# Matches any string of format '@p.cli_option'
option_pattern = r"@p\.\w+"
# Store results in a list of dictionaries
result_list = []
last_subcommand_match = None
for line in text.splitlines():
subcommand_match = re.search(subcommand_pattern, line)
if subcommand_match:
last_subcommand_match = subcommand_match.group()
option_match = re.search(option_pattern, line)
if option_match and last_subcommand_match:
result_list.append(
{"subcommand": last_subcommand_match, "option": option_match.group()}
)
return pd.DataFrame(result_list)
def read_file(file_path):
with open(file_path, "r") as file:
text = file.read()
return text
def main():
file_path = "main.py"
text = read_file(file_path)
result_df = find_instances(text)
# Represents the presence of a pair with the value 1
result_df["count"] = 1
# Create a pivot table
table = pd.pivot_table(
result_df,
values="count",
index=["option"],
columns=["subcommand"],
aggfunc=np.sum,
fill_value=0,
)
# Convert 1's and 0's to applicable icons
table.replace(to_replace=1, value="✅", inplace=True)
table.replace(to_replace=0, value="", inplace=True)
# Export to Markdown
print(table.to_markdown(stralign="center"))
if __name__ == "__main__":
main()
|
I moved the "user docs" label to the PR so the docs issue gets opened in docs.getdbt.com |
Is this your first time submitting a feature request?
Describe the feature
When attempting to dynamically configure a user's profile and target, We should prefer
dbt --target TARGET_NAME
overdbt run --target TARGET_NAME
, and similarly for--profile
.--profiles-dir
is essentially already a global config with its exposure as an environment variable, so I believe other associated configurations with profiles (e.g.--target
and--profile
) should also be a global config.Describe alternatives you've considered
To achieve dynamic behavior, environment variables (e.g.
DBT_TARGET
andDBT_PROFILE
) can be used to configure--target
and--profile
by templating these values intoprofiles.yml
.However:
env_var(...)
for theirprofile.yml
. Ideally, this feature should come out of the box.Who will this benefit?
No response
Are you interested in contributing this feature?
No response
Anything else?
No response
The text was updated successfully, but these errors were encountered: