Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model intersection syntax #2167

Closed
clrcrl opened this issue Feb 27, 2020 · 5 comments · Fixed by #2417
Closed

Model intersection syntax #2167

clrcrl opened this issue Feb 27, 2020 · 5 comments · Fixed by #2417
Labels
dbt-docs [dbt feature] documentation site, powered by metadata artifacts enhancement New feature or request

Comments

@clrcrl
Copy link
Contributor

clrcrl commented Feb 27, 2020

Describe the feature

Currently, when you provide two args to the model selection syntax, dbt selects the union of the two parts of the graph:
e.g.

dbt run --models tag:hourly tag:marketing

would run all models that have either the hourly or marketing tag (or both)

I'd like a way to only run the intersection of these two parts of the graph:

dbt run --models [tag:hourly ∩ tag:marketing ]

That would only run models that have both the hourlyAND marketing tag. And of course is a terrible symbol to use here.

Describe alternatives you've considered

We've seen this come up a few times, normally when people want to run the models in a folder that have a certain tag. Our current advice is to add new tags.

@clrcrl clrcrl added enhancement New feature or request triage labels Feb 27, 2020
@drewbanin drewbanin removed the triage label Feb 28, 2020
@drewbanin
Copy link
Contributor

Let's prioritize this for the Octavius Catto release. We should use a , to separate selectors, eg:

dbt run --models tag:abc,tag:def tag:ghi,jkl

This will select models that have:

  • tags abc AND def, OR
  • tags ghi AND jkl

@drewbanin drewbanin added this to the Octavius Catto milestone Feb 28, 2020
@clrcrl
Copy link
Contributor Author

clrcrl commented Feb 28, 2020

do you mean:

dbt run --models tag:abc,tag:def tag:ghi,tag:jkl

FWIW, I actually kinda like the parentheses approach:

dbt run --models [tag:abc tag:def] [tag:ghi tag:jkl]

It's easier for me to see what's nested together. But I don't know what the norm is here/whether parens are a bad idea for a CLI arg!

@drewbanin drewbanin added the dbt-docs [dbt feature] documentation site, powered by metadata artifacts label Mar 16, 2020
@praktiskt
Copy link

praktiskt commented Apr 27, 2020

Anyone who's in a rush solving this (myself included) can use a bit of shell and jq to solve this when invoking dbt run.

dbt run -m $(dbt ls --resource-type model --output json | jq '. | select(.tags | index("abc")) | select(.tags | index("def")) .name' -r | tr '\n' ' ' && echo '""')

Edit: Added && echo '""' to ensure the statement returns at least one model ("").

@drewbanin drewbanin modified the milestones: Octavius Catto, dbt-next Apr 29, 2020
@drewbanin
Copy link
Contributor

@Raalsky I think you indicated that you were interested in picking this up -- is that still the case? If not, we're ready to prioritize this on our end :)

@Raalsky
Copy link
Contributor

Raalsky commented Apr 29, 2020

@drewbanin Sorry for being off. Yeah, I'm still interested. I'll post updates after the weekend, ok?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dbt-docs [dbt feature] documentation site, powered by metadata artifacts enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants