Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests are configurable from dbt_project.yml #3253

Closed
jtcohen6 opened this issue Apr 12, 2021 · 0 comments · Fixed by #3257
Closed

Tests are configurable from dbt_project.yml #3253

jtcohen6 opened this issue Apr 12, 2021 · 0 comments · Fixed by #3257
Labels
1.0.0 Issues related to the 1.0.0 release of dbt dbt tests Issues related to built-in dbt testing functionality enhancement New feature or request
Milestone

Comments

@jtcohen6
Copy link
Contributor

jtcohen6 commented Apr 12, 2021

Describe the feature

This issue brings tests up to parity with other resource types in terms of being able to configure them from dbt_project.yml. In particular, this is essential for being able to control the behavior of tests installed from packages.

dbt_project.yml

# top-level: both bespoke + generic tests
tests:
  +enabled: true
  +severity: error
  +tags: ["tag_one", "tag_two"]
  
# for specific instances of generic tests defined on models
models:
  my_project:
    my_subfolder:
      +tests:
        enabled: true
        severity: warn

# for specific instances of generic tests defined on seeds
seeds:
  installed_package:
    +tests:
      enabled: false

# for specific instances of generic tests defined on snapshots
snapshots:
  +tests:
    tags: ["tag_three"]

# for specific instances of generic tests defined on sources
sources:
  +tests:
    severity: warn

What does it mean to configure...

  1. A bespoke test (e.g. test/my_test.sql)? Same it does today when I put {{ config(severity = 'warn') }} at the top of test/my_test.sql.
  2. A generic test (e.g. unique)? I should be able to add either {{ config(severity = 'warn') }} to my test block definition, or severity=warn as a default argument in the test signature. All specific instances of the unique test will now be warn-severity by default, unless the specific test overrides as below. (Note: There was a regression for this in v0.19.1, see v0.19.1 test severity='warn' ignored #3240).
  3. A specific instance of a generic test (e.g. unique_model_a_id)? Same as it does today when I add an "argument" to the test definition:
        tests:
          - unique:
               severity: warn

In this way, for generic tests, configs are like a special type of test block (macro) argument. You could think about this issue as being a way to "pass" arguments to a bunch of instantiated generic tests all at once, right from dbt_project.yml. That said, there are a few differences between configs and arguments for generic tests:

Hierarchy

  1. Most specific always wins: The config() block in a data test, or the value passed to a specific instance of a generic test where it's defined in (e.g.) a models/*.yml file
  2. The default values set within the generic test block (either by config() block or by default arguments in the signature)
  3. Config in dbt_project.yml, applying to all tests
  4. The configs of installed packages (1-3)

Edited: I previously had 2 last, but this is going to be difficult given the way this is implemented today. I don't mind the idea that the generic test's default values are still more specific than dbt_project.yml config. I'm also not sure how to think about #3256 relative to the ordering above.

Additional context

Today, the only supported configs for tests are enabled, severity, and tags (docs). Generic tests support severity + tags, and they should enabled (#3252). We hope to add net-new configs very soon as well (e.g. where, limit).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1.0.0 Issues related to the 1.0.0 release of dbt dbt tests Issues related to built-in dbt testing functionality enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant