-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
v0.19.1 test severity='warn' ignored #3240
Comments
Thanks for the quick report @SteveDMurphy! If possible, could you share:
From looking at the log output you've shared above, it seems like:
So I can't tell if this is an issue with severity, or with test selection.
Could you say a bit more about this, too, or share an example? |
Thanks for the quick response @jtcohen6 !
🤦 I was moving too fast before having to split for a bit, my apologies! I'll repost the logs below:
Also here is the test in question that Warns on 0.19.0 and Errors on 0.19.1:
For the third bullet, that was something random I noticed just based on the snapshot I chose to test with today. It is a data test (if that makes a difference) but thought it might be worth mentioning. In regard to:
I also tested a separate model that I knew to have a failing relationship but was set to warn and mistakenly thought it was a relationship test with the severity set in the Based on my mistake, I went back and specified the severity in the
Thanks again for the help! Let me know if I can add anything else. Pumped to unleash the new power of 0.19.1, huge increase in dev quality of life on those compile times 🙌 |
@SteveDMurphy Thanks for the detailed report back, this is hugely helpful! RegressionThere's a regression from v0.19.0 to v0.19.1 around setting "default" severity within a custom schema test macro. I don't actually think we've documented that this is possible—rather, we document [ We'll investigate this to see if there's an obvious way we can switch back to the previous behavior. In any case, we're planning lots of (good) changes for custom tests in v0.20.0. The ability to configure "default" severity within a generic test wasn't previously on my mind, but it definitely feels possible with the new constructs we're developing, so thanks for raising! In the meantime, as you say, the right approach is to configure tests
- duplicate_snapshot_rows:
severity: warn QuestionI'm still not sure why your first (v0.19.0) invocation includes three tests and your second (v0.19.1) only includes two. The missing test is |
Thanks for the detailed explanation @jtcohen6 ! That all makes sense to me, I think we can go ahead and update For the missing invocation of Thanks again! |
Appreciate your understanding + flexibility! Just to clarify: Is What's the relationship between |
custom schema test, my bad on the terminology! 😅 we have moved solely towards that setup (defined in
|
No worries on the terminology—I know it's confusing and, in fact, would like to change it in v0.20.0 to be a bit more intuitive :) Got it, ok. That sounds like it's a very similar issue to the one over in #3229 (I'm actually surprised you didn't run into that same compilation error), and closely related to parsing config/severity out of the test definition. Really appreciate the clarification, thank you! |
Resolved by #3272 @SteveDMurphy Thanks again for your help getting to the bottom of this one! |
@SteveDMurphy We're including a fix for this in v0.19.2, for which we just cut a release candidate. Any chance you could test locally and confirm that it fixes the issue you described above? pip install dbt==0.19.2rc1
# or
brew install dbt@0.19.2-rc1 |
Describe the bug
A failing test with the severity set to
warn
in the config block is treated as an error. Additionally the same model selection syntax in v0.19.1 is missing a test that is executed in v0.19.0Steps To Reproduce
While validating there would be no issues with bumping from 0.19.0 to 0.19.1 this issue surfaced where a test with severity set to warn will instead return a hard failure.
Expected behavior
Warning in test...
is expected instead ofFailure in test...
Screenshots and log output
v0.19.0
v0.19.1
System information
Which database are you using dbt with?
The output of
dbt --version
:The operating system you're using: OSX / dbt via Docker (3.8-slim)
The output of
python --version
: Python 3.8.7Additional context
Other testing seemed to show that the severity being set to warn in the yml is not respected on out of the box tests as well
The text was updated successfully, but these errors were encountered: