Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Microbatch models with only one batch raise unhandled error #11112

Closed
2 tasks done
QMalcolm opened this issue Dec 9, 2024 · 0 comments · Fixed by #11113
Closed
2 tasks done

[Bug] Microbatch models with only one batch raise unhandled error #11112

QMalcolm opened this issue Dec 9, 2024 · 0 comments · Fixed by #11113
Assignees
Labels
bug Something isn't working microbatch Issues related to the microbatch incremental strategy

Comments

@QMalcolm
Copy link
Contributor

QMalcolm commented Dec 9, 2024

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

Currently on 1.9.latest if you run a microbatch model with only one batch (can be achieved by setting lookback=0 for the model), an unhandled exception gets raised (but doesn't actually break anything?)

Expected Behavior

There shouldn't be an unhandled exception for simply having a microbatch model with lookback=0

Steps To Reproduce

  1. Ensure you're running on dbt-core 1.9.latest via pip install -e <path>/<to>/<core>/<repo>/dbt-core/core
    • repository must be checked out on 1.9.latest branch
  2. Create a dbt project with a microbatch incremental model (with lookback=0 in its config)
  3. Run the project via dbt run
  4. Observe the log weirdness

Relevant log output

(venv) quigleymalcolm@Quigley-Malcolm microbatch_demo % dbt run                               
18:10:03  Running with dbt=1.10.0-a1
18:10:03  Registered adapter: postgres=1.9.0-rc1
18:10:03  Found 3 models, 429 macros
18:10:03  
18:10:03  Concurrency: 1 threads (target='dev')
18:10:03  
18:10:03  1 of 3 START sql table model microbatch_demo.input_model ....................... [RUN]
18:10:03  1 of 3 OK created sql table model microbatch_demo.input_model .................. [SELECT 4 in 0.08s]
18:10:03  2 of 3 START sql microbatch model microbatch_demo.microbatch_model ............. [RUN]
18:10:03  Batch 1 of 1 START batch 2024-12-09 of microbatch_demo.microbatch_model .............. [RUN]
18:10:03  Batch 1 of 1 OK created batch 2024-12-09 of microbatch_demo.microbatch_model ......... [MERGE 0 in 0.09s]
18:10:03  Unhandled error while executing 
1
18:10:03  2 of 3 OK created sql microbatch model microbatch_demo.microbatch_model ........ [SUCCESS in 0.12s]
18:10:03  3 of 3 START sql view model microbatch_demo.dependent_model .................... [RUN]
18:10:03  3 of 3 OK created sql view model microbatch_demo.dependent_model ............... [CREATE VIEW in 0.05s]
18:10:03  
18:10:03  Finished running 1 incremental model, 1 table model, 1 view model in 0 hours 0 minutes and 0.39 seconds (0.39s).
18:10:03  
18:10:03  Completed successfully
18:10:03  
18:10:03  Done. PASS=3 WARN=0 ERROR=0 SKIP=0 TOTAL=3

Environment

- OS: macOS 15.1
- Python: 3.10.15
- dbt branch: 1.9.latest (and also main)

Which database adapter are you using with dbt?

postgres

Additional Context

Screenshot 2024-12-09 at 12 43 50
@QMalcolm QMalcolm added bug Something isn't working microbatch Issues related to the microbatch incremental strategy labels Dec 9, 2024
@QMalcolm QMalcolm self-assigned this Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working microbatch Issues related to the microbatch incremental strategy
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant