Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

csv file fixtures #9044

Merged
merged 18 commits into from
Nov 9, 2023
Merged

Conversation

emmyoop
Copy link
Member

@emmyoop emmyoop commented Nov 9, 2023

resolves #8290

Problem

No support for csv files in unit testing.

Solution

Support a csv file in addition to an inline csv definition.

  - name: test_my_model_empty
    model: my_model
    given:
      - input: ref('my_model_a')
        format: csv
        fixture: test_my_model_a_empty_fixture_csv
      - input: ref('my_model_b')
        format: csv
        fixture: test_my_model_fixture

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX
  • This PR includes type annotations for new and modified functions

@cla-bot cla-bot bot added the cla:yes label Nov 9, 2023
Copy link

codecov bot commented Nov 9, 2023

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (f629baa) 64.65% compared to head (07267a5) 86.67%.
Report is 9 commits behind head on unit_testing_feature_branch.

Additional details and impacted files
@@                       Coverage Diff                        @@
##           unit_testing_feature_branch    #9044       +/-   ##
================================================================
+ Coverage                        64.65%   86.67%   +22.02%     
================================================================
  Files                              181      181               
  Lines                            26973    27033       +60     
================================================================
+ Hits                             17440    23432     +5992     
+ Misses                            9533     3601     -5932     
Flag Coverage Δ
integration 83.57% <95.12%> (?)
unit 64.59% <34.14%> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
core/dbt/config/project.py 97.47% <100.00%> (+7.18%) ⬆️
core/dbt/parser/unit_tests.py 93.45% <ø> (+40.18%) ⬆️
core/dbt/contracts/graph/unparsed.py 94.20% <94.28%> (+7.43%) ⬆️

... and 120 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@gshank gshank left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@emmyoop emmyoop marked this pull request as ready for review November 9, 2023 19:29
@emmyoop emmyoop requested a review from a team as a code owner November 9, 2023 19:29
@emmyoop emmyoop requested a review from QMalcolm November 9, 2023 19:29
@emmyoop emmyoop merged commit 3b033ac into unit_testing_feature_branch Nov 9, 2023
49 checks passed
@emmyoop emmyoop deleted the er/8290-csv-file-fixture3 branch November 9, 2023 20:35
gshank added a commit that referenced this pull request Jan 16, 2024
* Initial implementation of unit testing (from pr #2911)

Co-authored-by: Michelle Ark <michelle.ark@dbtlabs.com>

* 8295 unit testing artifacts (#8477)

* unit test config: tags & meta (#8565)

* Add additional functional test for unit testing selection, artifacts, etc (#8639)

* Enable inline csv format in unit testing (#8743)

* Support unit testing incremental models (#8891)

* update unit test key: unit -> unit-tests (#8988)


* convert to use unit test name at top level key (#8966)

* csv file fixtures (#9044)

* Unit test support for `state:modified` and `--defer` (#9032)

Co-authored-by: Michelle Ark <michelle.ark@dbtlabs.com>

* Allow use of sources as unit testing inputs (#9059)

* Use daff for diff formatting in unit testing (#8984)

* Fix #8652: Use seed file from disk for unit testing if rows not specified in YAML config (#9064)

Co-authored-by: Michelle Ark <MichelleArk@users.noreply.github.com>
Fix #8652: Use seed value if rows not specified

* Move unit testing to test and build commands (#9108)

* Enable unit testing in non-root packages (#9184)

* convert test to data_test (#9201)

* Make fixtures files full-fledged members of manifest and enable partial parsing (#9225)

* In build command run unit tests before models (#9273)

---------

Co-authored-by: Michelle Ark <michelle.ark@dbtlabs.com>
Co-authored-by: Michelle Ark <MichelleArk@users.noreply.github.com>
Co-authored-by: Emily Rockman <emily.rockman@dbtlabs.com>
Co-authored-by: Jeremy Cohen <jeremy@dbtlabs.com>
Co-authored-by: Kshitij Aranke <kshitij.aranke@dbtlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants