Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-3176] [spike] infer fixture data types for unit testing #8757

Closed
1 task done
Tracked by #8283
graciegoheen opened this issue Oct 2, 2023 · 4 comments
Closed
1 task done
Tracked by #8283

[CT-3176] [spike] infer fixture data types for unit testing #8757

graciegoheen opened this issue Oct 2, 2023 · 4 comments
Assignees
Labels
enhancement New feature or request user docs [docs.getdbt.com] Needs better documentation

Comments

@graciegoheen
Copy link
Contributor

graciegoheen commented Oct 2, 2023

Housekeeping

  • I am a maintainer of dbt-core

Short description

From #8649

Instead of requiring folks to run dbt run --select +my_model --exclude my_model before executing unit tests on my_model or requiring the use of --deferral (which would cause issues in a CI context when you're deferring to a production environment and may have changes to your inputs as part of the PR)...

we should be able to run dbt unit-test --select my_model in a new environment without having previously run dbt run --select +my_model --exclude my_model or dbt docs generate

  • we will use the data_types defined in the direct parent's yml config
  • if no data_type is provided...
    • we will have the warehouse "presume" the data type based on the mock data provided (users could put cast statements in their fixture)
    • if no mock data is provided for a column, we will guess that the data type is a string
      raise a warning during unit test execution if an input fixture specifies columns that don’t have user-provided data_type values.

We will need to confirm when we should add '' around the mocked data.

We can get all column names from adapter.get_column_schema_from_query so users don't have to provide a yml name for all columns of direct parents.

Acceptance criteria

Impact to Other Teams

No

Will backports be required?

No

Context

No response

@graciegoheen graciegoheen added the user docs [docs.getdbt.com] Needs better documentation label Oct 2, 2023
@github-actions github-actions bot changed the title [implementation] infer fixture data types for unit testing [CT-3176] [implementation] infer fixture data types for unit testing Oct 2, 2023
@graciegoheen
Copy link
Contributor Author

Let's make sure we get feedback on this during the beta!

@graciegoheen graciegoheen added the enhancement New feature or request label Oct 5, 2023
@graciegoheen graciegoheen changed the title [CT-3176] [implementation] infer fixture data types for unit testing [CT-3176] [spike] infer fixture data types for unit testing Oct 25, 2023
@graciegoheen
Copy link
Contributor Author

The goal of this spike is to determine an appropriate path forward - additional ideas tracked here #8649

@graciegoheen
Copy link
Contributor Author

Opened - #8979 and #8980

@graciegoheen
Copy link
Contributor Author

graciegoheen commented Nov 6, 2023

Our assumption:

We can get all column names from adapter.get_column_schema_from_query so users don't have to provide a yml name for all columns of direct parents.

was incorrect. So closing in favor of:

  • use --empty to build "empty" versions of models
  • dbt build includes unit tests to materialize upstream nodes before running unit tests

@graciegoheen graciegoheen closed this as not planned Won't fix, can't repro, duplicate, stale Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request user docs [docs.getdbt.com] Needs better documentation
Projects
None yet
Development

No branches or pull requests

2 participants