Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Incremental SQL validation #414

Closed
wants to merge 43 commits into from
Closed

Incremental SQL validation #414

wants to merge 43 commits into from

Commits on Oct 14, 2021

  1. Ignore Pyright config file

    @joshtemple
    joshtemple committed Oct 14, 2021
    Configuration menu
    Copy the full SHA
    2dc9ea7 View commit details
    Browse the repository at this point in the history

  2. Update pre commit to match requirements.txt

    @joshtemple
    joshtemple committed Oct 14, 2021
    Configuration menu
    Copy the full SHA
    a263c19 View commit details
    Browse the repository at this point in the history

  3. Update abstract interface for validators

    @joshtemple
    joshtemple committed Oct 14, 2021
    Configuration menu
    Copy the full SHA
    6341789 View commit details
    Browse the repository at this point in the history

  4. Remove project from all_folders args

    @joshtemple
    joshtemple committed Oct 14, 2021
    Configuration menu
    Copy the full SHA
    bb679d1 View commit details
    Browse the repository at this point in the history

  5. Rework content validator to return errors directly 

    Also removes the parent Validator class entirely, which was violating Liskov substitution principle and frankly wasn't very useful as a parent class anyway.
    @joshtemple
    joshtemple committed Oct 14, 2021
    Configuration menu
    Copy the full SHA
    b09e215 View commit details
    Browse the repository at this point in the history

  6. Move public method to the top

    @joshtemple
    joshtemple committed Oct 14, 2021
    Configuration menu
    Copy the full SHA
    f384c0c View commit details
    Browse the repository at this point in the history

  7. Fix mypy config in pre-commit

    @joshtemple
    joshtemple committed Oct 14, 2021
    Configuration menu
    Copy the full SHA
    139d0ed View commit details
    Browse the repository at this point in the history

  8. Fix content validation tests 

    Includes moving build_project to its own method in lookml.py and an update to explore selection that uses a ~/repos/spectacles to represent excluded explores in a single list instead of using two lists (selectors/exclusions) to represent.
    @joshtemple
    joshtemple committed Oct 14, 2021
    Configuration menu
    Copy the full SHA
    eb4a3a1 View commit details
    Browse the repository at this point in the history

  9. Rework data test validator to return errors directly

    @joshtemple
    joshtemple committed Oct 14, 2021
    Configuration menu
    Copy the full SHA
    cad6bf8 View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2021

  1. Fix cassette names

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    eebdd37 View commit details
    Browse the repository at this point in the history

  2. Use validator's client instead of fixture

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    5cfb999 View commit details
    Browse the repository at this point in the history

  3. Update data test validator tests

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    1467e3f View commit details
    Browse the repository at this point in the history

  4. Simplify skipping in loop

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    1c74eba View commit details
    Browse the repository at this point in the history

  5. Add method to iterate through explores in project

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    462df2d View commit details
    Browse the repository at this point in the history

  6. Allow explores to be in a skipped state

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    96be6aa View commit details
    Browse the repository at this point in the history

  7. Update runner method for data tests

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    1cdaa66 View commit details
    Browse the repository at this point in the history

  8. Update runner method for content validator

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    98f1365 View commit details
    Browse the repository at this point in the history

  9. Remove QueryMode type

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    39def00 View commit details
    Browse the repository at this point in the history

  10. Apply errors to the project as well as returning

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    ee88b82 View commit details
    Browse the repository at this point in the history

  11. Use fail fast instead of query mode

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    e26d8c3 View commit details
    Browse the repository at this point in the history

  12. Update branch manager to handle ephemeral correctly

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    c986e5e View commit details
    Browse the repository at this point in the history

  13. Add run query method to client

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    7aa9f1f View commit details
    Browse the repository at this point in the history

  14. Add incremental SQL implementation

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    d747629 View commit details
    Browse the repository at this point in the history

  15. Make test creation its own method

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    eff1903 View commit details
    Browse the repository at this point in the history

  16. Restore deleted line for query result handling

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    5732737 View commit details
    Browse the repository at this point in the history

  17. Use safer default args

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    60bddd0 View commit details
    Browse the repository at this point in the history

  18. Fix bug in repr

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    1afb25a View commit details
    Browse the repository at this point in the history

  19. Update tests for SQL validation 

    I chose to remove the tests that included warnings. Fail and Pass should be sufficient.
    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    0f81666 View commit details
    Browse the repository at this point in the history

  20. Update select tests to new filters logic

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    00b91e8 View commit details
    Browse the repository at this point in the history

  21. Don't include models without selected explores

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    9372f0f View commit details
    Browse the repository at this point in the history

  22. Update client tests for and

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    7cffc4d View commit details
    Browse the repository at this point in the history

  23. Move build project tests to test_lookml

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    64b30d0 View commit details
    Browse the repository at this point in the history

  24. Create client outside of Runner constructor

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    8021c28 View commit details
    Browse the repository at this point in the history

  25. Add defaults for ref and filters for data/content validators

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    3eba9db View commit details
    Browse the repository at this point in the history

  26. Revert printing of SQL validation errors to old approach

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    c3da7f8 View commit details
    Browse the repository at this point in the history

  27. Add some basic tests for naive runner usage

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    eb6c5b1 View commit details
    Browse the repository at this point in the history

  28. Test JSON schema is correct

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    95741d5 View commit details
    Browse the repository at this point in the history

  29. Parameterize test on fail fast

    @joshtemple
    joshtemple committed Oct 15, 2021
    Configuration menu
    Copy the full SHA
    b66f731 View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2021

  1. Fix bug in validate_sql 

    Didn't set dimension tests correctly in non-incremental case.
    @joshtemple
    joshtemple committed Oct 19, 2021
    Configuration menu
    Copy the full SHA
    d44f327 View commit details
    Browse the repository at this point in the history

  2. Fix bug in SqlTest __eq__ 

    Because .sql can sometimes be None (non incremental-case) this was causing all SqlTests to be equal, causing some really weird errors in how we removed tests from the  attribute.
    @joshtemple
    joshtemple committed Oct 19, 2021
    Configuration menu
    Copy the full SHA
    85f3845 View commit details
    Browse the repository at this point in the history

  3. Remove test that is no longer valid

    @joshtemple
    joshtemple committed Oct 19, 2021
    Configuration menu
    Copy the full SHA
    0673ef4 View commit details
    Browse the repository at this point in the history

  4. Use loops instead of set operations 

    Set operations are problematic here because we don't have control over which object is returned (from set A or set B). This is important because we only want to use the objects from the base project, not the target project.
    @joshtemple
    joshtemple committed Oct 19, 2021
    Configuration menu
    Copy the full SHA
    1ed54fe View commit details
    Browse the repository at this point in the history

  5. Change logic on creating dimension tests

    @joshtemple
    joshtemple committed Oct 19, 2021
    Configuration menu
    Copy the full SHA
    502b2fa View commit details
    Browse the repository at this point in the history