Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DO NOT MERGE / TEST: only for diff #800

Closed
wants to merge 54 commits into from
Closed

DO NOT MERGE / TEST: only for diff #800

wants to merge 54 commits into from

Conversation

dataders
Copy link
Contributor

just doing to review changes to write the upgrade guide

emmyoop and others added 30 commits July 17, 2023 17:19
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
* Clean up changelog on main

* Bumping version to 1.7.0a1

* pre-commit

* changie merge

* add 1.6 to 0.0.md

* re trigger tests

---------

Co-authored-by: Matthew McKnight <matthew.mcknight@dbtlabs.com>
Co-authored-by: Matthew McKnight <91097623+McKnight-42@users.noreply.github.com>
* adding option for pem formatted private key

* adding changie

---------

Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com>
* update stale workflow to use the centralized action

* changie
* add alter scenario for dynamic tables
* remove pytest skip statements on change tests
* update target lag tests
* fix target_lag in create dynamic table statement
* fixed name changes during model production
* clean up dynamic table changeset
* update target lag input options

* changie
* remove unnecessary asserts
* skip F401 on raising objects in __init__.py
* update RELEASE_BRANCH env

* add teardown_method to clean up hanging artfiact after test runs
* Update wheel requirement from ~=0.40 to ~=0.41

Updates the requirements on [wheel](https://github.com/pypa/wheel) to permit the latest version.
- [Changelog](https://github.com/pypa/wheel/blob/main/docs/news.rst)
- [Commits](pypa/wheel@0.40.0...0.41.0)

---
updated-dependencies:
- dependency-name: wheel
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
* Update pip-tools requirement from ~=6.14 to ~=7.0

Updates the requirements on [pip-tools](https://github.com/jazzband/pip-tools) to permit the latest version.
- [Release notes](https://github.com/jazzband/pip-tools/releases)
- [Changelog](https://github.com/jazzband/pip-tools/blob/main/CHANGELOG.md)
- [Commits](jazzband/pip-tools@6.14.0...7.0.0)

---
updated-dependencies:
- dependency-name: pip-tools
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
* Update flake8 requirement from ~=6.0 to ~=6.1

Updates the requirements on [flake8](https://github.com/pycqa/flake8) to permit the latest version.
- [Commits](PyCQA/flake8@6.0.0...6.1.0)

---
updated-dependencies:
- dependency-name: flake8
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

* removed extra newline from changelog and workflow that generates changelog

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
Co-authored-by: Mike Alfare <mike.alfare@dbtlabs.com>
* Update pip-tools requirement from ~=7.0 to ~=7.1

Updates the requirements on [pip-tools](https://github.com/jazzband/pip-tools) to permit the latest version.
- [Release notes](https://github.com/jazzband/pip-tools/releases)
- [Changelog](https://github.com/jazzband/pip-tools/blob/main/CHANGELOG.md)
- [Commits](jazzband/pip-tools@7.0.0...7.1.0)

---
updated-dependencies:
- dependency-name: pip-tools
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

* removed extra newline from changelog

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
Co-authored-by: Mike Alfare <mike.alfare@dbtlabs.com>
* Update black requirement from ~=23.3 to ~=23.7

Updates the requirements on [black](https://github.com/psf/black) to permit the latest version.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](psf/black@23.3.0...23.7.0)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
* Update ddtrace requirement from ~=1.16 to ~=1.17

Updates the requirements on [ddtrace](https://github.com/DataDog/dd-trace-py) to permit the latest version.
- [Release notes](https://github.com/DataDog/dd-trace-py/releases)
- [Changelog](https://github.com/DataDog/dd-trace-py/blob/1.x/CHANGELOG.md)
- [Commits](DataDog/dd-trace-py@v1.16.0...v1.17.0)

---
updated-dependencies:
- dependency-name: ddtrace
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
* Update pip-tools requirement from ~=7.1 to ~=7.2

Updates the requirements on [pip-tools](https://github.com/jazzband/pip-tools) to permit the latest version.
- [Release notes](https://github.com/jazzband/pip-tools/releases)
- [Changelog](https://github.com/jazzband/pip-tools/blob/main/CHANGELOG.md)
- [Commits](jazzband/pip-tools@7.1.0...7.2.0)

---
updated-dependencies:
- dependency-name: pip-tools
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
…oes not wrap dml with explicit transaction (#673)

* [ADAP-620] [Bug] Incremental materialization with "append" strategy does not wrap dml with explicit transaction
Fixes #656

* changie

---------

Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
…716)

* Update connections.py so it doesn’t run anything after its canceled
* Update expected cancellation query
* Changelog entry

---------

Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
Co-authored-by: Doug Beatty <doug.beatty@dbtlabs.com>
* update RELEASE_BRANCH env

* start work on adap-774 to migrate dynmic tables to use snowflake_warehouse vs warehouse

* revert some stuff back to wareshouse as it is what snwoflake expects, and add a alias on describe macro so we are taking in the warehouse field as snowflake_warehouse

* change location of comparion to snowflake_warehouse and warehouse

* remove uneeded addition

* revert a field
* Update pip-tools requirement from ~=7.2 to ~=7.3

Updates the requirements on [pip-tools](https://github.com/jazzband/pip-tools) to permit the latest version.
- [Release notes](https://github.com/jazzband/pip-tools/releases)
- [Changelog](https://github.com/jazzband/pip-tools/blob/main/CHANGELOG.md)
- [Commits](jazzband/pip-tools@7.2.0...7.3.0)

---
updated-dependencies:
- dependency-name: pip-tools
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
* Update tox requirement from ~=4.6 to ~=4.7

Updates the requirements on [tox](https://github.com/tox-dev/tox) to permit the latest version.
- [Release notes](https://github.com/tox-dev/tox/releases)
- [Changelog](https://github.com/tox-dev/tox/blob/main/docs/changelog.rst)
- [Commits](tox-dev/tox@4.6.0...4.7.0)

---
updated-dependencies:
- dependency-name: tox
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
* Bump mypy from 1.4.1 to 1.5.0

Bumps [mypy](https://github.com/python/mypy) from 1.4.1 to 1.5.0.
- [Commits](python/mypy@v1.4.1...v1.5.0)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
)

* update RELEASE_BRANCH env

* initial push for ADAP-631 to convert target_lag to a str type

* readd SnowflakeDynamicTableTargetLagConfigChange class as part of dynamic_table.py

* pull in ADAP-774 and add changelog

* add missing changelog entry for pr 727

* update to main, and add basic test case for passing downstream via alter
* Update tox requirement from ~=4.7 to ~=4.8

Updates the requirements on [tox](https://github.com/tox-dev/tox) to permit the latest version.
- [Release notes](https://github.com/tox-dev/tox/releases)
- [Changelog](https://github.com/tox-dev/tox/blob/main/docs/changelog.rst)
- [Commits](tox-dev/tox@4.7.0...4.8.0)

---
updated-dependencies:
- dependency-name: tox
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
* Update ddtrace requirement from ~=1.17 to ~=1.18

Updates the requirements on [ddtrace](https://github.com/DataDog/dd-trace-py) to permit the latest version.
- [Release notes](https://github.com/DataDog/dd-trace-py/releases)
- [Changelog](https://github.com/DataDog/dd-trace-py/blob/1.x/CHANGELOG.md)
- [Commits](DataDog/dd-trace-py@v1.17.0...v1.18.0)

---
updated-dependencies:
- dependency-name: ddtrace
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
* Update tox requirement from ~=4.8 to ~=4.9

Updates the requirements on [tox](https://github.com/tox-dev/tox) to permit the latest version.
- [Release notes](https://github.com/tox-dev/tox/releases)
- [Changelog](https://github.com/tox-dev/tox/blob/main/docs/changelog.rst)
- [Commits](tox-dev/tox@4.8.0...4.9.0)

---
updated-dependencies:
- dependency-name: tox
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
* Update tox requirement from ~=4.9 to ~=4.10

Updates the requirements on [tox](https://github.com/tox-dev/tox) to permit the latest version.
- [Release notes](https://github.com/tox-dev/tox/releases)
- [Changelog](https://github.com/tox-dev/tox/blob/main/docs/changelog.rst)
- [Commits](tox-dev/tox@4.9.0...4.10.0)

---
updated-dependencies:
- dependency-name: tox
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
* restructure macros directory, create separate files and directories for each ddl statement and relation type
McKnight-42 and others added 18 commits August 28, 2023 15:53
* update RELEASE_BRANCH env

* remove some connection fields that may be sensitive user information that we don't want showing up in dbt debug commands

* add changelog and comment pointing to docs
* Update tox requirement from ~=4.10 to ~=4.11

Updates the requirements on [tox](https://github.com/tox-dev/tox) to permit the latest version.
- [Release notes](https://github.com/tox-dev/tox/releases)
- [Changelog](https://github.com/tox-dev/tox/blob/main/docs/changelog.rst)
- [Commits](tox-dev/tox@4.10.0...4.11.0)

---
updated-dependencies:
- dependency-name: tox
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
* Bump mypy from 1.5.0 to 1.5.1

Bumps [mypy](https://github.com/python/mypy) from 1.5.0 to 1.5.1.
- [Commits](python/mypy@v1.5.0...v1.5.1)

---
updated-dependencies:
- dependency-name: mypy
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
Co-authored-by: Matthew McKnight <91097623+McKnight-42@users.noreply.github.com>
* update RELEASE_BRANCH env

* push up fix to core change in agate_helper

* add changelog
* Update black requirement from ~=23.7 to ~=23.9

Updates the requirements on [black](https://github.com/psf/black) to permit the latest version.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](psf/black@23.7.0...23.9.1)

---
updated-dependencies:
- dependency-name: black
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
* Update pre-commit requirement from ~=3.3 to ~=3.4

Updates the requirements on [pre-commit](https://github.com/pre-commit/pre-commit) to permit the latest version.
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md)
- [Commits](pre-commit/pre-commit@v3.3.0...v3.4.0)

---
updated-dependencies:
- dependency-name: pre-commit
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add automated changelog yaml from template for bot PR

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Github Build Bot <buildbot@fishtownanalytics.com>
* bug/Dynamic-table-comment-on-syntax
* Add changelog
* Fix relation type
* Fix alter
…and vice versa (#753)

* restructure macros directory, create separate files and directories for each ddl statement and relation type
* pull create_or_replace_view into dbt-snowflake, update conditional to drop all non-views versus drop all tables
* removed unnecessary create abstractions to simplify diffs
* implement create or replace
* test against clone replace branch
* update renameable_relations and replaceable_relations to use frozensets for unmutable defaults

---------

Co-authored-by: Matthew McKnight <91097623+McKnight-42@users.noreply.github.com>
Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com>
* Add failing test

* Fix model test naming

* Add redaction looping logic

* Apply suggestions from code review

Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com>

* Add changelog

* Fix test case

* Rename test class

* Colocate redaction tests

* Ignore if dbt run passes or fails

* Materialize as a table instead of a view to trigger an error

* Expect the run to fail with a specific error message

* Reverse order of dict, assert that sensitive data is replaced

* Add newline

---------

Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com>
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
* set snowflake connector log level on env variable

* set snowflake connector log level on env variable

* set snowflake connector log level on env variable

* add boto packages

* add logging about stting up logging

* merge paw/type-fix

* add changie

* fix event import

* add unit tests

* add unit tests
* add dbt show tests

* changelog entry

* repoint to core main

* reuse core fixture for dbt show sql header test

* point to @improve-show-fixture dbt-core branch

* undo dev-requirements changes

---------

Co-authored-by: Matthew McKnight <91097623+McKnight-42@users.noreply.github.com>
Co-authored-by: Matthew McKnight <matthew.mcknight@dbtlabs.com>
* update RELEASE_BRANCH env

* test semver in 1.6
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
* bug/Dynamic-table-comment-on-syntax
* Add changelog
* Fix relation type
* Fix alter
* Update dynamic table comment alter statement
* update comment on syntax

---------

Co-authored-by: Pat Kearns <pat.kearns@dbtlabs.com>
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com>
@cla-bot cla-bot bot added the cla:yes label Oct 11, 2023
mikealfare and others added 6 commits October 11, 2023 13:21
* implement persist test results tests, leave querying relation type as open issue

* update to use centralized relation type tests

* update changelog and test names to reflect renamed parameter

* implement store-failures-as test cases

* revert dev branch update

* update tests to cover new test cases in dbt-core
…ations (#758)

* parameterize where clause, add option to supply list of relations

* parameterize where clause, add option to supply list of relations

* revert whitespace fix

* revert whitespace fix

* revert whitespace fix

* fix missing macro keyword

* point to the dev branch on core, revert before pushing to main

* add new macro get_catalog_relations, update get_catalog to share common logic with get_catalog_relations

* fixed reference in get_catalog_relations, added original dict version of relations temporarily for testing

* remove dict version of relations based get_catalog, point to List[BaseRelation] version

* point dev reqs back to main on core

* fix typo in schemas argument

* add feature flag to turn on relation filtering for get_catalog

* update changelog to point to the PR instead of a broken url

* Support changes to dbt-core capability system

---------

Co-authored-by: Peter Allen Webb <peter.webb@dbtlabs.com>
* add transient configs

* Backwards-compatible handling of `temporary` and `transient` configs for dbt python models

* Fix Jinja syntax errors

* add test for table_type on python models

---------

Co-authored-by: jeremyyeo <jeremyyeo@gmail.com>
Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com>
Co-authored-by: Mike Alfare <mike.alfare@dbtlabs.com>
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
* Draft implementation of metadata-based freshness

* Add test of metadata-based freshness mechanism

* Strengthen test case.

* Fix some overzealous escaping.

* Simplifications per review

* Update for core code review changes

* Repoint to main core branch before merge

* Temporarily skip test

* test using custom schema for test

* use specific schema env var

* use specific schema env var

* cleanup env var

* skip test_get_last_relation_modified

* remove extra whitespace

* remove Capability.TableLastModifiedMetadata: CapabilitySupport(support=Support.Full) for CI troubleshooting

* remove metadata.sql for CI troubleshooting

* remove test_get_last_relation_modified.py for CI troubleshooting

* add non-test files back for CI troubleshooting

* add test files back for CI troubleshooting

* remove run order dependency from test cases within their test class

---------

Co-authored-by: colin-rogers-dbt <111200756+colin-rogers-dbt@users.noreply.github.com>
Co-authored-by: Colin <colin.rogers@dbtlabs.com>
Co-authored-by: Mike Alfare <13974384+mikealfare@users.noreply.github.com>
Co-authored-by: Mike Alfare <mike.alfare@dbtlabs.com>
* Clean up changelog on main

* Bumping version to 1.8.0a1

* Code quality cleanup
@dataders dataders closed this Oct 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.