Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-2790] Support dispatch for materialization macros #8031

Open
jtcohen6 opened this issue Jul 5, 2023 · 1 comment
Open

[CT-2790] Support dispatch for materialization macros #8031

jtcohen6 opened this issue Jul 5, 2023 · 1 comment
Assignees
Labels
tech_debt Behind-the-scenes changes, with little direct impact on end-user functionality

Comments

@jtcohen6
Copy link
Contributor

jtcohen6 commented Jul 5, 2023

Definition of done:

  • It's possible to call adapter.dispatch for materialization macros, and return the appropriate macro for this adapter, which can then be called/executed from within another macro/materialization

In conversation with Kshitij just now, it seems that the main changes need to be in core/dbt/context/macros.py, in MacroNamespace.get_from_package where we can add special processing of "materialization" macros. The "get_from_package" method is called from adapter.dispatch, which comes from the adapter "contextproperty" of the ProviderContext in core/dbt/context/providers.py. The actual "dispatch" method is in the BaseDatabaseWrapper.

The materialization macro can be found in the "execute" method of the clone task (and probably the same method in runnable.py) by doing context.adapter.dispatch from the constructed context. This should already be a MacroGenerator callable, so we wouldn't need to construct a MacroGenerator as we do further down in "execute".

Originally posted by @gshank in #7799 (comment)


Motivation

This would enable us to refactor the implementation in the clone materialization here:

-- reuse the view materialization
-- TODO: support actual dispatch for materialization macros
-- Tracking ticket: https://github.com/dbt-labs/dbt-core/issues/7799
{% set search_name = "materialization_view_" ~ adapter.type() %}
{% if not search_name in context %}
{% set search_name = "materialization_view_default" %}
{% endif %}
{% set materialization_macro = context[search_name] %}
{% set relations = materialization_macro() %}
{{ return(relations) }}

It might also unblock an approach for this desirable UX improvement to model versions:

@jtcohen6 jtcohen6 added the tech_debt Behind-the-scenes changes, with little direct impact on end-user functionality label Jul 5, 2023
@github-actions github-actions bot changed the title Support dispatch for materialization macros [CT-2790] Support dispatch for materialization macros Jul 5, 2023
@gshank
Copy link
Contributor

gshank commented Jul 24, 2023

This would constitute a new way of calling materialization macros with a different method of inheritance. There's also some complexity in that materialization macros can have parameters that don't exist in "normal" macros which would need to be handled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tech_debt Behind-the-scenes changes, with little direct impact on end-user functionality
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants