Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

draft for support python in base mat macros #6127

Closed
wants to merge 1 commit into from
Closed

draft for support python in base mat macros #6127

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 17 additions & 14 deletions core/dbt/include/global_project/macros/materializations/models/incremental/incremental.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@

{% materialization incremental, default -%}

{%- set language = model['language'] -%}
-- relations
{%- set existing_relation = load_cached_relation(this) -%}
{%- set target_relation = this.incorporate(type='table') -%}
Expand Down Expand Up @@ -33,12 +34,21 @@
{% set to_drop = [] %}

{% if existing_relation is none %}
{% set build_sql = get_create_table_as_sql(False, target_relation, sql) %}
{%- call statement('main', language=language) -%}
{{ create_table_as(False, target_relation, compiled_code, language) }}
{%- endcall -%}
{% elif full_refresh_mode %}
{% set build_sql = get_create_table_as_sql(False, intermediate_relation, sql) %}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jtcohen6 any concern moving from get_create_table_as_sql to create_table_as?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heh — technically, create_table_as is the old/originally name, get_create_table_as_sql was meant to be more consistent with the fact that macros which don't actually perform a side effect (run_query), but simply return code (SQL) as a string, were named get_x_sql. It's a convention we have been very imperfect about following.

We made this change in the leadup to v1.0, based on feedback from @dataders (before he worked with us!): #4154 (review)

{% set need_swap = true %}
{%- call statement('main', language=language) -%}
{{ create_table_as(False, intermediate_relation, compiled_code, language) }}
{%- endcall -%}
{% do adapter.rename_relation(target_relation, backup_relation) %}
{% do adapter.rename_relation(intermediate_relation, target_relation) %}
{% do to_drop.append(backup_relation) %}

{% else %}
{% do run_query(get_create_table_as_sql(True, temp_relation, sql)) %}
{%- call statement('create_tmp_relation', language=language) -%}
{{ create_table_as(True, tmp_relation, compiled_code, language) }}
{%- endcall -%}
{% do adapter.expand_target_column_types(
from_relation=temp_relation,
to_relation=target_relation) %}
Expand All @@ -53,18 +63,11 @@
{% set incremental_predicates = config.get('incremental_predicates', none) %}
{% set strategy_sql_macro_func = adapter.get_incremental_strategy_macro(context, incremental_strategy) %}
{% set strategy_arg_dict = ({'target_relation': target_relation, 'temp_relation': temp_relation, 'unique_key': unique_key, 'dest_columns': dest_columns, 'predicates': incremental_predicates }) %}
{% set build_sql = strategy_sql_macro_func(strategy_arg_dict) %}

{% endif %}

{% call statement("main") %}
{{ build_sql }}
{% endcall %}
{%- call statement('main') -%}
{{ strategy_sql_macro_func(strategy_arg_dict) }}
{%- endcall -%}

{% if need_swap %}
{% do adapter.rename_relation(target_relation, backup_relation) %}
{% do adapter.rename_relation(intermediate_relation, target_relation) %}
{% do to_drop.append(backup_relation) %}
{% endif %}

{% set should_revoke = should_revoke(existing_relation, full_refresh_mode) %}
Expand Down
5 changes: 3 additions & 2 deletions core/dbt/include/global_project/macros/materializations/models/table/table.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
{% materialization table, default %}

{%- set language = model['language'] -%}
{%- set existing_relation = load_cached_relation(this) -%}
{%- set target_relation = this.incorporate(type='table') %}
{%- set intermediate_relation = make_intermediate_relation(target_relation) -%}
Expand Down Expand Up @@ -27,8 +28,8 @@
{{ run_hooks(pre_hooks, inside_transaction=True) }}

-- build model
{% call statement('main') -%}
{{ get_create_table_as_sql(False, intermediate_relation, sql) }}
{% call statement('main', language=language) -%}
{{- create_table_as(False, intermediate_relation, compiled_code, language) }}
{%- endcall %}

-- cleanup
Expand Down