V0.19.0rc2 #40
V0.19.0rc2 #40
Conversation
|
ok THIS is the error I've been getting locally... tox was doing something caching thing I don't understand... Normally with |
|
I'm very much clueless here. Still getting the empty table name issue for ts_snapshot.sql that is fixed in @jtcohen6, if you have time this week, would you mind taking this branch for a spin and running pytest locally? also happy to team up and tackle this together synchronously if you'd like. |
|
It looks like the seeds fails. The tests first loads the database with data from seed files. Then executes the tests. The failure is that it tries to execute a snapshot on an empty table. If you add all tests to the spec-file again, I guess that you should see that all tests are failing. |
|
Here's some best-guesswork from me. The immediate error is related to the fact that It looks to be using the default dbt seed materialization, which has So it does seem like an issue with properly dispatching Otherwise, I'm just observing that:
|
@mikaelene I just enabled the previous tests and I'm still getting the same error. very strange |
|
This T-SQL syntax is unfamiliar to me. What should this look like? EXEC('create view dbt_test_210119001001306735066970.cc_all_snapshot_temp_view as
select *,
CONVERT(VARCHAR(32), HashBytes(''MD5'',
coalesce(CONVERT(varchar, id, 121), '''') + ''|'' +
coalesce(CONVERT(varchar, CONVERT(DATETIME2, ''2021-01-19 00:12:40.323000''), 121), '''')
), 2)
as dbt_scd_id,
CONVERT(DATETIME2, ''2021-01-19 00:12:40.323000'') as dbt_updated_at,
CONVERT(DATETIME2, ''2021-01-19 00:12:40.323000'') as dbt_valid_from,
nullif(CONVERT(DATETIME2, ''2021-01-19 00:12:40.323000''), CONVERT(DATETIME2, ''2021-01-19 00:12:40.323000'')) as dbt_valid_to
from (
) sbq
'); |
@jtcohen6 Had a breakthrough thanks to your prompting and realized that I hadn't yet purged
So now, i've done that, but the place where I'm stuck now is that For reference, here's a Gist of all the SQL called by |
|
You're right, doesn't seem to be anything here: I'm guessing that the issue is Do temp tables show up in So I think your options are:
{% macro synapse__get_columns_in_relation(relation) -%}
{# temp tables need different handling #}
{% if relation.identifier.startswith("#") %}
{% set select_star %} select * from {{ relation }} {% select_star %}
{{ return(get_columns_in_query(select_star)) }}
{% endif %}
... as normal ...
{% endmacro %}
{% macro synapse__snapshot_merge_sql(target, source, insert_cols) -%}
{%- set target_columns = adapter.get_columns_in_relation(target)
| rejectattr('name', 'equalto', 'dbt_change_type')
| rejectattr('name', 'equalto', 'DBT_CHANGE_TYPE')
| rejectattr('name', 'equalto', 'dbt_unique_key')
| rejectattr('name', 'equalto', 'DBT_UNIQUE_KEY')
| list %} -%}
{% set quoted_target_columns = [] %}
{% for column in target_columns %}
{% do quoted_source_columns.append(adapter.quote(column.name)) %}
{% endfor %}
{%- set insert_cols_csv = quoted_target_columns | join(', ') -%}
{{ log('insert_cols_csv: ' ~ insert_cols_csv, info=True) }}
EXEC('
...Only other thing, you'll want this line This enables the optional "hard delete" capture introduced in v0.19.0. (See dbt-labs/dbt-core#3004, it looks like |
🤦 ❌ 💯 ❌ 💯 (i.e. 10K facepalms). It's slowly coming back to me now. I've solved this exact problem but I don't remember/can't pinpoint my exact work-around yet. My working theory is that I made made the staging table an actual table instead of temp. Thankfully 2020 Anders left Stack Overflow breadcrumbs so that 2021 Jeremy could help! Your x-ray vision has proved crucial. For me, I think that your first suggestion is the more robust solution, in case more temp table troubles surface in the future. While it's a bit of unexpected work, lowering the code footprint of both |
|
sort of blocked by: dbt-msft/dbt-sqlserver#89 |
see changelog