Typed materialized views as views

[luisleon90]

This is a:

• bug fix PR with no breaking changes — please ensure the base branch is main
• new functionality — please ensure the base branch is the latest dev/ branch
• a breaking change — please ensure the base branch is the latest dev/ branch

Description & motivation

Checklist

• I have verified that these changes work locally on the following warehouses (Note: it's okay if you do not have access to all warehouses, this helps us understand what has been covered)
  • BigQuery
  • Postgres
  • Redshift
  • Snowflake
• I followed guidelines to ensure that my changes will work on "non-core" adapters by:
  • dispatching any new macro(s) so non-core adapters can also use them (e.g. the star() source)
  • using the limit_zero() macro in place of the literal string: limit 0
  • using dbt_utils.type_* macros instead of explicit datatypes (e.g. dbt_utils.type_timestamp() instead of TIMESTAMP
• I have updated the README.md (if applicable)
• I have added tests & descriptions to my models (and macros if applicable)
• I have added an entry to CHANGELOG.md

[joellabes]

Hi @luisleon90 - thank you for opening this! We solved a similar problem in #476, and the solution there is very transferable to your use case.

I had to spend some time digging into this to work out what was going on, but here's what I've discovered:

• the documented entry point to the macros is indeed get_relations_by_pattern(), but that doesn't load any data itself, instead shuttling it out to the similarly-named get_tables_by_pattern(). I suspect this was a backwards-compatibility thing - when we release dbt_utils 1.0 in the future, I'd love to officially deprecate the get_tables* stuff altogether and move it into get_relations* instead. (See also get_*_by_prefix())
• In Added case for handling postgres foreign tables... #476, we discovered that Postgres calls external tables foreign instead of external. To fix this, we made a new macro called get_table_types_sql(), which has a default implementation and a Postgres-specific implementation. This is called from default__get_tables_by_pattern
• BigQuery has a different version, bigquery__get_tables_by_pattern, which doesn't call that new macro so is missing out.

The actual best place to fix this up would be in here. Specifically, by swapping out the BQ-specific case statement in lines 33-36 with {{ dbt_utils.get_table_types_sql() }}.

By doing that, you'll automatically get the BASE TABLE and MATERIALIZED VIEW conversions 🎉

Let me know if you need any extra clarification on this!

[luisleon90]

Joel, thanks for the helpful feedback. I followed your advice and called the get_table_types_sql macro within get_tables_by_pattern_sql to better handle the cases of materialized views and base tables. I was getting an unexpected string literal error, in order to resolve this I removed some quoting from the table_type variable at the end of the default__fet_table_types_sql macro. I wasn't super confident of this change but it seems to have resolved the error on my end and I am not getting the expected output.

The base branch was changed.

[joellabes]

This looks great! Thanks for working through it 🎉