Skip to content

Commit

Permalink
[#3961] Enable cataloging of unlogged Postgres tables (#3993)
Browse files Browse the repository at this point in the history
* Update catalog macro & add tests

* [#3961] Enable cataloging of unlogged Postgres tables

* Update contributors and add new lines to test
  • Loading branch information
Sam Lader authored Oct 6, 2021
1 parent 6fc64f0 commit 91b43f7
Show file tree
Hide file tree
Showing 5 changed files with 76 additions and 1 deletion.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
### Fixes
- Add generic tests defined on sources to the manifest once, not twice ([#3347](https://github.com/dbt-labs/dbt/issues/3347), [#3880](https://github.com/dbt-labs/dbt/pull/3880))
- Skip partial parsing if certain macros have changed ([#3810](https://github.com/dbt-labs/dbt/issues/3810), [#3982](https://github.com/dbt-labs/dbt/pull/3892))
- Enable cataloging of unlogged Postgres tables ([3961](https://github.com/dbt-labs/dbt/issues/3961), [#3993](https://github.com/dbt-labs/dbt/pull/3993))

### Under the hood

Expand All @@ -32,6 +33,7 @@ Contributors:

- [@dave-connors-3](https://github.com/dave-connors-3) ([#3920](https://github.com/dbt-labs/dbt/issues/3920))
- [@kadero](https://github.com/kadero) ([#3952](https://github.com/dbt-labs/dbt/issues/3952))
- [@samlader](https://github.com/samlader) ([#3993](https://github.com/dbt-labs/dbt/pull/3993))

## dbt 0.21.0 (October 04, 2021)

Expand Down
2 changes: 1 addition & 1 deletion plugins/postgres/dbt/include/postgres/macros/catalog.sql
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
{%- endfor -%}
)
and not pg_is_other_temp_schema(sch.oid) -- not a temporary schema belonging to another session
and tbl.relpersistence = 'p' -- [p]ermanent table. Other values are [u]nlogged table, [t]emporary table
and tbl.relpersistence in ('p', 'u') -- [p]ermanent table or [u]nlogged table. Exclude [t]emporary tables
and tbl.relkind in ('r', 'v', 'f', 'p') -- o[r]dinary table, [v]iew, [f]oreign table, [p]artitioned table. Other values are [i]ndex, [S]equence, [c]omposite type, [t]OAST table, [m]aterialized view
and col.attnum > 0 -- negative numbers are used for system columns such as oid
and not col.attisdropped -- column as not been dropped
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
version: 2
models:
- name: table_unlogged
description: "Unlogged table model"
columns:
- name: column_a
description: "Sample description"
quote: true
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{{ config(materialized = 'table', unlogged = True) }}

select 1 as column_a
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
import re
import json
from test.integration.base import DBTIntegrationTest, use_profile


class TestPostgresUnloggedTable(DBTIntegrationTest):
@property
def schema(self):
return "postgres_unlogged_074"

@property
def models(self):
return "models"

@property
def project_config(self):
return {
'config-version': 2,
'models': {
'test': {
'materialized': 'table',
'+persist_docs': {
"relation": True,
"columns": True,
},
}
}
}

@use_profile('postgres')
def test__postgres__unlogged__table__catalog(self):
table_name = 'table_unlogged'

results = self.run_dbt(['run', '--models', table_name])
self.assertEqual(len(results), 1)

assert self.get_table_persistence(table_name) == 'u'

self.run_dbt(['docs', 'generate'])

with open('target/catalog.json') as fp:
catalog_data = json.load(fp)

assert len(catalog_data['nodes']) == 1

table_node = catalog_data['nodes'][f'model.test.{table_name}']
assert 'column_a' in table_node['columns']

def get_table_persistence(self, table_name):
sql = """
SELECT
relpersistence
FROM pg_class
WHERE relname = '{table_name}'
"""
sql = sql.format(table_name=table_name, schema=self.unique_schema())

result, = self.run_sql(sql, fetch='one')

self.assertEqual(len(result), 1)

return result

0 comments on commit 91b43f7

Please sign in to comment.