Forced Replacement of one resource renders another resource invalid

[justin-ramirez-gametime]

Terraform CLI and Provider Versions

Terraform v1.7.0 on darwin_arm64
provider registry.terraform.io/snowflake-labs/snowflake v0.84.0

Terraform Configuration

resource "snowflake_materialized_view" "materialized_view" {
  provider   = snowflake.sys_admin
  database   = var.database_name
  schema     = var.schema_name
  name       = var.materialized_view_name
  warehouse  = var.warehouse_name
  comment    = var.description
  statement  = var.sql_statement
  or_replace = var.replace_if_exists
  is_secure  = var.is_secure
}

locals {
  materialized_views = [
    {
      database_name          = module.snowflake_external_table["<resource_name>"].database_name
      schema_name            = module.snowflake_external_table["<resource_name>"].schema_name
      materialized_view_name = "<resource_name>_MV"
      sql_statement          = <<SQL
      SELECT *
      FROM ${module.snowflake_external_table["<resource_name>"].database_name}.${module.snowflake_external_table["<resource_name>"].schema_name}.${module.snowflake_external_table["<resource_name>"].external_table_name};
      SQL
      warehouse_name         = module.team_warehouses["DATA_ENGINEER_LARGE"].warehouse_name
      description            = "Depends on table: ${module.snowflake_external_table["<resource_name>"].id}"  # our attempt to update whenever the source table changes
      is_secure              = false
      replace_if_exists      = true
    },
    #...
  ]
}

module "snowflake_materialized_view" {
  source = "../../../modules/snowflake/materialized-view"
  providers = {
    snowflake.sys_admin = snowflake.sys_admin
  }
  for_each = {
    for materialized_view in local.materialized_views :
    "${materialized_view.database_name}.${materialized_view.schema_name}.${materialized_view.materialized_view_name}"
    => materialized_view
  }
  database_name          = each.value.database_name
  schema_name            = each.value.schema_name
  materialized_view_name = each.value.materialized_view_name
  sql_statement          = each.value.sql_statement
  warehouse_name         = each.value.warehouse_name
  description            = lookup(each.value, "description", null)
  is_secure              = lookup(each.value, "is_secure", null)
  replace_if_exists      = lookup(each.value, "replace_if_exists", null)
}

Expected Behavior

Make changes to an external table(i.e. change comment field) -> it will be destroyed and replaced on apply(# forces replacement) -> if there is a materialized view which depends on it -> the materialized view is destroyed and recreated after the external table is recreated.

Actual Behavior

If any change is made to an external table, that table is destroyed and recreated on apply(# forces replacement).

When this happens, any materialized_view which is based on that table is marked as invalid because the resource it was based on has been dropped. We tried to make the materialized_view depend on the id of the external table(shown in the config code above). But this only led to an 'edit in place' for the materialized_view.

If any other changes are attempted on the invalid materialized_view on subsequent applies, you will get an error. The Materialized view needs to be destroyed and recreated in order to get it back.

Steps to Reproduce

1. terraform apply
2. query materialized view

How much impact is this issue causing?

High

Logs

No response

Additional Information

This same problem has affected external tables when a change was made to the stage which it depends on. It seems like this issue will affect any resource which depends on a particular underlying resource and is marked as invalid when that resource is dropped from the warehouse.

[sfc-gh-asawicki]

Hey @justin-ramirez-gametime. Thanks for reporting the issue.

Did you try to use https://developer.hashicorp.com/terraform/language/meta-arguments/lifecycle#replace_triggered_by?

We are currently working on updating the old underlying implementation of this resource to the new SDK. It won't change this behavior, though. We have a redesign of resources upcoming on our roadmap (https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md#19012024-roadmap-overview). I will add this as a topic to discuss then.

[justin-ramirez-gametime]

This was helpful. Here is what we did for now:

resource "snowflake_materialized_view" "materialized_view" {
  provider   = snowflake.sys_admin
  database   = var.database_name
  schema     = var.schema_name
  name       = var.materialized_view_name
  warehouse  = var.warehouse_name
  comment    = var.description
  statement  = var.sql_statement
  or_replace = var.replace_if_exists
  is_secure  = var.is_secure
  # This is a workaround to force the materialized view to be replaced when the source table is changed/recreated
  lifecycle {
    replace_triggered_by = [terraform_data.replacement]
  }
}

resource "terraform_data" "replacement" {
  input = var.source_id
}