Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Add documentation on unset and defaults #2882

Merged
merged 2 commits into from
Jun 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions docs/technical-documentation/restoring_lost_data.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Restoring Lost Data with Time-Travel
sfc-gh-asawicki marked this conversation as resolved.
Show resolved Hide resolved

If you've ever accidentally deleted important data either by using Terraform or by hand, there's still hope to recover the data.
By using the Snowflake's Time-Travel feature, you can restore lost data and undo those accidental deletions.

> Note: Currently, the recovery process is predominantly manual, relying on SQL commands and the Terraform CLI.
We made a strategic decision not to integrate it as a provider feature at this time, as demand for this functionality was not significant.
Following the release of V1, we intend to reassess the topic of data recovery and UNDROP functionality to explore potential integration into the provider, evaluating its necessity and feasibility.

You should be prepared beforehand by specifying how much of the historical data Snowflake should keep by setting the [DATA_RETENTION_TIME_IN_DAYS](https://docs.snowflake.com/en/sql-reference/parameters#data-retention-time-in-days) parameter.
When using our provider, you can set this by using one of our parameter-setting resources (like [snowflake_account_parameter](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/resources/account_parameter) or [snowflake_object_parameter](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/resources/object_parameter))
or set it on the resource level (e.g. `data_retention_time_in_days` in [snowflake_database](https://registry.terraform.io/providers/Snowflake-Labs/snowflake/latest/docs/resources/database)).

> Note: If some of the resources support `data_retention_time_in_days` parameter in Snowflake, but it's not available in the provider, we'll add it during [the resource preparation for V1](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md#preparing-essential-ga-objects-for-the-provider-v1).

Now, with [DATA_RETENTION_TIME_IN_DAYS](https://docs.snowflake.com/en/sql-reference/parameters#data-retention-time-in-days) set up,
let's imagine we accidentally dropped a database that was managed by Terraform and contained a lot of important data we would like to recover.
But before we start, let's clearly understand the initial state of the database we'll be recovering.

The configuration will contain only a database as we want to mainly focus on the data recovery from the Terraform point of view:
```terraform
resource "snowflake_database" "test" {
name = "TEST_DATABASE"
}
```

The rest of the objects will be created in SQL by hand (e.g. in the worksheet):
```sql
CREATE SCHEMA TEST_DATABASE.TEST_SCHEMA;
CREATE TABLE TEST_DATABASE.TEST_SCHEMA.TEST_TABLE(id INT, name STRING);
INSERT INTO TEST_DATABASE.TEST_SCHEMATEST_TABLE(id, name) VALUES (0, 'john'), (1, 'doe');
```
As you can see, we created a schema and a table with "important" data.

As the next step we will remove the database from the configuration and run `terraform apply` that will trigger the `DROP DATABASE`.
Now, we're in a state where the data is lost and can only be recovered by Time-Travel. By checking up the [DATA_RETENTION_TIME_IN_DAYS](https://docs.snowflake.com/en/sql-reference/parameters#data-retention-time-in-days)
parameter, we can calculate if the dropped database can still be recovered or not (that's why it's important to prepare for such situations beforehand,
to avoid situations where the parameter was set to low value and the data is lost).

To recover the database (and the data inside it), we have to call `UNDROP DATABASE TEST_DATABASE` manually.
To bring the database back to the Terraform configuration, we have to specify the same configuration as previously, but now,
instead of running `terraform apply` we have to import it by calling `terraform import 'TEST_DATABASE'`.
After successful import the `terraform plan` shouldn't produce any plan for the database.
To ensure all the important data we inserted before is there, we can call `SELECT * FROM TEST_DATABASE.TEST_SCHEMA.TEST_TABLE;`.

To learn more about how to use Time-Travel, check out the links below:
1. [Understanding & using Time-Travel](https://docs.snowflake.com/en/user-guide/data-time-travel)
2. [DATA_RETENTION_TIME_IN_DAYS parameter](https://docs.snowflake.com/en/sql-reference/parameters#data-retention-time-in-days)
3. [UNDROP command with available objects to restore](https://docs.snowflake.com/en/sql-reference/sql/undrop)
16 changes: 15 additions & 1 deletion v1-preparations/CHANGES_BEFORE_V1.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,20 @@
# Changes before v1

This document is a changelog of resources and datasources as part of the https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md#preparing-essential-ga-objects-for-the-provider-v1. Each provider version lists changes made in resources and datasources definitions during v1 preparations, like added, modified and removed fields.
This document is a changelog of resources and datasources as part of the https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md#preparing-essential-ga-objects-for-the-provider-v1.
Each provider version lists changes made in resources and datasources definitions during v1 preparations, like added, modified and removed fields.

## Default values
For any resource that went through the rework as part of the [resource preparation for V1](https://github.com/Snowflake-Labs/terraform-provider-snowflake/blob/main/ROADMAP.md#preparing-essential-ga-objects-for-the-provider-v1),
the behaviour for default values may change from the previous one.

In the past, the provider copied defaults from Snowflake, creating a tight coupling between them.
However, this approach posed a challenge as the defaults on the Snowflake side could change and vary between accounts based on their configurations.

Now, whenever the value is not specified in the configuration, we let the Snowflake fill out the default value for a given field
(if there is one). Using such defaults may lead to non-idempotent cases where the same configuration may
create a resource with slightly different configuration in Snowflake (depending on the Snowflake Edition and Version,
current account configuration, and most-likely other factors). That is why we recommend setting optional fields where
you want to ensure that the specified value has been set on the Snowflake side.

## v0.91.0 ➞ v0.92.0
### snowflake_scim_integration resource changes
Expand Down
Loading