Skip to content

Commit

Permalink
Overhaul Postgres reference page with more references
Browse files Browse the repository at this point in the history
  • Loading branch information
Starefossen committed Oct 4, 2024
1 parent 921137a commit 53d0da8
Show file tree
Hide file tree
Showing 3 changed files with 85 additions and 47 deletions.
18 changes: 9 additions & 9 deletions docs/persistence/postgres/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ spec:
...
gcp:
sqlInstances:
- type: POSTGRES_14
- type: POSTGRES_16
tier: db-f1-micro
databases:
- name: mydb
Expand Down Expand Up @@ -71,11 +71,11 @@ Your application may not be able to connect to the database until the provisioni
#### Invalid request: backup retention should be >= transaction log retention

!!! faq "Answer"
This error occurs when the backup retention is set to a value lower than the transaction log retention.
The backup retention should be equal to or greater than the transaction log retention.
You can fix this by setting the `retainedBackups` to a value equal to or greater than the transaction log retention or
This error occurs when the backup retention is set to a value lower than the transaction log retention.
The backup retention should be equal to or greater than the transaction log retention.
You can fix this by setting the `retainedBackups` to a value equal to or greater than the transaction log retention or
by setting the `transactionLogRetentionDays` to a value equal to or less than the backup retention.
This can be configured in the [NAIS manifest](../../workloads/application/reference/application-spec.md#gcpsqlinstances).
This can be configured in the [NAIS manifest](../../workloads/application/reference/application-spec.md#gcpsqlinstances).
Read more about [automated backups with cloud sql](https://cloud.google.com/sql/docs/mysql/backup-recovery/backups#automated-backups).

#### Cannot disable cloudsql.logical_decoding while there are logical replication slots
Expand All @@ -87,17 +87,17 @@ Your application may not be able to connect to the database until the provisioni
#### ...immutable field(s): [Field Name: settings.0.diskSize, Got: x, Wanted: xx]...

!!! faq "Answer"
This error occurs when you try to change the disk size of the database instance.
The disk size settings of the database instance cannot be less then current size after the instance is created.
You can fix this by specifying in the [NAIS manifest](../../workloads/application/reference/application-spec.md#gcpsqlinstancesdisksize)
This error occurs when you try to change the disk size of the database instance.
The disk size settings of the database instance cannot be less then current size after the instance is created.
You can fix this by specifying in the [NAIS manifest](../../workloads/application/reference/application-spec.md#gcpsqlinstancesdisksize)
the desired disk size of the database instance to be equal to or greater than the current size.
If you want to control the disk size of the instance you should disable [automatic storage increase](../../workloads/application/reference/application-spec.md#gcpsqlinstancesdiskautoresize).

#### Not allowed to do major version upgrade from POSTGRES_x to POSTGRES_xx

!!! faq "Answer"
This error occurs when you try to 'upgrade' the major version of the database instance to a version that is not allowed.
You cant downgrade the major version of the database instance, if you want to downgrade the version you need to create
You cant downgrade the major version of the database instance, if you want to downgrade the version you need to create
a new instance with the desired version.
You can fix this by specifying in the [NAIS manifest](../../workloads/application/reference/application-spec.md#gcpsqlinstancestype)
the version type to the same as the current or a higher version.
Expand Down
4 changes: 2 additions & 2 deletions docs/persistence/postgres/how-to/change-tier.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Choosing an appropriate tier depends on your application's requirements such as:

The tier you choose will affect the performance and cost of your database.

See [the reference page for a list of available tiers](../reference/README.md#instance-tiers).
See [the reference page for a list of available tiers](../reference/README.md#instance-tiers).

## Change tier

Expand All @@ -22,7 +22,7 @@ To change the tier, modify the `tier` field in your application manifest:
spec:
gcp:
sqlInstances:
- type: POSTGRES_14
- type: POSTGRES_16
tier: db-custom-1-3840
databases:
- name: mydb
Expand Down
110 changes: 74 additions & 36 deletions docs/persistence/postgres/reference/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,11 @@ tags: [postgres, reference]

## Configuration

To connect your application to the database, use information from the environment variables below.
The full configuration options for the Postgres instance can be found in the [application spec reference](../../../workloads/application/reference/application-spec.md#gcpsqlinstances).

## Database connnection

The prefix `NAIS_DATABASE_MYAPP_MYDB` is automatically generated from the instance name `myapp` \(defaults to application name\) and `mydb` \(from database spec\). You can customize these environment variable names by setting `.spec.gcp.sqlInstances[].databases[].envVarPrefix`. For instance, setting this to `DB` will give you `DB_HOST`, `DB_USERNAME`, etc. Note that changing or adding `envVarPrefix` requires you to manually delete the `google-sql-<MYAPP>` secret and `SQLUser` with the same name as the application, see below.
To connect your application to the database, use information from the environment variables below.

| description | environment variable | example |
| :------------------------------------------------------------- | :------------------------------------- | :------------------------------------------------------------------------- |
Expand All @@ -29,28 +31,33 @@ The prefix `NAIS_DATABASE_MYAPP_MYDB` is automatically generated from the instan
The application is the only application that can access the database instance. Other applications can not connect. It is not, for instance,
possible to have two applications (e.g. producer and consumer) connecting directly to the database.

### Naming conventions

The prefix `NAIS_DATABASE_MYAPP_MYDB` is automatically generated from the instance name `myapp` \(defaults to application name\) and `mydb` \(from database spec\).

You can customize these environment variable names by setting `.spec.gcp.sqlInstances[].databases[].envVarPrefix`.

For instance, setting this to `DB` will give you `DB_HOST`, `DB_USERNAME`, etc. Note that changing or adding `envVarPrefix` requires you to manually delete the `google-sql-<MYAPP>` secret and `SQLUser` with the same name as the application, see below.

If you habe hyphens (`-`) in your application name or database name, they will be converted to underscores (`_`). E.g. `my-awesome-app` = `MY_AWESOME_APP`.

!!! info
Note that if you change your application name, database name or envVarPrefix, and then change it later,
you have to manually [reset database credentials](../how-to/reset-database-credentials.md).

!!! info
Note that if your application name or database has hyphens in the name, the hyphens are converted to underscores. E.g. my-awesome-app = MY_AWESOME_APP

[^1]: jdbc url can be generated for instances without private IP by using [nais-cli] to rotate the password.

### Database flags
## Database flags

!!! info
!!! warning
Use database flags with caution, they will alter the behaviour of your postgres instance.

Postgres in gcp supports setting database flags to alter the database instance performance and functionality,
the flags available are listed here: [Google Cloud SQL supported flags](https://cloud.google.com/sql/docs/postgres/flags#list-flags-postgres).
This listing specifies what value types are expected, which ranges are allowed and if a restart is required.

!!! info
The value is always required to be a string in [`nais.yaml`](../../../workloads/application/reference/application-spec.md).

Example of setting database flags:

```yaml
...
kind: Application
Expand All @@ -60,6 +67,7 @@ spec:
gcp:
sqlInstances:
- type: POSTGRES_14
tier: db-f1-micro
databases:
- name: mydb
flags:
Expand All @@ -71,7 +79,10 @@ spec:
value: "2" #float in google spec
```
#### Database max connections
!!! info
The value is always required to be a string in [`nais.yaml`](../../../workloads/application/reference/application-spec.md).

### Database max connections

The default maximum number of connections is dependent on the memory size of the instance and can be configured with the `max_connections` flag. The smallest instance size has a default of 25 maximum connections.

Expand All @@ -80,7 +91,7 @@ Default connection limits for different memory sizes can be found on [Google Clo
!!! info
The actual connection limit for your application may be lower than the configured value due some connections being reserved for superusers and other internal processes.

### Database Metrics
## Database metrics

Postgres instances automatically collect metrics that can be viewed in the following locations:

Expand All @@ -92,7 +103,7 @@ A list of all avaialbe metrics can be found on [Cloud SQL metrics](https://cloud

[:dart: Learn how to view Postgres metrics in the Google Cloud Console, Grafana, and NAIS Console](../how-to/database-observability.md)

### Query Insights
### Query insights

Query insights are now enabled by default in GCP. This feature provides query overview and analysis.
The data is available in the Google cloud console.
Expand All @@ -102,19 +113,20 @@ For further reading see [Google Cloud SQL Query Insights](https://cloud.google.c
!!! info
Data is available for seven days, increasing this will incur extra cost.

### Maintenance window
## Maintenance window

Google will automatically perform upgrades, fix bugs and apply security patches to prevent exploits. Your application should be able to handle occasional downtime as this maintenance is performed. [Read more on maintenance windows](https://cloud.google.com/sql/docs/postgres/maintenance). NAIS does not configure the maintenance window, but this can be set up in the application spec: [`nais.yaml`](../../../workloads/application/reference/application-spec.md#gcpsqlinstances).
If you wish to be notified about upcoming maintenance, you can opt-in for this on the [Communications page](https://console.cloud.google.com/user-preferences/communication) in the GCP console.

### Automated backup
## Automated backup

The database is backed up nightly at 3 AM \(GMT+1\) by default, but can be overridden in [`nais.yaml`](../../../workloads/application/reference/application-spec.md#gcpsqlinstancesautobackuphour) by setting `spec.gcp.sqlInstances[].autoBackupTime`.
By default, seven backups will be kept. More info [about Cloud SQL backups](https://cloud.google.com/sql/docs/postgres/backup-recovery/backups).

The backups can be found in the [Google Cloud SQL instance](https://cloud.google.com/sql) dashboard.

#### Point-in-time recovery
### Point-in-time recovery

Point-in-time recovery can be enabled by configuring this in the sql instance for your application spec.
This feature allows you to recover your database to a specific point in time.

Expand All @@ -127,38 +139,64 @@ This feature allows you to recover your database to a specific point in time.
See [application spec reference.](https://doc.nais.io/nais-application/application/#gcpsqlinstancespointintimerecovery)
For further reading see [google Cloud SQL PIT recovery](https://cloud.google.com/sql/docs/postgres/backup-recovery/pitr)

#### Disaster backup
### Disaster backup

In case of catastrophic failure in GCP we are running a daily complete backup of the postgresql databases in GCP to an on-prem location. This backup currently runs at 5 am. This is in addition to the regular backups in GCP.

### Instance tiers
## Postgres vesion

List of commonly used database instance tiers:
The Postgres version can be configured in the application spec. The version is defined by the `type` field in the `nais.yaml` file.

```yaml
spec:
gcp:
sqlInstances:
- type: POSTGRES_16
```

The full list of supported versions can be found in the [application spec reference](../../../workloads/application/reference/application-spec.md#gcpsqlinstancestype).

### Upgrading Postgres version

To upgrade the Postgres version, modify the `type` field in your application manifest to the desired version and redeploy the application. The upgrade will start automatically.

:dart: [Learn how to safely upgrade the Postgres version](../how-to/upgrade-postgres.md)

## Server size

The server instance size can be configured in the application spec. The instance size is defined by the `tier` field in the `nais.yaml` file.

```yaml
spec:
gcp:
sqlInstances:
- type: POSTGRES_164
tier: db-custom-2-5120
```

The tier can be set to one of the following values:

[Cloud SQL SLA]: https://cloud.google.com/sql/sla

| Tier | Description |
|-------------------------|--------------------------------------------------------|
| ----------------------- | ------------------------------------------------------ |
| `db-f1-micro` | Shared CPU, 0.6 GB RAM, not covered by [Cloud SQL SLA] |
| `db-g1-small` | Shared CPU, 1.7 GB RAM, not covered by [Cloud SQL SLA] |
| `db-custom-1-3840` | |
| `db-custom-2-5120` | |
| `db-custom-2-7680` | |
| `db-custom-4-15360` | |
| `db-custom-<CPU>-<RAM>` | |
| `db-custom-1-3840` | 1 CPU, 3.75 GB RAM |
| `db-custom-2-5120` | 2 CPUs, 5 GB RAM |
| `db-custom-2-7680` | 2 CPUs, 7.5 GB RAM |
| `db-custom-4-15360` | 4 CPUs, 15 GB RAM |
| `db-custom-<CPU>-<RAM>` | Custom tier, see below |

The custom tiers can be customized to fit your application's requirements, following this format:

```
db-custom-<CPU>-<RAM>
```

- Replace the `<CPU>` placeholder with the number of CPUs
- Replace the `<RAM>` placeholder with the amount of memory.
* Replace the `<CPU>` placeholder with the number of CPUs
* Replace the `<RAM>` placeholder with the amount of memory.

When selecting the number of CPUs and amount of memory, there are some restrictions on the configuration you choose:

- CPU must be either 1 or an even number between 2 and 96.
- Memory must be:
- 0.9 to 6.5 GB per vCPU
- A multiple of 256 MB
- At least 3.75 GB (3840 MB)
* CPU must be either 1 or an even number between 2 and 96.
* Memory must be:
* 0.9 to 6.5 GB per vCPU
* A multiple of 256 MB
* At least 3.75 GB (3840 MB)

0 comments on commit 53d0da8

Please sign in to comment.