diff --git a/docs/administration/index.md b/docs/administration/index.md index fbf4e19f1..f79cab983 100644 --- a/docs/administration/index.md +++ b/docs/administration/index.md @@ -25,6 +25,7 @@ Specific information: - [Applying a permanent local change to a ruleset](local-ruleset-patch.md) - [Working with the local Postfix MTA](local-postfix-mta.md) - [Upgrading metadata schemas](upgrading-metadata-schemas.md) +- [Upgrading PostgreSQL](upgrading-postgresql.md) ## Operations diff --git a/docs/administration/upgrading-postgresql.md b/docs/administration/upgrading-postgresql.md new file mode 100644 index 000000000..7b088d324 --- /dev/null +++ b/docs/administration/upgrading-postgresql.md @@ -0,0 +1,124 @@ +--- +parent: Administration Tasks +title: Upgrading PostgreSQL +nav_order: 11 +--- + +# Upgrading PostgreSQL + +This page has information about how to upgrade the PostgreSQL database in Yoda to a newer version. + +## Background + +Yoda uses PostgreSQL, an open source relational database system, as the iRODS internal database (iCAT database), +and for storing External User Service (EUS) data. + +Yoda versions up to and including 1.8.6 work with PostgreSQL 9.2. Yoda 1.8.7 and later 1.8.x versions work with PostgreSQL +9.2 by default, with an optional upgrade to PostgreSQL 15. Yoda 1.9 uses PostgreSQL 15 by default. Although Yoda 1.9 +should work with PostgreSQL 9.2, it is recommended to upgrade to PostgreSQL 15 either before/during the upgrade to Yoda 1.9, or +immediately afterwards. + +## Upgrade methods + +Existing Yoda environments can be upgraded to PostgreSQL 15 by either starting the upgrade from Ansible, or by dumping and restoring +the database manually onto a new server. + +Regardless of which method you choose, the following advice applies: +- Ensure that that you have a recent database backup before starting the upgrade process. +- First perform the upgrade process on an (acceptance) test environment that resembles your production + environments. This also helps with estimating how much time and diskspace you will need for upgrading the + production environment. +- A database upgrade is a good opportunity to review the Yoda [configuration parameters](configuring-yoda.md) for + PostgreSQL, and to see if any improvements can be made in areas such as performance tuning. + +## Upgrading the database using Ansible + +The Ansible playbook contains an option for upgrading the database automatically. Note that upgrading the database may +take a significant time on large environments. + +In Yoda 1.8.7 and later 1.8.x versions, use the following configuration parameters: + +```yaml +enable_pgbouncer: true +postgresql_use_native_packages: false +pgsql_version: 15 + +postgresql_perform_db_upgrade: true +postgresql_remove_old_data_after_upgrade: false +``` + +In Yoda 1.9.x, use just the following configuration parameters: + +```yaml +postgresql_perform_db_upgrade: true +postgresql_remove_old_data_after_upgrade: false +``` + +## Upgrading (and migrating) the database manually + +It is also possible to manually upgrade the database. This can be useful if you want to, for example, +combine the database upgrade with a migration to a new server. + +The steps are: +1. Deploy a new database server using Ansible. You can do this by adding the server that is to be the new + database server to the database group in Ansible. Don't remove the current database server yet. If the + environment is not yet on Yoda 1.9.x, you will have to set the database version explicitly in the `host_vars` + of the new database server (not the `group_vars`), like so: + +```yaml +enable_pgbouncer: true +postgresql_use_native_packages: false +pgsql_version: 15 +``` + + Ensure that locale settings for the old and new database server are identical, and that basic performance + tuning has been configured (e.g. setting `effective_cache_size`). + +2. Verify that locale of the old and new database are identical, using the `\l` command in `psql`. + +3. Disable iRODS on the consumer (`sudo systemctl stop irods`), disable DavRODS and the portal on the portal/DavRODS/combi servers + (`sudo systemctl stop httpd`) and disable iRODS on the provider (`sudo systemctl stop irods`). + +4. Dump the present ICAT database on the old database server (you may want to do this in a tmux session on large environments). For example: + +``` +sudo -iu postgres pg_dump ICAT | gzip | sudo tee /var/dbbackup/icat-migration.sql.gz > /dev/null +``` + +5. Transfer the dump file to the new database server. + +6. Load the dump file on the new database server: + +``` +sudo gunzip -c /var/dbbackup/icat-migration.sql.gz | sudo -iu postgres psql ICAT +``` + +7. Disable the old database: + +``` +sudo systemctl stop postgresql +sudo systemctl disable postgresql +``` + +8. Move the new database configuration to the `group_vars`, and remove the old database server from the database group. It is + generally also recommended to enable Yoda-specific database indexes at this point (`irods_database_enable_yoda_indexes: true`), + unless there is a specific reason not to use them on the environment. Adjust `irods_database_fqdn` to point to the new + database server. + +9. Adjust the `/etc/irods/server_config.json` Postgres database plugin configuration by setting `db_host` and `db_port` to the values + for the new database server. The port is typically 6432 if PgBouncer is enabled, otherwise 5432. + +10. Restart iRODS. + +11. Run the Ansible playbook. You should now have an environment where the new database server has been deployed, iRODS should now talk + to the new database server. Verify that basic functionality of iRODS works (e.g. by examining the output of the `ils` command) and + check that iRODS is talking to the new database server (e.g. by viewing the number of transactions in the pgbouncer log file on the + new database server). Disregard any PostgreSQL warning messages in the logs at this step. + +12. Check the database plugin configuration in the `/etc/irods/server_config.json` configuration file. Ensure that `db_odbc_driver` is + set to `"PostgreSQL"`. If the database plugin still has a `db_odbc_type` parameter, remove it. Restart iRODS if any manual changes + were needed in this step. + +13. Verify that Yoda works as expected. + +14. Remember to update database backup and monitoring scripts so that they point to the new database. diff --git a/docs/release-notes/release-1.9.md b/docs/release-notes/release-1.9.md index a057e2fd5..4c5ccda97 100644 --- a/docs/release-notes/release-1.9.md +++ b/docs/release-notes/release-1.9.md @@ -85,8 +85,8 @@ metadata_schemas: Please see the [configuration guide](../administration/configuring-yoda.md) for more information. -8. Set `postgresql_perform_db_upgrade` to `true` in the configuration to perform the database upgrade from Postgresql 9 to 15. -Optionally set `postgresql_remove_old_data_after_upgrade` to `true` in the configuration to clean up PostgreSQL 9 data and shim after the upgrade. +8. Unless your Yoda environment has already been upgraded to PostgreSQL 15, you should upgrade PostgreSQL during or immediately after the Yoda upgrade. Please consult [the PostgreSQL upgrade information page](../administration/upgrading-postgresql.md) for information about how to perform the upgrade. Example configuration: + ```yaml postgresql_perform_db_upgrade: true postgresql_remove_old_data_after_upgrade: false