terraform-rds

Terraform modules to manage RDS resources

rds

Creates a RDS instance, security_group, subnet_group and parameter_group

Requirements

Name	Version
terraform	>= 0.12

Providers

Name	Version
aws	n/a

Modules

No modules.

Resources

Name	Type
aws_db_instance.rds	resource
aws_db_parameter_group.rds	resource
aws_db_subnet_group.rds	resource
aws_security_group.sg_rds	resource
aws_security_group_rule.rds_cidr_in	resource
aws_security_group_rule.rds_sg_in	resource

Inputs

Name	Description	Type	Default	Required
rds_password	RDS root password	any	n/a	yes
security_groups	Security groups that are allowed to access the RDS	list(string)	n/a	yes
security_groups_count	Number of security groups provided in security_groups variable	any	n/a	yes
subnets	Subnets to deploy in	list(string)	n/a	yes
vpc_id	ID of the VPC where to deploy in	any	n/a	yes
allowed_cidr_blocks	CIDR blocks that are allowed to access the RDS	list(string)	[]	no
apply_immediately	Apply changes immediately	bool	true	no
auto_minor_version_upgrade	Indicates that minor engine upgrades will be applied automatically to the DB instance during the maintenance window.	bool	true	no
availability_zone	The availability zone where you want to launch your instance in	string	""	no
backup_retention_period	How long do you want to keep RDS backups	string	"14"	no
default_parameter_group_family	Parameter group family for the default parameter group, according to the chosen engine and engine version. Defaults to mysql5.7	string	"mysql5.7"	no
deletion_protection	If the DB instance should have deletion protection enabled. The database can't be deleted when this value is set to true	bool	false	no
enabled_cloudwatch_logs_exports	List of log types to enable for exporting to CloudWatch logs. You can check the available log types per engine in the AWS RDS documentation.	list(string)	[]	no
engine	RDS engine: mysql, oracle, postgres. Defaults to mysql	string	"mysql"	no
engine_version	Engine version to use, according to the chosen engine. You can check the available engine versions using the AWS CLI. Defaults to 5.7.17 for MySQL.	string	"5.7.25"	no
environment	How do you want to call your environment, this is helpful if you have more than 1 VPC.	string	"production"	no
extra_tags	A mapping of extra tags to assign to the resource	map(string)	{}	no
maintenance_window	The window to perform maintenance in. Syntax: "ddd:hh24:mi-ddd:hh24:mi". Eg: "Mon:00:00-Mon:03:00"	string	null	no
max_allocated_storage	When configured, the upper limit to which Amazon RDS can automatically scale the storage of the DB instance. Configuring this will automatically ignore differences to allocated_storage. Must be greater than or equal to allocated_storage or 0 to disable Storage Autoscaling.	string	"0"	no
monitoring_interval	The interval, in seconds, between points when Enhanced Monitoring metrics are collected for the DB instance. To disable collecting Enhanced Monitoring metrics, specify 0. Valid Values: 0, 1, 5, 10, 15, 30, 60.	string	"0"	no
multi_az	Multi AZ true or false	bool	true	no
name	The name of the RDS instance	string	""	no
number	number of the database default 01	string	"01"	no
performance_insights_enabled	Specifies whether Performance Insights is enabled or not.	bool	false	no
performance_insights_kms_key_id	Custom KMS key to use to encrypt the performance insights data	string	null	no
performance_insights_retention_period	Amount of time in days to retain Performance Insights data. Valid values are 7, 731 (2 years) or a multiple of 31. When specifying performance_insights_retention_period	number	7	no
project	The current project	string	""	no
publicly_accessible	Bool to control if instance is publicly accessible	bool	false	no
rds_custom_parameter_group_name	A custom parameter group name to attach to the RDS instance. If not provided a default one will be used	string	""	no
rds_username	RDS root user	string	"root"	no
size	Instance size	string	"db.t2.small"	no
skip_final_snapshot	Skip final snapshot when destroying RDS	bool	false	no
snapshot_identifier	Specifies whether or not to create this database from a snapshot. This correlates to the snapshot ID you'd find in the RDS console, e.g: rds:production-2015-06-26-06-05.	string	""	no
storage	How many GBs of space does your database need?	string	"10"	no
storage_encrypted	Encrypt RDS storage	bool	true	no
storage_kms_key_id	Custom KMS key to use to encrypt the storage. Will use the AWS key if left null (default)	string	null	no
storage_type	Type of storage you want to use	string	"gp2"	no
subnet_group_name_override	Override the name of the created subnet group	string	null	no
tag	A tag used to identify an RDS in a project that has more than one RDS	string	""	no

Outputs

Name	Description
aws_db_subnet_group_id	The subnet group id of the RDS instance
rds_address	The hostname of the RDS instance
rds_arn	The arn of the RDS instance
rds_id	The id of the RDS instance
rds_identifier	The identifier of the RDS instance
rds_port	The port of the RDS instance
rds_sg_id	The security group id of the RDS instance

Example

module "rds" {
  source                          = "github.com/skyscrapers/terraform-rds//rds"
  vpc_id                          = "vpc-e123bc45"
  subnets                         = ["subnet-12345d67", "subnet-12345d68", "subnet-12345d69"]
  project                         = "myproject"
  environment                     = "production"
  size                            = "db.t2.small"
  security_groups                 = ["sg-12be345678905ebf1", "sg-1234567890aef"]
  enabled_cloudwatch_logs_exports = ["audit", "error", "slowquery"]
  security_groups_count           = 2
  rds_password                    = "supersecurepassword"
  multi_az                        = "false"
}

Aurora

Creates a Aurora cluster + instances, security_group, subnet_group and parameter_group

Requirements

Name	Version
terraform	>= 0.12

Providers

Name	Version
aws	n/a

Modules

No modules.

Resources

Name	Type
aws_db_parameter_group.aurora_mysql	resource
aws_db_subnet_group.aurora	resource
aws_rds_cluster.aurora	resource
aws_rds_cluster_instance.cluster_instances	resource
aws_security_group.sg_aurora	resource
aws_security_group_rule.sg_aurora_in	resource
aws_availability_zones.available	data source
aws_subnet.subnet	data source

Inputs

Name	Description	Type	Default	Required
password	RDS root password	any	n/a	yes
security_groups	Security groups that are allowed to access the RDS on port 3306	list(string)	n/a	yes
subnets	Subnets to deploy in	list(string)	n/a	yes
amount_of_instances	The amount of Aurora instances you need, for HA you need minumum 2	number	1	no
apply_immediately	Apply changes immediately	bool	true	no
backup_retention_period	How long do you want to keep RDS backups	string	"14"	no
cluster_parameter_group_name	Optional parameter group you can set for the RDS Aurora cluster	string	""	no
default_ports	n/a	map	{ "aurora": "3306", "aurora-postgresql": "5432" }	no
enabled_cloudwatch_logs_exports	List of log types to enable for exporting to CloudWatch logs. You can check the available log types per engine in the AWS Aurora documentation.	list(string)	[]	no
engine	Optional parameter to set the Aurora engine	string	"aurora"	no
engine_version	Optional parameter to set the Aurora engine version	string	"5.6.10a"	no
environment	How do you want to call your environment, this is helpful if you have more than 1 VPC.	string	"production"	no
extra_tags	A mapping of extra tags to assign to the resource	map(string)	{}	no
family	n/a	string	"aurora5.6"	no
instance_parameter_group_name	Optional parameter group you can set for the RDS instances inside an Aurora cluster	string	""	no
instance_promotion_tiers	Set promotion tier for each instance in the cluster. The size of the list must be equal to var.amount_of_instances. If ommitted or set to [], the default of 0 will be used.	list(number)	[]	no
instance_size_override	Provide different instance sizes for each individual aurora instance in the cluster. The size of the list must be equal to var.amount_of_instances. If ommitted or set to [], this module will use var.size for all the instances in the cluster.	list(string)	[]	no
performance_insights_enabled	Specifies whether Performance Insights is enabled or not.	bool	false	no
project	The current project	string	""	no
rds_instance_name_overrides	List of names to override the default RDS instance names / identifiers.	list(string)	null	no
rds_username	RDS root user	string	"root"	no
size	Instance size	string	"db.t2.small"	no
skip_final_snapshot	Skip final snapshot when destroying RDS	bool	false	no
snapshot_identifier	Specifies whether or not to create this cluster from a snapshot. You can use either the name or ARN when specifying a DB cluster snapshot, or the ARN when specifying a DB snapshot	string	null	no
storage_encrypted	Encrypt RDS storage	bool	true	no
tag	A tag used to identify an RDS in a project that has more than one RDS	string	""	no

Outputs

Name	Description
aurora_cluster_id	n/a
aurora_cluster_instances_id	n/a
aurora_port	n/a
aurora_sg_id	n/a
endpoint	n/a
reader_endpoint	n/a

Example

module "aurora" {
  source                          = "github.com/skyscrapers/terraform-rds//aurora"
  project                         = "myproject"
  environment                     = "production"
  size                            = "db.t2.small"
  password                        = "supersecurepassword"
  subnets                         = ["subnet-12345d67", "subnet-12345d68", "subnet-12345d69"]
  amount_of_instances             = 1
  security_groups                 = ["sg-12be345678905ebf1", "sg-1234567890aef"]
  enabled_cloudwatch_logs_exports = ["audit", "error", "slowquery"]
}

rds-replica

Creates an RDS read replica instance, the replica security_group and a subnet_group if not passed as parameter

Inputs

Name	Description	Type	Default	Required
engine		string	n/a	yes
subnets	Subnets to deploy in	list(string)	n/a	yes
vpc_id	ID of the VPC where to deploy in	string	n/a	yes
allowed_cidr_blocks	CIDR blocks that are allowed to access the RDS	list(string)	[]	no
auto_minor_version_upgrade	Indicates that minor engine upgrades will be applied automatically to the DB instance during the maintenance window.	string	"true"	no
backup_retention_period	How long do you want to keep RDS Slave backups	number	"14"	no
custom_parameter_group_name	A custom parameter group name to attach to the RDS instance. If not provided it will use the default from the master instance	string	""	no
enabled_cloudwatch_logs_exports	List of log types to enable for exporting to CloudWatch logs. You can check the available log types per engine in the [AWS RDS documentation](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER\_LogAccess.html#USER\_LogAccess.Procedural.UploadtoCloudWatch\).	list(string)	[]	no
environment	How do you want to call your environment, this is helpful if you have more than 1 VPC.	string	"production"	no
name	An optional custom name to give to the module's resources	string	""	no
number_of_replicas	number of database repliacs default 1	string	"1"	no
multi_az	Multi AZ true or false	bool	false	no
ports		map	{ "mysql": "3306", "oracle": "1521", "postgres": "5432" }	no
project	The current project	string	""	no
replicate_source_db	RDS source to replicate from	string	""	no
publicly_accessible	Bool to control if instance is publicly accessible	false	no
security_groups	Security groups that are allowed to access the RDS	list(string)	[]	no
size	Instance size	string	"db.t2.small"	no
storage_encrypted	Encrypt RDS storage	string	"true"	no
tag	A tag used to identify an RDS in a project that has more than one RDS	string	""	no
max_allocated_storage	When configured, the upper limit to which Amazon RDS can automatically scale the storage of the DB instance. Configuring this will automatically ignore differences to allocated_storage. Must be greater than or equal to allocated_storage or 0 to disable Storage Autoscaling. If not set the default of the master instance is set.	string	null	no
allocated_storage	How many GBs of space does your database need? If not set the default of the master instance is set.	string	null	no

Outputs

Name	Description
rds_address
rds_arn
rds_sg_id

Example

module "rds" {
  source              = "github.com/skyscrapers/terraform-rds//rds-replica"
  project             = "myproject"
  environment         = "production"
  size                = "db.t2.small"
  engine              = "postgres"
  security_groups     = ["sg-12be345678905ebf1", "sg-1234567890aef"]
  replicate_source_db = "arn:aws:rds:eu-west-1:123456789012:db:myproject-production-something-rds01"
  vpc_id              = "vpc-e123bc45"
  subnets             = ["subnet-12345d67", "subnet-12345d68", "subnet-12345d69"]
}

snapshot-cross-account-replicator

This module creates snapshots of RDS instances based on a configured frequency, and replicates them to a different region in a different AWS account. To achieve this it creates several Lambda functions that take care of the copy operations in the different steps.

As an example, let's say we want to back up an RDS instance in AWS account 111111111111 in region eu-west-1 to the AWS account 222222222222 in region eu-central-1. The whole replication process takes place in 4 steps:

1. A snapshot is created from the RDS instance, in the account 111111111111 in region eu-west-1 . If the instance is KMS encrypted, the snapshot will be encrypted with the same key
2. The initial snapshot is copied to region eu-central-1 within the source account 111111111111. Snapshots cannot be copied to a different AWS account and region in the same copy operation, so it needs to happen in two steps. In this step, the snapshot is re-encrypted using a KMS key in the target AWS account and region (222222222222 & eu-central-1)
3. The resulting snapshot from step (2) is then copied over to its final destination, in account 222222222222 in region eu-central-1.

There are Lambda functions in place that will take care of cleaning up the initial and intermediate snapshots resulting from steps (1) and (2).

There's another Lambda function running in account 222222222222 in region eu-central-1 that will periodically run and delete those snapshots that are older than the configured retention period.

For monitoring, the module creates two SNS topics where CloudWatch will post alert messages in case there's problems running the Lambda functions. These SNS topics can be subscribed to upstream monitoring services like OpsGenie.

Take into account that for the copy operation and re-encryption process to work properly, the policy of the provided KMS key in the target account needs to allow usage access to the root user of the source account. IAM policies to further grant access to the Lambda functions will be created within the module. Check this AWS documentation page to know more about how encrpyted snapshots can be shared between different accounts.

Example

data "aws_caller_identity" "source" {
  provider = aws.source
}

module "rds_replication" {
  source = "github.com/skyscrapers/terraform-rds//snapshot-cross-account-replicator?ref=6.1.0"

  name                      = "AuroraReplicator"
  is_aurora_cluster         = true
  rds_instance_ids          = var.rds_cluster_ids
  snapshot_schedule_period  = 12
  retention_period          = 4
  target_account_kms_key_id = aws_kms_key.rds_target.id

  providers = {
    aws.source       = aws.source
    aws.intermediate = aws.intermediate
    aws.target       = aws.target
  }
}

resource "aws_kms_key" "rds_target" {
  provider = aws.target

  description = "KMS key used to encrypt RDS"
  policy      = data.aws_iam_policy_document.rds_replication_key.json

  lifecycle {
    prevent_destroy = true
  }
}

data "aws_iam_policy_document" "rds_replication_key" {
  provider = aws.target

  statement {
    sid       = "Enable IAM policies in source & target accounts"
    effect    = "Allow"
    actions   = ["kms:*"]
    resources = ["*"]

    principals {
      type             = "AWS"
      identifiers = [
        "arn:aws:iam::${data.aws_caller_identity.source.account_id}:root",
        "arn:aws:iam::${var.target_aws_account}:root"
      ]
    }
  }
}

Requirements

Name	Version
terraform	~> 1.0
aws	~> 3.61

Providers

Name	Version
archive	n/a
aws.intermediate	~> 3.61
aws.source	~> 3.61
aws.target	~> 3.61

Modules

Name	Source	Version
cleanup_intermediate_lambda_monitoring	github.com/skyscrapers/terraform-cloudwatch//lambda_function	2.0.1
cleanup_source_lambda_monitoring	github.com/skyscrapers/terraform-cloudwatch//lambda_function	2.0.1
cleanup_target_lambda_monitoring	github.com/skyscrapers/terraform-cloudwatch//lambda_function	2.0.1
step_1_lambda_monitoring	github.com/skyscrapers/terraform-cloudwatch//lambda_function	2.0.1
step_2_lambda_monitoring	github.com/skyscrapers/terraform-cloudwatch//lambda_function	2.0.1
step_3_lambda_monitoring	github.com/skyscrapers/terraform-cloudwatch//lambda_function	2.0.1

Resources

Name	Type
aws_cloudwatch_event_rule.invoke_cleanup_intermediate_lambda	resource
aws_cloudwatch_event_rule.invoke_cleanup_source_lambda	resource
aws_cloudwatch_event_rule.invoke_cleanup_target_lambda	resource
aws_cloudwatch_event_rule.invoke_step_1_lambda	resource
aws_cloudwatch_event_rule.invoke_step_2_lambda	resource
aws_cloudwatch_event_rule.invoke_step_3_lambda	resource
aws_cloudwatch_event_target.invoke_cleanup_intermediate_lambda	resource
aws_cloudwatch_event_target.invoke_cleanup_source_lambda	resource
aws_cloudwatch_event_target.invoke_cleanup_target_lambda	resource
aws_cloudwatch_event_target.invoke_step_1_lambda	resource
aws_cloudwatch_event_target.invoke_step_2_lambda	resource
aws_cloudwatch_event_target.invoke_step_3_lambda	resource
aws_iam_role.source_lambda	resource
aws_iam_role.target_lambda	resource
aws_iam_role_policy.source_lambda	resource
aws_iam_role_policy.source_lambda_kms	resource
aws_iam_role_policy.target_lambda	resource
aws_iam_role_policy.target_lambda_kms	resource
aws_iam_role_policy_attachment.source_lambda_exec_role	resource
aws_iam_role_policy_attachment.target_lambda_exec_role	resource
aws_lambda_function.cleanup_intermediate	resource
aws_lambda_function.cleanup_source	resource
aws_lambda_function.cleanup_target	resource
aws_lambda_function.step_1	resource
aws_lambda_function.step_2	resource
aws_lambda_function.step_3	resource
aws_lambda_permission.invoke_cleanup_intermediate_lambda	resource
aws_lambda_permission.invoke_cleanup_source_lambda	resource
aws_lambda_permission.invoke_cleanup_target_lambda	resource
aws_lambda_permission.invoke_step_1_lambda	resource
aws_lambda_permission.invoke_step_2_lambda	resource
aws_lambda_permission.invoke_step_3_lambda	resource
aws_sns_topic.source_region_topic	resource
aws_sns_topic.target_region_topic	resource
aws_sns_topic_policy.source_region_topic	resource
aws_sns_topic_policy.target_region_topic	resource
archive_file.lambda_zip	data source
aws_caller_identity.source	data source
aws_caller_identity.target	data source
aws_db_instance.rds	data source
aws_iam_policy_document.lambda_kms_permissions	data source
aws_iam_policy_document.source_lambda_assume_role_policy	data source
aws_iam_policy_document.source_lambda_permissions	data source
aws_iam_policy_document.source_retion_sns_policy	data source
aws_iam_policy_document.target_lambda_assume_role_policy	data source
aws_iam_policy_document.target_lambda_permissions	data source
aws_iam_policy_document.target_retion_sns_policy	data source
aws_kms_key.target_key	data source
aws_rds_cluster.rds	data source
aws_region.intermediate	data source
aws_region.source	data source
aws_region.target	data source

Inputs

Name	Description	Type	Default	Required
name	Name of the setup	string	n/a	yes
rds_instance_ids	List of IDs of the RDS instances to back up. If using Aurora, provide the cluster IDs instead	list(string)	n/a	yes
target_account_kms_key_id	KMS key to use to encrypt replicated RDS snapshots in the target AWS account	string	n/a	yes
is_aurora_cluster	Whether we're backing up Aurora clusters instead of RDS instances	bool	false	no
lambda_monitoring_metric_period	The metric period to use for the Lambdas CloudWatch alerts for monitoring. This should be equal or higher than the snapshoting period	number	21600	no
retention_period	Snapshot retention period in days	number	14	no
snapshot_schedule_expression	Snapshot frequency specified as a CloudWatch schedule expression. Can either be a rate() or cron() expression. Check the AWS documentation on how to compose such expression.	string	"cron(0 */6 * * ? *)"	no
storage_type	Determine storage type for your Aurora RDS. Valid values for Single-AZ: aurora, "" (default, both refer to Aurora Standard), aurora-iopt1 (Aurora I/O Optimized). Valid values for Multi-AZ: io1 (default).	string	null	no

Outputs

Name	Description
source_region_sns_topic_arn	SNS topic ARN for the lambdas in the source region
target_region_sns_topic_arn	SNS topic ARN for the lambdas in the target region

rds-proxy

Create an RDS proxy and configure IAM role to use for reading AWS Secrets to access the database.

Inputs

Name	Description	Type	Default	Required
db_cluster_identifier	ID of the database instance to set as the proxy target	any	n/a	yes
db_secret_arns	AWS Secret Manager ARNs to use to access the database credentials	list	n/a	yes
engine	RDS engine: MYSQL or POSTGRES	any	n/a	yes
environment	The current environment	any	n/a	yes
project	The current project	any	n/a	yes
security_groups	Security groups that are allowed to access the RDS	list(string)	n/a	yes
subnets	Subnets to deploy in	list(string)	n/a	yes
proxy_connection_timeout	The number of seconds for a proxy to wait for a connection to become available in the connection pool	number	120	no
proxy_max_connection_percent	The maximum size of the connection pool for each target in a target group	number	100	no
idle_client_timeout	The number of seconds that a connection to the proxy can be inactive before the proxy disconnects it.	number	1800	no
debug_logging	Whether the proxy includes detailed information about SQL statements in its logs.	bool	false	no
reader_endpoint	Whether to include an extra reader endpoint.	bool	false	no

Outputs

Name	Description
proxy_endpoint	Endpoint of the created proxy

Example

module "rds_proxy" {
  source = "github.com/skyscrapers/terraform-rds//rds_proxy"
  subnets                    = data.terraform_remote_state.networking.outputs.private_db_subnets
  project                    = var.project
  environment                = terraform.workspace
  engine                     = "MYSQL"
  security_groups            = ["sg-aaaaa", "sg-bbbb"]
  db_cluster_identifier     = module.rds_database.rds_id
  db_secret_arns             = [aws_secretsmanager_secret.db_user_rw.arn, aws_secretsmanager_secret.db_user_ro.arn]
}