Skip to content

Commit

Permalink
resource/aws_rds_cluster: Consolidate CreateDBCluster logic and preve…
Browse files Browse the repository at this point in the history 
…nt creation issue when global_cluster_identifier and replication_source_identifier are both configured (#14490)

Reference: #13715

After adding new acceptance testing with previous resource logic:

```
    TestAccAWSRDSCluster_GlobalClusterIdentifier_ReplicationSourceIdentifier: testing.go:684: Step 0 error: errors during apply:

        Error: error creating RDS cluster: InvalidDBClusterStateFault: Source cluster arn:aws:rds:us-west-2:--OMITTED--:cluster:tf-acc-test-728428284997379009-primary doesn't have binlogs enabled.
        	status code: 400, request id: 36e4f744-9080-4a6c-adca-fb2fc660d66e
```

After consolidating `CreateDBCluster` logic (allowing both `global_cluster_identifier` and `replication_source_identifier` to be set in the same call):

```
    TestAccAWSRDSCluster_GlobalClusterIdentifier_ReplicationSourceIdentifier: testing.go:684: Step 0 error: errors during apply:

        Error: error creating RDS cluster: InvalidParameterCombination: Value for replicationSourceIdentifier should not be specified for db cluster that is a member of global cluster
        	status code: 400, request id: f8558f28-14d1-49b3-9d94-1a607b1b689d
```

Opt to conditionalize the creation handling for this situation rather than return an error for the conflicting arguments since the existing configuration may be prevalent and the end result is the same. Document `ignore_changes`.

Output from acceptance testing (omitting failures from #14384):

```
--- PASS: TestAccAWSRDSCluster_AvailabilityZones (138.84s)
--- PASS: TestAccAWSRDSCluster_BacktrackWindow (166.46s)
--- PASS: TestAccAWSRDSCluster_backupsUpdate (161.00s)
--- PASS: TestAccAWSRDSCluster_basic (143.12s)
--- PASS: TestAccAWSRDSCluster_ClusterIdentifierPrefix (137.99s)
--- PASS: TestAccAWSRDSCluster_copyTagsToSnapshot (205.95s)
--- PASS: TestAccAWSRDSCluster_DbSubnetGroupName (159.06s)
--- PASS: TestAccAWSRDSCluster_DeletionProtection (160.99s)
--- PASS: TestAccAWSRDSCluster_EnabledCloudwatchLogsExports (341.44s)
--- PASS: TestAccAWSRDSCluster_EnableHttpEndpoint (356.65s)
--- PASS: TestAccAWSRDSCluster_encrypted (121.15s)
--- PASS: TestAccAWSRDSCluster_EngineMode (432.72s)
--- PASS: TestAccAWSRDSCluster_EngineMode_Global (139.87s)
--- PASS: TestAccAWSRDSCluster_EngineMode_Multimaster (139.86s)
--- PASS: TestAccAWSRDSCluster_EngineMode_ParallelQuery (137.74s)
--- PASS: TestAccAWSRDSCluster_EngineVersion (425.30s)
--- PASS: TestAccAWSRDSCluster_EngineVersionWithPrimaryInstance (1107.25s)
--- PASS: TestAccAWSRDSCluster_generatedName (126.84s)
--- PASS: TestAccAWSRDSCluster_GlobalClusterIdentifier_EngineMode_Global (189.88s)
--- PASS: TestAccAWSRDSCluster_GlobalClusterIdentifier_EngineMode_Global_Add (163.70s)
--- PASS: TestAccAWSRDSCluster_GlobalClusterIdentifier_EngineMode_Global_Remove (162.57s)
--- PASS: TestAccAWSRDSCluster_GlobalClusterIdentifier_EngineMode_Global_Update (172.66s)
--- PASS: TestAccAWSRDSCluster_GlobalClusterIdentifier_EngineMode_Provisioned (157.23s)
--- PASS: TestAccAWSRDSCluster_GlobalClusterIdentifier_PrimarySecondaryClusters (1768.71s)
--- PASS: TestAccAWSRDSCluster_GlobalClusterIdentifier_ReplicationSourceIdentifier (1747.31s)
--- PASS: TestAccAWSRDSCluster_iamAuth (127.32s)
--- PASS: TestAccAWSRDSCluster_kmsKey (161.41s)
--- PASS: TestAccAWSRDSCluster_missingUserNameCausesError (4.87s)
--- PASS: TestAccAWSRDSCluster_Port (253.12s)
--- PASS: TestAccAWSRDSCluster_ScalingConfiguration (386.00s)
--- PASS: TestAccAWSRDSCluster_ScalingConfiguration_DefaultMinCapacity (379.58s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier (371.73s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_DeletionProtection (409.17s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_EncryptedRestore (358.98s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_EngineMode_ParallelQuery (439.76s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_EngineMode_Provisioned (333.04s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_EngineVersion_Different (359.99s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_EngineVersion_Equal (337.24s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_MasterPassword (347.53s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_MasterUsername (381.60s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_PreferredBackupWindow (379.98s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_PreferredMaintenanceWindow (363.89s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_Tags (381.05s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_VpcSecurityGroupIds (362.04s)
--- PASS: TestAccAWSRDSCluster_SnapshotIdentifier_VpcSecurityGroupIds_Tags (369.15s)
--- PASS: TestAccAWSRDSCluster_Tags (136.51s)
--- PASS: TestAccAWSRDSCluster_takeFinalSnapshot (207.97s)
--- PASS: TestAccAWSRDSCluster_updateIamRoles (180.35s)
```
  • Loading branch information
@bflad
bflad authored Aug 6, 2020
1 parent e803cac commit a217f76
Show file tree
Hide file tree
Showing 3 changed files with 128 additions and 93 deletions.
96 changes: 4 additions & 92 deletions aws/resource_aws_rds_cluster.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -541,98 +541,6 @@ func resourceAwsRDSClusterCreate(d *schema.ResourceData, meta interface{}) error
if err != nil {
return fmt.Errorf("Error creating RDS Cluster: %s", err)
}
} else if _, ok := d.GetOk("replication_source_identifier"); ok {
createOpts := &rds.CreateDBClusterInput{
CopyTagsToSnapshot: aws.Bool(d.Get("copy_tags_to_snapshot").(bool)),
DBClusterIdentifier: aws.String(identifier),
DeletionProtection: aws.Bool(d.Get("deletion_protection").(bool)),
Engine: aws.String(d.Get("engine").(string)),
EngineMode: aws.String(d.Get("engine_mode").(string)),
ReplicationSourceIdentifier: aws.String(d.Get("replication_source_identifier").(string)),
ScalingConfiguration: expandRdsClusterScalingConfiguration(d.Get("scaling_configuration").([]interface{})),
Tags: tags,
}

// Need to check value > 0 due to:
// InvalidParameterValue: Backtrack is not enabled for the aurora-postgresql engine.
if v, ok := d.GetOk("backtrack_window"); ok && v.(int) > 0 {
createOpts.BacktrackWindow = aws.Int64(int64(v.(int)))
}

if attr, ok := d.GetOk("port"); ok {
createOpts.Port = aws.Int64(int64(attr.(int)))
}

if attr, ok := d.GetOk("db_subnet_group_name"); ok {
createOpts.DBSubnetGroupName = aws.String(attr.(string))
}

if attr, ok := d.GetOk("db_cluster_parameter_group_name"); ok {
createOpts.DBClusterParameterGroupName = aws.String(attr.(string))
}

if attr, ok := d.GetOk("engine_version"); ok {
createOpts.EngineVersion = aws.String(attr.(string))
}

if attr := d.Get("vpc_security_group_ids").(*schema.Set); attr.Len() > 0 {
createOpts.VpcSecurityGroupIds = expandStringList(attr.List())
}

if attr := d.Get("availability_zones").(*schema.Set); attr.Len() > 0 {
createOpts.AvailabilityZones = expandStringList(attr.List())
}

if v, ok := d.GetOk("backup_retention_period"); ok {
createOpts.BackupRetentionPeriod = aws.Int64(int64(v.(int)))
}

if v, ok := d.GetOk("preferred_backup_window"); ok {
createOpts.PreferredBackupWindow = aws.String(v.(string))
}

if v, ok := d.GetOk("preferred_maintenance_window"); ok {
createOpts.PreferredMaintenanceWindow = aws.String(v.(string))
}

if attr, ok := d.GetOk("kms_key_id"); ok {
createOpts.KmsKeyId = aws.String(attr.(string))
}

if attr, ok := d.GetOk("source_region"); ok {
createOpts.SourceRegion = aws.String(attr.(string))
}

if attr, ok := d.GetOkExists("storage_encrypted"); ok {
createOpts.StorageEncrypted = aws.Bool(attr.(bool))
}

if attr, ok := d.GetOk("enabled_cloudwatch_logs_exports"); ok && len(attr.([]interface{})) > 0 {
createOpts.EnableCloudwatchLogsExports = expandStringList(attr.([]interface{}))
}

log.Printf("[DEBUG] Create RDS Cluster as read replica: %s", createOpts)
var resp *rds.CreateDBClusterOutput
err := resource.Retry(1*time.Minute, func() *resource.RetryError {
var err error
resp, err = conn.CreateDBCluster(createOpts)
if err != nil {
if isAWSErr(err, "InvalidParameterValue", "IAM role ARN value is invalid or does not include the required permissions") {
return resource.RetryableError(err)
}
return resource.NonRetryableError(err)
}
return nil
})
if isResourceTimeoutError(err) {
resp, err = conn.CreateDBCluster(createOpts)
}
if err != nil {
return fmt.Errorf("error creating RDS cluster: %s", err)
}

log.Printf("[DEBUG]: RDS Cluster create response: %s", resp)

} else if v, ok := d.GetOk("s3_import"); ok {
if _, ok := d.GetOk("master_password"); !ok {
return fmt.Errorf(`provider.aws: aws_db_instance: %s: "master_password": required field is not set`, d.Get("name").(string))
Expand Down Expand Up @@ -849,6 +757,10 @@ func resourceAwsRDSClusterCreate(d *schema.ResourceData, meta interface{}) error
createOpts.EnableCloudwatchLogsExports = expandStringList(attr.([]interface{}))
}

if attr, ok := d.GetOk("replication_source_identifier"); ok && createOpts.GlobalClusterIdentifier == nil {
createOpts.ReplicationSourceIdentifier = aws.String(attr.(string))
}

if attr, ok := d.GetOkExists("storage_encrypted"); ok {
createOpts.StorageEncrypted = aws.Bool(attr.(bool))
}
Expand Down
123 changes: 123 additions & 0 deletions aws/resource_aws_rds_cluster_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1291,6 +1291,35 @@ func TestAccAWSRDSCluster_GlobalClusterIdentifier_PrimarySecondaryClusters(t *te
})
}

// Reference: https://github.com/terraform-providers/terraform-provider-aws/issues/13715
func TestAccAWSRDSCluster_GlobalClusterIdentifier_ReplicationSourceIdentifier(t *testing.T) {
var providers []*schema.Provider
var primaryDbCluster, secondaryDbCluster rds.DBCluster

rName := acctest.RandomWithPrefix("tf-acc-test")
resourceNamePrimary := "aws_rds_cluster.primary"
resourceNameSecondary := "aws_rds_cluster.secondary"

resource.ParallelTest(t, resource.TestCase{
PreCheck: func() {
testAccPreCheck(t)
testAccMultipleRegionPreCheck(t, 2)
testAccPreCheckAWSRdsGlobalCluster(t)
},
ProviderFactories: testAccProviderFactories(&providers),
CheckDestroy: testAccCheckAWSClusterDestroy,
Steps: []resource.TestStep{
{
Config: testAccAWSRDSClusterConfig_GlobalClusterIdentifier_ReplicationSourceIdentifier(rName),
Check: resource.ComposeTestCheckFunc(
testAccCheckAWSClusterExistsWithProvider(resourceNamePrimary, &primaryDbCluster, testAccAwsRegionProviderFunc(testAccGetRegion(), &providers)),
testAccCheckAWSClusterExistsWithProvider(resourceNameSecondary, &secondaryDbCluster, testAccAwsRegionProviderFunc(testAccGetAlternateRegion(), &providers)),
),
},
},
})
}

func TestAccAWSRDSCluster_Port(t *testing.T) {
var dbCluster1, dbCluster2 rds.DBCluster
rInt := acctest.RandInt()
Expand Down Expand Up @@ -3260,6 +3289,100 @@ resource "aws_rds_cluster_instance" "secondary" {
`, rNameGlobal, rNamePrimary, rNameSecondary))
}

func testAccAWSRDSClusterConfig_GlobalClusterIdentifier_ReplicationSourceIdentifier(rName string) string {
return composeConfig(
testAccMultipleRegionProviderConfig(2),
fmt.Sprintf(`
data "aws_region" "current" {}
data "aws_availability_zones" "alternate" {
provider = "awsalternate"
state = "available"
filter {
name = "opt-in-status"
values = ["opt-in-not-required"]
}
}
resource "aws_rds_global_cluster" "test" {
global_cluster_identifier = %[1]q
engine = "aurora-postgresql"
engine_version = "10.11"
}
resource "aws_rds_cluster" "primary" {
cluster_identifier = "%[1]s-primary"
database_name = "mydb"
engine = aws_rds_global_cluster.test.engine
engine_version = aws_rds_global_cluster.test.engine_version
global_cluster_identifier = aws_rds_global_cluster.test.id
master_password = "barbarbar"
master_username = "foo"
skip_final_snapshot = true
}
resource "aws_rds_cluster_instance" "primary" {
cluster_identifier = aws_rds_cluster.primary.id
engine = aws_rds_cluster.primary.engine
engine_version = aws_rds_cluster.primary.engine_version
identifier = "%[1]s-primary"
instance_class = "db.r4.large" # only db.r4 or db.r5 are valid for Aurora global db
}
resource "aws_vpc" "alternate" {
provider = "awsalternate"
cidr_block = "10.0.0.0/16"
tags = {
Name = %[1]q
}
}
resource "aws_subnet" "alternate" {
provider = "awsalternate"
count = 3
vpc_id = aws_vpc.alternate.id
availability_zone = data.aws_availability_zones.alternate.names[count.index]
cidr_block = "10.0.${count.index}.0/24"
tags = {
Name = %[1]q
}
}
resource "aws_db_subnet_group" "alternate" {
provider = "awsalternate"
name = "%[1]s"
subnet_ids = aws_subnet.alternate[*].id
}
resource "aws_rds_cluster" "secondary" {
provider = "awsalternate"
depends_on = [aws_rds_cluster_instance.primary]
cluster_identifier = "%[1]s-secondary"
db_subnet_group_name = aws_db_subnet_group.alternate.name
engine = aws_rds_global_cluster.test.engine
engine_version = aws_rds_global_cluster.test.engine_version
global_cluster_identifier = aws_rds_global_cluster.test.id
replication_source_identifier = aws_rds_cluster.primary.arn
skip_final_snapshot = true
source_region = data.aws_region.current.name
}
resource "aws_rds_cluster_instance" "secondary" {
provider = "awsalternate"
cluster_identifier = aws_rds_cluster.secondary.id
engine = aws_rds_cluster.secondary.engine
engine_version = aws_rds_cluster.secondary.engine_version
identifier = "%[1]s-secondary"
instance_class = aws_rds_cluster_instance.primary.instance_class
}
`, rName))
}

func testAccAWSRDSClusterConfig_ScalingConfiguration(rName string, autoPause bool, maxCapacity, minCapacity, secondsUntilAutoPause int, timeoutAction string) string {
return fmt.Sprintf(`
resource "aws_rds_cluster" "test" {
Expand Down
2 changes: 1 addition & 1 deletion website/docs/r/rds_cluster.html.markdown
View file
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ The following arguments are supported:
* `port` - (Optional) The port on which the DB accepts connections
* `preferred_backup_window` - (Optional) The daily time range during which automated backups are created if automated backups are enabled using the BackupRetentionPeriod parameter.Time in UTC. Default: A 30-minute window selected at random from an 8-hour block of time per region. e.g. 04:00-09:00
* `preferred_maintenance_window` - (Optional) The weekly time range during which system maintenance can occur, in (UTC) e.g. wed:04:00-wed:04:30
* `replication_source_identifier` - (Optional) ARN of a source DB cluster or DB instance if this DB cluster is to be created as a Read Replica.
* `replication_source_identifier` - (Optional) ARN of a source DB cluster or DB instance if this DB cluster is to be created as a Read Replica. If DB Cluster is part of a Global Cluster, use the [`lifecycle` configuration block `ignore_changes` argument](/docs/configuration/resources.html#ignore_changes) to prevent Terraform from showing differences for this argument instead of configuring this value.
* `scaling_configuration` - (Optional) Nested attribute with scaling properties. Only valid when `engine_mode` is set to `serverless`. More details below.
* `skip_final_snapshot` - (Optional) Determines whether a final DB snapshot is created before the DB cluster is deleted. If true is specified, no DB snapshot is created. If false is specified, a DB snapshot is created before the DB cluster is deleted, using the value from `final_snapshot_identifier`. Default is `false`.
* `snapshot_identifier` - (Optional) Specifies whether or not to create this cluster from a snapshot. You can use either the name or ARN when specifying a DB cluster snapshot, or the ARN when specifying a DB snapshot.
Expand Down

0 comments on commit a217f76

Please sign in to comment.