Allow RDS Cluster / Cluster instance to specify the engine #1591

miromad · 2017-09-05T16:23:07Z

Making sure terraform supports Aurora PostgreSQL now that we're getting close to its GA.

allows explicitly setting one of two supported engine types
aurora (default) or aurora-postgresql
adds optional engine_version which may be useful for pinning specific
postgres engine version
maintains backward compatibility with existing aurora rds clusters
with combinaton of defaults and optional parameters
used and tested under "Aurora PostgreSQL Preview" authorized aws account in us-east-1 region

github.com/terraform-providers/terraform-provider-aws$ TF_ACC=1 go test -v github.com/terraform-providers/terraform-provider-aws/aws -timeout=7200s -run "TestAccAWSRDSClusterInstance_.*"
=== RUN   TestAccAWSRDSClusterInstance_importBasic
--- PASS: TestAccAWSRDSClusterInstance_importBasic (770.69s)
=== RUN   TestAccAWSRDSClusterInstance_basic
--- PASS: TestAccAWSRDSClusterInstance_basic (1440.30s)
=== RUN   TestAccAWSRDSClusterInstance_namePrefix
--- PASS: TestAccAWSRDSClusterInstance_namePrefix (764.54s)
=== RUN   TestAccAWSRDSClusterInstance_generatedName
--- PASS: TestAccAWSRDSClusterInstance_generatedName (720.21s)
=== RUN   TestAccAWSRDSClusterInstance_kmsKey
--- PASS: TestAccAWSRDSClusterInstance_kmsKey (765.96s)
=== RUN   TestAccAWSRDSClusterInstance_disappears
--- PASS: TestAccAWSRDSClusterInstance_disappears (722.53s)
=== RUN   TestAccAWSRDSClusterInstance_withInstanceEnhancedMonitor
--- PASS: TestAccAWSRDSClusterInstance_withInstanceEnhancedMonitor (735.03s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	5919.258s

github.com/terraform-providers/terraform-provider-aws$ TF_ACC=1 go test -v github.com/terraform-providers/terraform-provider-aws/aws -timeout=7200s -run "TestAccAWSRDSCluster_.*"
=== RUN   TestAccAWSRDSCluster_importBasic
--- PASS: TestAccAWSRDSCluster_importBasic (155.95s)
=== RUN   TestAccAWSRDSCluster_basic
--- PASS: TestAccAWSRDSCluster_basic (119.60s)
=== RUN   TestAccAWSRDSCluster_namePrefix
--- PASS: TestAccAWSRDSCluster_namePrefix (185.24s)
=== RUN   TestAccAWSRDSCluster_generatedName
--- PASS: TestAccAWSRDSCluster_generatedName (196.65s)
=== RUN   TestAccAWSRDSCluster_takeFinalSnapshot
--- PASS: TestAccAWSRDSCluster_takeFinalSnapshot (197.50s)
=== RUN   TestAccAWSRDSCluster_missingUserNameCausesError
--- PASS: TestAccAWSRDSCluster_missingUserNameCausesError (9.30s)
=== RUN   TestAccAWSRDSCluster_updateTags
--- PASS: TestAccAWSRDSCluster_updateTags (175.84s)
=== RUN   TestAccAWSRDSCluster_updateIamRoles
--- PASS: TestAccAWSRDSCluster_updateIamRoles (189.80s)
=== RUN   TestAccAWSRDSCluster_kmsKey
--- PASS: TestAccAWSRDSCluster_kmsKey (195.33s)
=== RUN   TestAccAWSRDSCluster_encrypted
--- PASS: TestAccAWSRDSCluster_encrypted (152.58s)
=== RUN   TestAccAWSRDSCluster_backupsUpdate
--- PASS: TestAccAWSRDSCluster_backupsUpdate (165.99s)
=== RUN   TestAccAWSRDSCluster_iamAuth
--- PASS: TestAccAWSRDSCluster_iamAuth (141.66s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	1885.459s

- allows explicitly setting one of two supported engine types aurora (default) or aurora-postgresql - adds optional engine_version which may be useful for pinning specific postgres engine version - maintains backward compatibility with existing aurora rds clusters with combinaton of defaults and optional parameters - used and tested under "Aurora PostgreSQL Preview" authorized aws account in us-east-1 region ``` github.com/terraform-providers/terraform-provider-aws$ TF_ACC=1 go test -v github.com/terraform-providers/terraform-provider-aws/aws -timeout=7200s -run "TestAccAWSRDSClusterInstance_.*" === RUN TestAccAWSRDSClusterInstance_importBasic --- PASS: TestAccAWSRDSClusterInstance_importBasic (770.69s) === RUN TestAccAWSRDSClusterInstance_basic --- PASS: TestAccAWSRDSClusterInstance_basic (1440.30s) === RUN TestAccAWSRDSClusterInstance_namePrefix --- PASS: TestAccAWSRDSClusterInstance_namePrefix (764.54s) === RUN TestAccAWSRDSClusterInstance_generatedName --- PASS: TestAccAWSRDSClusterInstance_generatedName (720.21s) === RUN TestAccAWSRDSClusterInstance_kmsKey --- PASS: TestAccAWSRDSClusterInstance_kmsKey (765.96s) === RUN TestAccAWSRDSClusterInstance_disappears --- PASS: TestAccAWSRDSClusterInstance_disappears (722.53s) === RUN TestAccAWSRDSClusterInstance_withInstanceEnhancedMonitor --- PASS: TestAccAWSRDSClusterInstance_withInstanceEnhancedMonitor (735.03s) PASS ok github.com/terraform-providers/terraform-provider-aws/aws 5919.258s github.com/terraform-providers/terraform-provider-aws$ TF_ACC=1 go test -v github.com/terraform-providers/terraform-provider-aws/aws -timeout=7200s -run "TestAccAWSRDSCluster_.*" === RUN TestAccAWSRDSCluster_importBasic --- PASS: TestAccAWSRDSCluster_importBasic (155.95s) === RUN TestAccAWSRDSCluster_basic --- PASS: TestAccAWSRDSCluster_basic (119.60s) === RUN TestAccAWSRDSCluster_namePrefix --- PASS: TestAccAWSRDSCluster_namePrefix (185.24s) === RUN TestAccAWSRDSCluster_generatedName --- PASS: TestAccAWSRDSCluster_generatedName (196.65s) === RUN TestAccAWSRDSCluster_takeFinalSnapshot --- PASS: TestAccAWSRDSCluster_takeFinalSnapshot (197.50s) === RUN TestAccAWSRDSCluster_missingUserNameCausesError --- PASS: TestAccAWSRDSCluster_missingUserNameCausesError (9.30s) === RUN TestAccAWSRDSCluster_updateTags --- PASS: TestAccAWSRDSCluster_updateTags (175.84s) === RUN TestAccAWSRDSCluster_updateIamRoles --- PASS: TestAccAWSRDSCluster_updateIamRoles (189.80s) === RUN TestAccAWSRDSCluster_kmsKey --- PASS: TestAccAWSRDSCluster_kmsKey (195.33s) === RUN TestAccAWSRDSCluster_encrypted --- PASS: TestAccAWSRDSCluster_encrypted (152.58s) === RUN TestAccAWSRDSCluster_backupsUpdate --- PASS: TestAccAWSRDSCluster_backupsUpdate (165.99s) === RUN TestAccAWSRDSCluster_iamAuth --- PASS: TestAccAWSRDSCluster_iamAuth (141.66s) PASS ok github.com/terraform-providers/terraform-provider-aws/aws 1885.459s ```

jcomeaux · 2017-09-09T16:05:51Z

Hi @miromad .... I believe a very similar effort has already been merged? #1415

Is the engine selection not working (e.g. setting engine = aurora-postgresql)

***also probably relevant: aws/aws-sdk-go#1470

miromad · 2017-09-11T11:21:03Z

Hi @jcomeaux.
My changes are merged with #1415. It added engine as an option to instance but not to cluster where it was left hardcoded. I also added type validation to make it more user friendly. Regarding #1470, since I tested this under preview account and everything worked no changes are needed in SDK to support it.

Ninir

Hi @miromad

Thanks for the work here! 👍

While this looks good, just left a few comments to discuss :)

We should also update the documentation, for instance to expose that it defaults to aurora.

Ninir · 2017-09-11T19:34:02Z

aws/resource_aws_rds_cluster.go

+				Optional:     true,
+				Default:      "aurora",
+				ForceNew:     true,
+				ValidateFunc: validateRdsEngine,


I don't think we should validate the engine value, as we would need to update the code each time a new engine is available. Thoughts?

Valid point however, amazon introduced aurora for MySQL in 2014 and added PostgreSQL in 2017. Don't have insights into their product plans but so far if they keep adding one every 3 years updating code won't be an issue. I think having validation is more useful/user friendly than flexibility.

Looks good to me :)

Ninir · 2017-09-11T19:37:27Z

aws/resource_aws_rds_cluster.go

+				ValidateFunc: validateRdsEngine,
+			},
+
+			"engine_version": {


We should also read the engine version around line 575

right, missed that one.

Ninir · 2017-09-11T19:37:45Z

aws/resource_aws_rds_cluster_instance.go

@@ -292,6 +311,7 @@ func resourceAwsRDSClusterInstanceRead(d *schema.ResourceData, meta interface{})

 	d.Set("publicly_accessible", db.PubliclyAccessible)
 	d.Set("cluster_identifier", db.DBClusterIdentifier)
+	d.Set("engine", db.Engine)


We should also read the engine_version here

will fix it too

miromad · 2017-09-15T22:19:50Z

Hi @Ninir,
Thanks for feedback. Will make changes (updating doc and engine_version sets in 2 places you identified) over the weekend. Let me know if anything else is needed.

Ninir

Hey @miromad

This looks very good to me!
I just added attributes validation in the tests part:

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSRDSCluster_basic'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -run=TestAccAWSRDSCluster_basic -timeout 120m
=== RUN   TestAccAWSRDSCluster_basic
--- PASS: TestAccAWSRDSCluster_basic (119.98s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	120.019s

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSRDSClusterInstance_basic'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -run=TestAccAWSRDSClusterInstance_basic -timeout 120m
=== RUN   TestAccAWSRDSClusterInstance_basic
--- PASS: TestAccAWSRDSClusterInstance_basic (1408.46s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	1408.507s

Thank you for the work!

Aurora for postgres RDS engine support

ghost · 2017-09-27T17:31:35Z

Hi @Ninir ,so does terraform supports the cluster creation in aurora postgresql

Ninir · 2017-09-27T17:46:48Z

Hi @phanindra2407

It is still not globally available, so we all need to be more patient here! 😄

jude-pieries · 2017-10-25T17:36:40Z

Aurora-postgres is GA starting 24/10/2017 and is available in the following regions us-east-1,2 , us-west-2 and EU-west-1

Ninir · 2017-10-25T18:05:21Z

Thanks for letting us know @jude-pieries :)

Here's the link: https://aws.amazon.com/fr/blogs/aws/now-available-amazon-aurora-with-postgresql-compatibility/

atrepca · 2017-10-26T12:05:14Z

Confirming this works now, adding engine = "aurora-postgresql" does the trick.

ghost · 2020-04-10T17:48:21Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

Ninir added the enhancement Requests to existing resources that expand the functionality or scope. label Sep 6, 2017

Ninir reviewed Sep 11, 2017

View reviewed changes

Miro Madecki and others added 2 commits September 18, 2017 15:03

Making sure engine_version is read and docs are updated.

32217a0

Added attributes validation

38b8544

Ninir approved these changes Sep 21, 2017

View reviewed changes

Ninir merged commit be6ca29 into hashicorp:master Sep 21, 2017

Ninir changed the title ~~Aurora for postgres RDS engine support~~ Allow RDS Cluster / Cluster instance to specify the engine Sep 21, 2017

nbaztec pushed a commit to nbaztec/terraform-provider-aws that referenced this pull request Sep 26, 2017

Merge pull request hashicorp#1591 from Shoobx/aurora-postgres

dc6668b

Aurora for postgres RDS engine support

ppearcy mentioned this pull request Oct 26, 2017

RDS cluster instance resource doesn't work with aurora-postgres preview #1443

Closed

ghost locked and limited conversation to collaborators Apr 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow RDS Cluster / Cluster instance to specify the engine #1591

Allow RDS Cluster / Cluster instance to specify the engine #1591

miromad commented Sep 5, 2017

jcomeaux commented Sep 9, 2017

miromad commented Sep 11, 2017

Ninir left a comment

Ninir Sep 11, 2017

miromad Sep 15, 2017

Ninir Sep 19, 2017

Ninir Sep 11, 2017

miromad Sep 15, 2017

Ninir Sep 11, 2017

miromad Sep 15, 2017

miromad commented Sep 15, 2017

Ninir left a comment

ghost commented Sep 27, 2017

Ninir commented Sep 27, 2017

jude-pieries commented Oct 25, 2017

Ninir commented Oct 25, 2017

atrepca commented Oct 26, 2017

ghost commented Apr 10, 2020

Allow RDS Cluster / Cluster instance to specify the engine #1591

Allow RDS Cluster / Cluster instance to specify the engine #1591

Conversation

miromad commented Sep 5, 2017

jcomeaux commented Sep 9, 2017

miromad commented Sep 11, 2017

Ninir left a comment

Choose a reason for hiding this comment

Ninir Sep 11, 2017

Choose a reason for hiding this comment

miromad Sep 15, 2017

Choose a reason for hiding this comment

Ninir Sep 19, 2017

Choose a reason for hiding this comment

Ninir Sep 11, 2017

Choose a reason for hiding this comment

miromad Sep 15, 2017

Choose a reason for hiding this comment

Ninir Sep 11, 2017

Choose a reason for hiding this comment

miromad Sep 15, 2017

Choose a reason for hiding this comment

miromad commented Sep 15, 2017

Ninir left a comment

Choose a reason for hiding this comment

ghost commented Sep 27, 2017

Ninir commented Sep 27, 2017

jude-pieries commented Oct 25, 2017

Ninir commented Oct 25, 2017

atrepca commented Oct 26, 2017

ghost commented Apr 10, 2020