RDS Proxy Support

[gazoakley]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Amazon RDS Proxy is a fully managed, highly available database proxy for Amazon Relational Database Service (RDS). RDS Proxy makes applications more scalable, more resilient to database failures, and more secure.

New or Affected Resource(s)

• aws_db_proxy
• aws_db_proxy_default_target_group
• aws_db_proxy_target

Potential Terraform Configuration

resource "aws_db_proxy" "example" {
  auth {
    auth_scheme = "SECRETS"
    description = "example"
    iam_auth    = "DISABLED"
    secret_arn  = ""
    username    = "example"
  }

  db_proxy_name       = "example"
  debug_logging       = false
  engine_family       = "MYSQL"
  idle_client_timeout = 60
  require_tls         = true
  role_arn            = "aws:"

  tags = {
    Name = "example"
    Key  = "value"
  }

  vpc_security_group_ids = ["sg-12345678901234567"]
  vpc_subnet_ids         = ["subnet-12345678901234567"]
}


resource "aws_db_proxy_default_target_group" "example" {
  connection_pool_config {
    connection_borrow_timeout    = 120
    init_query                   = "SET x=1, y=2"
    max_connections_percent      = 100
    max_idle_connections_percent = 50
    session_pinning_filters      = ["SELECT"]
  }

  db_proxy_name     = aws_db_proxy.example.db_proxy_name
}

resource "aws_db_instance" "example" {
  # ...
}

resource "aws_db_proxy_target" "example" {
  # db_cluster_identifier  = ""
  db_instance_identifier = aws_db_instance.example.name
  db_proxy_name          = aws_db_proxy.example.db_proxy_name
  target_group_name      = aws_db_proxy_default_target_group.example.name
}

References

• https://aws.amazon.com/about-aws/whats-new/2019/12/amazon-rds-proxy-available-in-preview/
• https://aws.amazon.com/about-aws/whats-new/2020/04/amazon-rds-proxy-with-postgresql-compatibility-preview/

[phanimullapudi]

Hi - We are currently blocked using RDS proxy @ Adobe due to resource not available in TF. All our AWS code is TF based and we were planning to use RDS Proxy + Aurora.

[gazoakley]

@phanimullapudi Please add a 👍 reaction here: #12690 (comment)

[jrobison-sb]

This is now generally available, no longer in preview.

https://aws.amazon.com/blogs/aws/amazon-rds-proxy-now-generally-available/

[brannondorsey]

Any updates on when support for this could be expected? Like @phanimullapudi some of our work is blocked until the AWS TF provider supports RDS proxy. Thanks!

[jrobison-sb]

@apparentlymart @bflad Is there any reason this couldn't merge? It seems like the tests pass. We're also in a position where we would use this service if we could, but it's blocked until Terraform supports it.

[bflad]

Hi folks 👋 We are just as excited as you are that this functionality is now generally available. We are beginning work on breaking changes as part of the version 3.0.0 release of the Terraform AWS Provider now. The earliest consideration from the HashiCorp maintainers for review of existing contributions would be after 3.0 is released in (hopefully) two or three weeks. Any prioritization over existing roadmap priorities would need to be made based on our criteria which can be found in the FAQ and we are actively working on next quarter's roadmap so getting those 👍 upvotes in is very helpful.

[alexjurkiewicz]

Please break out the auth block to a separate resource, like:

resource aws_db_proxy_authorisation myuser {
  proxy_identifier = aws_db_proxy.default.id
  secret_arn = aws_secretsmanager_secret.myuser.arn
}

I see this situation as analogous to the way security group rules were broken out from the security group resource.

If secrets are managed inline as part of the aws_db_proxy resource, it will be impossible to dynamically manage the set of users independently of the Proxy resource itself. Here's an example workflow that would make use of this:

1. Stack A creates RDS database cluster & proxy resource
2. Stack B create a new DB user in the RDS cluster
3. To use this user with the proxy, you need to hardcode the username into stack A. Circular dependency!

This "two stack" workflow is required when managing MySQL users inside a database, because you need to know your RDS hostname before initialising the postgresql or mysql providers.

[gazoakley]

Hi @alexjurkiewicz - I don't think the AWS API deals well with this unfortunately:

• An array of UserAuthConfig objects must be provided when calling CreateDBProxy - I don't know if that could be an empty array, but I suspect it won't work
• There's no API to add/remove the authentication settings - instead there's a ModifyDBProxy call that replaces the settings as a whole. A resource designed to work as above would have to pull the existing settings, update them in memory and push them back - but that would likely result in race conditions (two aws_db_proxy_authorisation resources trying to call ModifyDBProxy on the same proxy would suffer from last write wins)

@bflad - Any design suggestions here?

[bflad]

Hi @gazoakley 👋 This functionality is on the maintainer's team roadmap for this quarter, but we haven't set aside a specific timeline to look into it yet. I think we might be able to look into this in about two weeks given some other in-flight work happening.

Generally speaking though, if the create API requires certain configuration (as it appears it does in this case), that configuration and handling has to live with the "parent" resource. Attempting to create a "child" association resource in this situation leads to complicated implementation details like we have with the aws_route53_zone_association resource. We also have this problem with ECS API in places as well. It is not something that we can workaround due to the API and Terraform Core/SDK design requirements.

[alexjurkiewicz]

I agree this design results in a more complex API for users ("do I use the X block or resource?"), but it supports a broader range of use cases. I see RDS Proxy as being used like security groups, where a "container" proxy is created and output in one Terraform configuration, then authorisations are added to it from other configurations. I hope you decide to support this 🙏

[RalphBragg]

Be really great to get support for this resource as well. This is the only item that isn't terraformed in our stack.

[bflad]

The first resource, aws_db_proxy, has been merged and will release with version 3.5.0 of the Terraform AWS Provider, likely on Thursday. Thanks to @gazoakley for the implementation! 👍 We'll be working through other potential resources the next few days/weeks.

[bflad]

The second (optional) resource, aws_db_proxy_default_target_group, was released with version 3.6.0 of the Terraform AWS Provider last week. Going to provide a review for aws_db_proxy_target today.

[nirsv]

How can I use it as data? I see that data "aws_db_proxy" is not supported. Is there any workaround?

[bflad]

Hi @nirsv 👋 If you have a use case for looking up RDS DB Proxies via a data source, it might be best to capture that (and other future) functionality in a new, separate feature request. This one will be closed once the aws_db_proxy_target resource is merged to round out the initial end-to-end configuration of this feature and prevent this generic issue from not having a definition of "done". 👍

[bflad]

Support for the aws_db_proxy_target resource has been merged and will release with version 3.9.0 of the Terraform AWS Provider, likely tomorrow. Thanks to @goakley for the implementation. 👍

If you have bug reports or additional feature requests with the RDS DB Proxy support, please submit new GitHub issues following the templates and we will take a look. 😄

[ghost]

This has been released in version 3.9.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!