[aws-msk] L2 construct for Cluster

[iliapolo]

Create an L2 Cluster construct that provisions an MSK cluster.

Use Case

Increase our L2 coverage and make it easier for users to get started with MSK

Proposed Solution

We want to provide a rudimentary MVP that users can use to create a cluster as quickly as possible. We want to reach a state where users can just do:

import * as msk from '@aws-cdk/aws-msk'

new msk.Cluster(this, 'Cluster', {
  version: msk.KafkaVersion.V2_2_1
});

The experience should be very similar to the EKS Cluster construct.

As a reference, we should emulate the console experience, and use the same default values it provides.

As far as which configuration properties should be supported in this initial implementation, here is a proposed list:

• KafkaVersion [REQUIRED]

• ClusterName [OPTIONAL]

• NumberOfBrokerNodes [OPTIONAL]

• ConfigurationInfo [OPTIONAL].

  Note that creating configuration sets is currently not supported by CloudFormation, you can only reference them. The creation is done out-of-band with the CLI. This is ok, I imagine users will have some configuration sets already created in their account, and we should allow them to use it. In the future, we can consider allowing to create configuration sets using a custom resource.

• ClientSubnets [OPTIONAL]

• SecurityGroups [OPTIONAL]

• InstanceType [OPTIONAL]

I think these set of properties will provide a good getting started experience, and be flexible enough to cover plenty of use-cases.

Other

CloudFormation specification: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-msk-cluster.html

Implementation Notice

To those who eventually pick this issue up:

Before diving deep into the implementation code, please create an initial PR that includes only the API design, i.e, the README of this new module. Since this is a brand new L2 library, a design discussion is warranted, and will take place on the PR reviewing the README.

Thanks

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[curtiseppel]

Hey, I've just ported our internal MSK Cluster construct and adapted it with the default values that the AWS console suggests. Admittedly, I didn't create/search for issues first so the PR contains the implementation also - #9908

[iliapolo]

@curtiseppel Awesome!

I'll try to get the review going soon.

Thanks :)

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.