v0.7.0

Release notes for Cluster API Provider AWS (CAPA) v0.7.0

Documentation

Changelog since v0.6.5

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

• v0.7.0 is based on Cluster API v1alpha4 and MUST be used in conjunction with Cluster API v0.4.x
• EKS support has graduated out of experimental and is now enabled by default. 🎉
  Please see the additional section below for details. (#2648, @richardcase)

All users of Cluster API Provider AWS (whether you use EKS or not) should read the notes below regarding the EKS graduation as it impacts the provider as a whole (e.g. IAM permissions)

• Cluster API Provider AWS will now be preferentially pinned to control plane nodes. This is especially helpful when running self-managed management clusters in AWS as for EC2-based control planes, the control plane EC2 instances have the controlplane.cluster-api.sigs.k8s.io IAM role which has sufficient permissions for Cluster API Provider AWS to run.
  Please ensure your control plane nodes have sufficient resources to run Cluster API Provider AWS. (#2377, @vespian)
• Controllers policy updated with missing KMS permissions required to use EKS encryption, if you are planning to use EKS encryption then you will need to update your controllers policy by running clusterawsadm bootstrap iam create-cloudformation-stack again. And then when you create a cluster with encryption enabled you will need to use a KMS key that has an alias name starting with cluster-api-provider-aws-. For further information see the docs. (#2447, @richardcase, #2505, @Ankitasw)
• Controllers policy updated with missing key pairs permission, if you are using or plan to use AWSManagedMachinePool with an SSH key then you will need to update your controllers policy by running clusterawsadm bootstrap iam create-cloudformation-stack again. (#2404, @richardcase)
• During v0.6.x to v0.7.x upgrade: All secrets that are used for AWSClusterStaticIdentity should be moved to controller namespace manually if they are not already in the capa-system namespace. (#2425, @sedefsavas)
• Renamed field networkSpec as network. Check AWSCluster and AWSManagedControlPlane manifests when switching versions. (#2571, @Ankitasw)

Changes by Kind

Feature

• Adds the ability to configure EBS volume throughput in supported EBS types (#2468, @cnmcavoy)
• Add controller related commands to clusterawsadm: zero/update/print bootstrap credentials and rollout controllers (#2457, @sedefsavas)
• Add externally managed predicate. Clusters marked with "cluster.x-k8s.io/managed-by" annotation should be skipped from reconciliation. (#2383, @alexander-demichev)
• Apply clusterctl.cluster.x-k8s.io/move-hierarchy label on the infrastructure cluster global identity CRDs. (#2524, @shivi28)
• CLI command to list AWS resources created by CAPA (#2509, @shivi28)
• Clusterawsadm ARM64 builds for both Linux and macOS are now available (#2557, @scottslowe)
• No longer mandatory to set encryption value to True for root volumes when using encrypted AMIs (#2556, @shivi28)
• OIDC provider association for EKS clusters. (#2422, @sadysnaat)
• Tagging elastic IPs on creation (#2551, @Madhur97)
• Taints supported on EKS node groups created via AWSmanagedMachinePool (#2405, @richardcase)
• NAT gateways are now deleted in parallel, which should reduce cluster deletion time by >50% for multi-AZ clusters (#2600, @shivi28)
• Add support for G4ad xlarge and 2xlarge instances powered by AMD Radeon Pro V520 GPUs and AMD 2nd Generation EPYC processors (#2626, @dependabot[bot])
• Enable usage of GPU optimized AMIs for EKS
  Removed unused fields like ARN and Filters from AMIReference (#2549, @shivi28)
• Validate label selector for AWS Identity CRDs
• Add AWSClusterStaticIdentity webhook with validation checks (#2436, @Ankitasw)

Bug or Regression

• AWSMachine objects successfully deleted in case of invalid credentials (#2601, @shivi28)
• AWSMachinePool controller removes one old LaunchTemplate version before creating a new version, preventing the number of versions from growing without bound, and reaching the maximum limit. (#2525, @dlipovetsky)
• Add root storage device tags through additionalTags in ec2 instance (#2463, @Ankitasw)
• Align region resolution in create/delete cloudformation stack commands (#2423, @Szymongib)
• Fixes bug in elb.DescribeTags when the user has more than 20 load balancers in an account (#2500, @faiq)
• Correct field being used for endpoint column on kubectl get AWSCluster (#2529, @njuettner)
• Do not delete security groups when provided as overrides (#2555, @sedefsavas)
• EKS Nodepool min/max will be updated to match the AWSManagedMachinePool spec, overriding changes to min/max made via the AWS Console, CLI, or SDK. (#2375, @richardcase)
• RBAC permission and update documentation for multi-tenancy (#2373, @paulcarlton-ww)
• Specifying no SSH key for machine pool launch templates. (#2362, @jimmidyson)
• When the AWSMachinePool controller scales an AWS Auto Scaling Group, it updates the Launch Template with a valid bootstrap token. (#2354, @dlipovetsky)
• Patch VPC ID immediately after VPC creation, to deal with edge case where multiple VPCs may get created with the same tags. (#2587, @sedefsavas)
• Process extra statements for Cluster API Controllers (#2437, @Szymongib)
• Update RBAC with missing awsclustercontrolleridentities permission (#2359, @martin-ducar-gd)
• Update EKSConfig secret on kubeletExtraArgs changes (#2579, @trutx)
• Fix for reconciling LaunchTemplates. (#2411, @dkoshkin)
• Fix typo in AWSFargateProfile validation webhook which cause the webhook not called. (#2445, @jzhoucliqr)

Documentation

• Fix typo in documentation ([#2365](#2...

v0.7.0-alpha.0

🚨 This is an ALPHA RELEASE. Use it only for testing purposes, if you find any bugs file an issue. v1alpha4 API is not yet complete.

The images for this release are:
k8s.gcr.io/cluster-api-aws/cluster-api-aws-controller:v0.7.0-alpha.0
k8s.gcr.io/cluster-api-aws/eks-controlplane-controller:v0.7.0-alpha.0
k8s.gcr.io/cluster-api-aws/eks-bootstrap-controller:v0.7.0-alpha.0

Thanks to all our contributors.

v0.6.8

Changelog since v0.6.7

Bug or Regression

• Fix for filtering managed SecurityGroups correctly.(#2620, @sedefsavas)

The images for this release are:
k8s.gcr.io/cluster-api-aws/cluster-api-aws-controller:v0.6.8
k8s.gcr.io/cluster-api-aws/eks-controlplane-controller:v0.6.8
k8s.gcr.io/cluster-api-aws/eks-bootstrap-controller:v0.6.8

Thanks to all our contributors.

v0.6.7

IMPORTANT:

!!Do not use this release!! There is a critical bug in this release that causes cluster deletion failures, which is solved in v0.6.8. This bug does not impact any other release.

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

• Action required
  Controllers policy updated with missing KMS permissions required to use EKS encryption, if you are planning to use EKS encryption then you will need to update your controllers policy by running clusterawsadm bootstrap iam create-cloudformation-stack again. And then when you create a cluster with encryption enabled you will need to use a KMS key that has an alias name starting with cluster-api-provider-aws-. For further information see docs. (#2448, @richardcase)

Changes by Kind

Feature

• Taints supported on EKS node groups created via AWSManagedMachinePool (#2450, @richardcase)

Bug or Regression

• AWSMachinePool controller removes one old LaunchTemplate version before creating a new version, preventing the number of versions from growing without bound, and reaching the maximum limit. (#2531, @dlipovetsky)
• Fix for ELB deletion when there are more than 20 ELBs. (#2512, @faiq)
• Fix for only deleting Security Groups managed by CAPA controllers, not the overridden ones. (#2560, @sedefsavas)
• Patch VPC ID immediately after VPC creation, to deal with edge case where multiple VPCs may get created with the same tags. (#2587, @sedefsavas)
• Fix for reconciling LaunchTemplates. (#2410, @dkoshkin)
• Fix typo in AWSFargateProfile validation webhook which cause the webhook not called. (#2446, @jzhoucliqr)

Other (Cleanup or Flake)

• Updated dependencies (#2486, @randomvariable)

See CHANGELOG.md for dependency updates.

v0.6.6

Release notes for Cluster API Provider AWS (CAPA) v0.6.6

Documentation

Changelog since v0.6.5

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

!Important Release Note !(This node is added after the release)
A new Launch Template Version gets created almost at each reconcile and since there is a 10,000 versions per launch template limitation, AWSMachinePools stop working after 2 months and needs to be recreated (#2368). This will be fixed in the next release.

!Action required!
Controllers policy updated with missing key pairs permission, if you are using or plan to use AWSManagedMachinePool with an SSH key then you will need to update your controllers policy by running clusterawsadm bootstrap iam create-cloudformation-stack again. (#2408, @richardcase)

Changes by Kind

Bug or Regression

• Update EKS Nodepool min/max to match the AWSManagedMachinePool spec, overriding changes to min/max made via the AWS Console, CLI, or SDK (#2407, @richardcase)
• Fix for reconciling LaunchTemplates after "clusterctl move"(#2394, @sedefsavas) (#2410, @dkoshkin)
• Fix specifying no SSH key for machine pool launch templates (#2362, @jimmidyson)
• Update LaunchTemplate with a valid bootstrap token after ASG scale (#2401, @dlipovetsky)
• Add identity ref support for fargate controller (#2406, @jzhoucliqr)

The images for this release are:
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.6.6
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-controlplane-controller:v0.6.6
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-bootstrap-controller:v0.6.6

Thanks to all our contributors.

v0.6.5

Release notes for Cluster API Provider AWS (CAPA) v0.6.5

Documentation

Changelog since v0.6.4

Urgent Upgrade Notes

(No, really, you MUST read this before you upgrade)

• The behaviour when selecting which subnets to use with machine pools (AWSMachinePool & AWSManagedMachinePool) when subnet ids haven't been included has changed. There is now a defined order of precedence that will determine which subnets to use:
  1. Subnets defined explicitly in the spec of AWSMachinePool/AWSManagedMachinePool
  2. If AvailabilityZones is specified on AWSMachinePool/AWSManagedMachinePool then the subnets associated with those AZs will be used
  3. If failureDomains are specified in the MachinePool then subnets that are in those failureDomains (a.k.a. Availability Zones) will be used
  4. All the private subnets from the control plane are used (#2302, @richardcase)
• EKS: New AWSManagedMachinePool resources with non-empty remoteAccess now require remoteAccess.public: true in order to allow public access to SSH on port 22 (#2243, @michaelbeaumont)

Changes by Kind

API Change

• Add the following cluster-scoped resources for multi-tenancy support:
  • AWSClusterStaticIdentity - Static credentials using a Access Key ID and Secret Key
  • AWSClusterControllerIdentity - A singleton resource that states a cluster can use inherited credentials
  • AWSClusterRoleIdentity - An IAM role definition (#2253)
• Add ability to toggle the new AWS Capacity Rebalance feature by setting a new .spec.capacityRebalance field in AWSMachinePool objects. (#2288, @trutx)

Feature

• Add Multi-tenancy support (multi-tenancy proposal) (#2253)
• For migration of current cluster resources to the new multi-tenancy model, there is a new experimental controller
  AutoControllerIdentityCreator that will create and apply AWSClusterControllerIdentity to all existing resources. This will be enabled by default until v1alpha4.
  No additional steps are needed to migrate existing clusters, multi-tenancy model is fully backward-compatible when this controller is kept enabled (#2253)
• Add clusterawsadm ami list command to list AMIs that can be filtered by region, OS, and Kubernetes version. (#2304, @sedefsavas)
• clusterawsadm ami commands now support --source-region to copy AMIs across regions (#2345)
• clusterawsadm ami commands now output versioned AWSAMIList and AWSAMI resources to stdout (#2345, @randomvariable)
• Add the ability to enable the AWS SDK debug logging (#2229, @shuheiktgw)
• Adopt the release-notes tool from kubernetes/release to generate the changelog for a release (#2247, @richardcase)
• PRs now require release-notes code block, which is used in the release notes generation (#2232, @richardcase)
• EKS: Ability to declaratively remove the Amazon VPC CNI when using an alternate CNI (#2292, @richardcase)
• EKS: Add the AWSFargateProfile resource for managing EKS Fargate profiles (#2265, @michaelbeaumont)
• EKS: Add/update conditions for the AWSManagedControlPlane to detect when EKS control plane is being created or updated. (#2246, @michaelbeaumont)
• EKS: Add new cluster template for a GPU-accelerated EKS cluster (#2278, @richardcase)

Documentation

• Add documentation for IAM permissions and clusterawsadm, dynamically generated via clusterawsadm itself (#2342, @randomvariable)
• Add Published AMIs page that gets updated by a lambda function every hour (#2345, @randomvariable)
• Add auto-generated CRD reference documentation for core APIs, EKS controlplane and experimental features (#2347) (#2352, @randomvariable)
• Add multitenancy documentation with examples (#2319, @sedefsavas)
• EKS: Add a guide on how to develop EKS Control Plane locally using Tilt (#2234, @kenichi-shibata)
• EKS: New ADR to document the decision of how Fargate Profiles will be represented. (#2250, @michaelbeaumont)

Failing Test

• Enable EventBridgeInstanceState feature in e2e tests (#2293, @sedefsavas)

Bug or Regression

• AWSMachine: Add filters support for additional security groups (#2241, @alexander-demichev)
• Fix AWSCluster & AWSMachine validation webhooks to accept an empty string for the SSHKeyName field (#2308, @dlipovetsky)
• Fix bug where custom bootstrap user name was not accepted (#2341, @randomvariable)
• Restore GovCloud and other AWS partition support for CloudFormation generation. If using EKS, you must provide the relevant value for partition in your clusterawsadm configuration file. (#2289, @randomvariable)

Other (Cleanup or Flake)

• Add validation for loadbalancer scheme to allow only Internet-facing and internal values (#2290, @sedefsavas)
• Add test coverage to test grid (#2350, @sedefsavas)
• Add upgrade to Kubernetes main test (#2313, @sedefsavas)
• EKS: Removal of AWSManagedCluster from templates/docs to help with the future deprecation in v1alpha4 (#2264, @richardcase)

Support

• @sedefsavas joined to the maintainers of Cluster API Provider AWS (#2279, @richardcase)

The images for this release is:
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.6.5
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-controlplane-controller:v0.6.5
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-bootstrap-controller:v0.6.5

Thanks to all our contributors!

Special thanks to @detiber for his contributions to CAPA from its inception who moves to emeritus status now.

v0.6.4

Changes since v0.6.3

✨ New Features

• Add fine-grained service rate limiters (#2201)
• Add list of instances to AWSMachinePool status (#2197)
• Add support for AWS_CONTROLLER_IAM_ROLE environment variable using Kiam or IRSA (#2100)
• clusterawsadm: Allow to take a configuration file to print IAM policy documents (#2147)
• clusterawsadm: Add ability to copy AMIs to a target account with encryption (#2112, #2203)
• EKS: addon support (#2202)
• EKS: Support custom AMI lookup (#2057)
• EKS: Secondary cidr support for workload clusters (#2086)
• EKS: Add provisioning of IRSA to workload clusters (#2070)

🐛 Bug Fixes

• Fix OpenAPI defaulting for secrets backend and add OpenAPI testing (#2135)
• Updated service account for leader election (#2183)
• AWSCluster: Error if a loadbalancer exist with the same name in the same region with a different scheme (#2154)
• AWSCluster: Fix NPE when comparing load balancers (#2163)
• AWSMachinePool: Trigger rolling replacement upon launch template change (#2193)
• AWSMachinePool: Remove unused ID field on launch template spec (#2184)
• EKS: Set Subnets as an optional property (#2140)
• clusterawsadm: Add UpdateAutoScalingGroup to controller IAM (#2194)

📖 Documentation

• EKS console documentation (#2187)
• Fix broken links and emojis (#2159, #2150)
• Add config example to docs specifying IAM role (#2151)
• Updated wording on prerequisites doc (#2149)
• Update consuming-existing-aws-infrastructure.md to provide context to where networkSpec belongs (#2103)
• ADR: EKS packaging (#2126)
• ADR: e2e test structure (#2127)
• AMI Updates (#2164, #2190, #2206)

🌱 Others

• EKS: e2e tests added (#2168, #2220, #2199, #2188, #2211, #2214)
• EKS: deletion checks (#2175)
• AWSCluster: CAPI E2E tests (#2138)
• Consolidating boskos scripts for CAPA (#2101)
• Upgrade AWS SDK version to 1.36.26 (#2204)
• Add Interruptible field to AWSMachine status (#2120)
• Upgrade CAPI version to v0.3.12 (#2129, #2133, #2198)
• Refactor image build and release process (#2213, #2215, #2216, #2217)
• Enable use of shared configuration file in clusterawsadm (#2077)
• Update CAPA maintainer and reviewers (#2031, #2161)
• Refactor release process for Github and Staging (#2096)

The images for this release is:
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.6.4
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-controlplane-controller:v0.6.4
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-bootstrap-controller:v0.6.4

Thanks to all our contributors!

Special thanks to @ncdc for his contributions to Cluster API from its inception who moves to emeritus status.

Shoutout to @richardcase for joining the maintainers of Cluster API Provider AWS, as well as @michaelbeaumont, @dthorsen, and @sedefsavas joining the project reviewers.

😊

v0.6.3

Changes since v0.6.2

🐛 Bug Fixes

• awsmachinetemplates: Allow cloudInit.secureSecretsBackend to be defaulted (#2111)
• fix ASG event message (#2108)

🌱 Others

• Upgrade e2e Kubernetes version to 1.19.4 (#2114)
• Remove old terraform scripts (#2113)
• Build new AMIs (#2109)
• refactor: e2e test reorganisation (#2102)

The images for this release are:
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.6.3
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-controlplane-controller:v0.6.3
us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-bootstrap-controller:v0.6.3

Thanks to all our contributors! 😊

v0.6.2

Changes since v0.6.1

🐛 Bug Fixes

• Validate that additional security groups can not have filters (#2072)
• AWSManagedMachinePool: Fix cloud provider key usage and nodegroup IAM role name (#2085)
• Fallback to cluster subnets on update ASG (#2095)
• Fix ownerref group on kube secret and configmap (#2092)
• Add CNI defaulting to the controller (#2090)
• Update in-controller AWSMachine CloudInit defaulting logic (#2082)
• Add fallback if subnets not provided on AWSMachinePool (#2051)
• Make the regex less restrictive to allow underscore etc. for ssh key file name (#2071)
• Add required iam permission for managed node groups (#2043)
• Ensure env var enables AWSMachinePool webhooks (#2046)
• Add more conditions for resource status filtering (#2049)
• Validate subnet's AZ with failure domain when subnet id is passed (#2011)

📖 Documentation

• Update docs to use capa-system namespace (#2094)
• Add ADR template and first ADR (#2080)
• Updated flavours and docs for machine pools (#2030, #2044)
• Fix typos in EKS commands (#2038)

🌱 Others

• Remove unused allow additional roles feature flag related code (#2035)
• Updated Bastion node's AMIs. Changed the OS version to Ubuntu 20.04 from Ubuntu 16.04 (#2068)
• Makefile: Ensure manifests compile during verify (#2098)
• Fix roundtrip conversions between v1alpha2 and v1alpha3 (#2074)

The images for this release are:

• us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.6.2
• us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-controlplane-controller:v0.6.2
• us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-bootstrap-controller:v0.6.2

Thanks to all our contributors! 😊

v0.6.1

Changes since v0.6.0

⚠️ Breaking Changes

• EKS Control Plane Provider (aws-eks) has been introduced (#1949, #1997, #1939, #1973, #1966, #1960, #1943)
  As part of this work the infrastructure manager has been refactored to remove the EKS control plane functionality. This means the new provider along with the existing EKS bootstrap provider (which has been renamed aws-eks) will need to be used if you want to create EKS based clusters with Cluster API Provider AWS. For example:

export EXP_EKS=true
clusterctl --infrastructure=aws --control-plane=aws-eks --bootstrap=aws-eks

✨ New Features

• Allow AWS Systems Manager Parameter Store to be used as a secrets backend for userdata for regions where AWS Secrets Manager is not available (#1924)
• Add a tag to instances during creation that matches the Cluster API machine name (#2015)
• Additional user & role mappings for aws-iam-authenticator are now configurable (#1995, #2002, #1938)
• AWSMachinePools support for EC2 autoscaling groups (#1860, #2010, #2014, #2007, #1863, #2021, #2006, #2000)
• AWSManagedMachinePool (EKS managed nodegroup) support (#1916, #2024, #2013)
• Allow specifying subnet IDs to be used for control plane load balancers (#1931)
• Add ability to specify instance tenancy, i.e. dedicated EC2 instances (#1926)
• Add conditions for the reconcile delete workflow (#1905)
• Cluster API Provider AWS can be configured to use custom endpoints for connecting to AWS services (#1858)
• Improved validation around fields that are passed in as AWS API parameters (#1978)

🐛 Bug Fixes

• Persist subnet changes into the AWSCluster spec early when using default subnets (#1915)
• Allow the usage of unmanaged VPCs without public subnets (#1884)
• Fix NPE when deleting security groups on cluster deletion (#1996)
• Add externalManagedControlPlane Status to allow node drains under EKS (#1992, #1994)
• Fix for allowing nodes to join the EKS cluster (#1962)
• Fix bastion reconcilation and connection error when using eks flavor (#1957)
• Retry with listing all ELBs when listing by tag fails, fixing an issue in environments where the ResourceTagging API is not available (#1952)
• AWSMachine ssh key should defer to that configured on the AWSCluster resource when nil (#1932)
• Add capi exp schema and fix manager args and rbac (#1936)
• Fix volume description to remove 'root' since it's also used for non-root volume (#2005)
• Fix incorrect capitalization for eks field (#1998)

📖 Documentation

• Cluster API Provider AWS has a new website: cluster-api-aws.sigs.k8s.io (#1947, #2017, #2028, #1981)
• Add docs on updating AWS credentials used by Cluster API Provider AWS (#1948)

🌱 Others

• Remove unused integration test on pull requests (#2012)
• e2e: Add test for spot instances (#1963)
• e2e: Verify code compiles on pull requests (#1953)
• e2e: Install CNI using ClusterResourceSet (#1816)
• unit tests: instances - Sort tag keys so unit tests can succeed (#1937)
• Show more helpful error message when duplicate clusters are created across namespaces, resulting in duplicate VPCs (#1880)
• Golang version updated to 1.13.15 (#1944)
• Controller runtime updated to 0.5.11 (#1950)

The images for this release is:

Core AWS Controller: us.gcr.io/k8s-artifacts-prod/cluster-api-aws/cluster-api-aws-controller:v0.6.1
EKS Bootstrap Controller: us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-bootstrap-controller:v0.6.1
EKS ControlPlane Controller: us.gcr.io/k8s-artifacts-prod/cluster-api-aws/eks-controlplane-controller:v0.6.1

Thanks to all our contributors! 😊