Skip to content

Commit

Permalink
Provide the default experience for setting your node role
Browse files Browse the repository at this point in the history
  • Loading branch information
@jonathan-innis
jonathan-innis committed Oct 26, 2023
1 parent 1eab693 commit 0bf3008
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 27 deletions.
43 changes: 19 additions & 24 deletions website/content/en/preview/upgrading/upgrade-guide.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -97,13 +97,11 @@ Add `~/go/bin` to your $PATH, if you have not already done so.
```

{{% alert title="Warning" color="warning" %}}
If you open a new shell to run steps in this procedure, you need to set some or all of the environment variables again.
To remind yourself of these values, type:
If you open a new shell to run steps in this procedure, you need to set some or all of the environment variables again. To remind yourself of these values, type:

```bash
echo $KARPENTER_VERSION $AWS_PARTITION $CLUSTER_NAME $AWS_REGION $AWS_ACCOUNT_ID $KARPENTER_IAM_ROLE_ARN $CLUSTER_ENDPOINT $TEMPOUT
```

{{% /alert %}}

4. Apply the new Karpenter policy and assign it to the existing Karpenter role:
Expand Down Expand Up @@ -150,79 +148,76 @@ Add `~/go/bin` to your $PATH, if you have not already done so.
Karpenter has deprecated and moved a number of Helm values as part of the v1beta1 release. Ensure that you upgrade to the newer version of these helm values during your migration to v1beta1. You can find detail for all the settings that were moved in the [v1beta1 Upgrade Reference]({{<ref "v1beta1-reference#helm-values" >}}).
{{% /alert %}}

7. Install the `karpenter-convert` tool to automatically convert the alpha Karpenter manifests to beta:
7. Install the `karpenter-convert` tool to help convert the alpha Karpenter manifests to beta manifests:

```bash
go install github.com/aws/karpenter/tools/karpenter-convert/cmd/karpenter-convert@latest
```

8. Convert each AWSNodeTemplate to an EC2NodeClass. To convert your v1alpha Karpenter manifests to v1beta1, you can either manually apply changes to API components or use the [Karpenter conversion tool](https://github.com/aws/karpenter/tree/main/tools/karpenter-convert).
See the [AWSNodeTemplate to EC2NodeClass]({{< relref "v1beta1-reference#awsnodetemplate-to-ec2nodeclass" >}}) section of the Karpenter Upgrade Reference for details on how to update to Karpenter AWSNodeTemplate objects. Here is an example of how to use the `karpenter-convert` CLI to convert an AWSNodeTemplate file to a EC2NodeClass file:
8. Convert each AWSNodeTemplate to an EC2NodeClass. To convert your v1alpha Karpenter manifests to v1beta1, you can either manually apply changes to API components or use the [`karpenter-convert`](https://github.com/aws/karpenter/tree/main/tools/karpenter-convert) CLI tool. See the [AWSNodeTemplate to EC2NodeClass]({{< relref "v1beta1-reference#awsnodetemplate-to-ec2nodeclass" >}}) section of the Karpenter Upgrade Reference for details on how to update to Karpenter AWSNodeTemplate objects.

For each EC2NodeClass, specify the `$KARPENTER_NODE_ROLE` you will use for nodes launched with this node class. Karpenter v1beta1 [drops the need for managing your own instance profile and uses node roles directly]({{< ref "v1beta1-reference#instanceprofile" >}}). The example below shows how to migrate your AWSNodeTemplate to an EC2NodeClass if your node role is the same role that was used when creating your cluster with the [Getting Started Guide]({{< ref "../getting-started/getting-started-with-karpenter" >}}).

```bash
karpenter-convert -f awsnodetemplate.yaml > ec2nodeclass.yaml
export KARPENTER_NODE_ROLE="KarpenterNodeRole-${CLUSTER_NAME}"
karpenter-convert -f awsnodetemplate.yaml | envsubst > ec2nodeclass.yaml
```

9. Edit the converted EC2NodeClass file manually:

* Specify your AWS role where there is a `$KARPENTER_NODE_ROLE` placeholder. For example, if you created your cluster using the [Getting Started with Karpenter]({{< ref "../getting-started/getting-started-with-karpenter" >}}) guide, you would use the name `KarpenterNodeRole-$CLUSTER_NAME`, substituting your cluster name for `$CLUSTER_NAME`.
* Otherwise, check the file for accuracy.

10. When you are satisfied with your EC2NodeClass file, apply it as follows:
9. When you are satisfied with your EC2NodeClass file, apply it as follows:

```bash
kubectl apply -f ec2nodeclass.yaml
```

11. Convert each Provisioner to a NodePool. Again, either manually update your Provisioner manifests or use the karpenter-convert CLI tool:
10. Convert each Provisioner to a NodePool. Again, either manually update your Provisioner manifests or use the [`karpenter-convert`](https://github.com/aws/karpenter/tree/main/tools/karpenter-convert) CLI tool:

```bash
karpenter-convert -f provisioner.yaml > nodepool.yaml
```

12. When you are satisfied with your NodePool file, apply it as follows:
11. When you are satisfied with your NodePool file, apply it as follows:

```bash
kubectl apply -f nodepool.yaml
```

13. Roll over nodes: With the new NodePool yaml in hand, there are several ways you can begin to roll over your nodes to use the new NodePool:
12. Roll over nodes: With the new NodePool yaml in hand, there are several ways you can begin to roll over your nodes to use the new NodePool:

14. Periodic Rolling with [Drift]({{< relref "../concepts/disruption#drift" >}}): Enable [drift]({{< relref "../concepts/disruption#drift" >}}) in your NodePool file, then do the following:
13. Periodic Rolling with [Drift]({{< relref "../concepts/disruption#drift" >}}): Enable [drift]({{< relref "../concepts/disruption#drift" >}}) in your NodePool file, then do the following:
- Add the following taint to the old Provisioner: `karpenter.sh/legacy=true:NoSchedule`
- Wait as Karpenter marks all machines owned by that Provisioner as having drifted.
- Watch as replacement nodes are launched from the new NodePool resource.

Because Karpenter will only roll of one node at a time, it may take some time for Karpenter to completely roll all nodes under a Provisioner.

15. Forced Deletion: For each Provisioner in your cluster:
14. Forced Deletion: For each Provisioner in your cluster:

- Delete the old Provisioner with: `kubectl delete provisioner <provisioner-name> --cascade=foreground`
- Wait as Karpenter deletes all the Provisioner's nodes. All nodes will drain simultaneously. New nodes are launched after the old ones have been drained.
16. Manual Rolling: For each Provisioner in your cluster:
15. Manual Rolling: For each Provisioner in your cluster:
- Add the following taint to the old Provisioner: `karpenter.sh/legacy=true:NoSchedule`
- For all the nodes owned by the Provisioner, delete one at a time as follows: `kubectl delete node <node-name>`
17. Update workload labels: Old alpha labels (`karpenter.sh/do-not-consolidate` and `karpenter.sh/do-not-evict`) are deprecated, but will not be dropped until Karpenter v1. However, you can begin updating those labels at any time with `karpenter.sh/do-not-disrupt`. Any pods that specified a `karpenter.sh/provisioner-name:DoesNotExist` requirement also need to add a `karpenter.sh/nodepool:DoesNotExist` requirement to ensure that the pods continue to not schedule to nodes unmanaged by Karpenter while migrating to v1beta1.
16. Update workload labels: Old alpha labels (`karpenter.sh/do-not-consolidate` and `karpenter.sh/do-not-evict`) are deprecated, but will not be dropped until Karpenter v1. However, you can begin updating those labels at any time with `karpenter.sh/do-not-disrupt`. Any pods that specified a `karpenter.sh/provisioner-name:DoesNotExist` requirement also need to add a `karpenter.sh/nodepool:DoesNotExist` requirement to ensure that the pods continue to not schedule to nodes unmanaged by Karpenter while migrating to v1beta1.
18. Check that there are no more Provisioner, AWSNodeTemplate, or Machine resources on your cluster. at which time you can delete the old CRDs. To validate this, run the following command and ensure that there are no outputs to any of them:
17. Check that there are no more Provisioner, AWSNodeTemplate, or Machine resources on your cluster. at which time you can delete the old CRDs. To validate this, run the following command and ensure that there are no outputs to any of them:
```bash
kubectl get machines
kubectl get awsnodetemplates
kubectl get provisioners
```
19. Remove the alpha Karpenter CRDs from the cluster.
18. Remove the alpha Karpenter CRDs from the cluster.
```bash
kubectl delete crd machines.karpenter.sh
kubectl delete crd awsnodetemplates.karpenter.k8s.aws
kubectl delete crd provisioners.karpenter.sh
```
20. Remove the alpha instance profile(s). If you were just using the InstanceProfile deployed through the [Getting Started Guide]({{< ref "../getting-started/getting-started-with-karpenter" >}}), delete the `KarpenterNodeInstanceProfile` section from the CloudFormation. Alternatively, if you want to remove the instance profile manually, you can run the following command
19. Remove the alpha instance profile(s). If you were just using the InstanceProfile deployed through the [Getting Started Guide]({{< ref "../getting-started/getting-started-with-karpenter" >}}), delete the `KarpenterNodeInstanceProfile` section from the CloudFormation. Alternatively, if you want to remove the instance profile manually, you can run the following command
```bash
ROLE_NAME="KarpenterNodeRole-${ClusterName}"
Expand All @@ -231,7 +226,7 @@ Add `~/go/bin` to your $PATH, if you have not already done so.
aws iam delete-instance-profile --instance-profile-name "${INSTANCE_PROFILE_NAME}"
```
21. Finally, remove the alpha policy from the controller role: This will remove any remaining permissions from the alpha APIs. You can orchestrate the removal of this policy with the following command:
20. Finally, remove the alpha policy from the controller role: This will remove any remaining permissions from the alpha APIs. You can orchestrate the removal of this policy with the following command:
```bash
ROLE_NAME="${CLUSTER_NAME}-karpenter"
Expand Down
6 changes: 3 additions & 3 deletions website/content/en/preview/upgrading/v1beta1-reference.md
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "v1beta1 Upgrade Reference"
linkTitle: "v1beta1 Upgrade Reference"
title: "v1beta1 Reference"
linkTitle: "v1beta1 Reference"
weight: 30
description: >
API information for upgrading Karpenter
API information for upgrading Karpenter to v1beta1
---

Significant changes to the Karpenter APIs have been introduced in Karpenter v0.32.x.
Expand Down

0 comments on commit 0bf3008

Please sign in to comment.