Skip to content

Commit

Permalink
Merge pull request #48110 from sftim/20240927_tweak_kubeadm_certs
Browse files Browse the repository at this point in the history
Tweak kubeadm certs advice
  • Loading branch information
k8s-ci-robot authored Sep 27, 2024
2 parents 9a96b8a + e3c1c5b commit 7961595
Show file tree
Hide file tree
Showing 3 changed files with 61 additions and 52 deletions.
4 changes: 4 additions & 0 deletions content/en/docs/tasks/administer-cluster/cluster-upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ weight: 350
This page provides an overview of the steps you should follow to upgrade a
Kubernetes cluster.

The Kubernetes project recommends upgrading to the latest patch releases promptly, and
to ensure that you are running a supported minor release of Kubernetes.
Following this recommendation helps you to to stay secure.

The way that you upgrade a cluster depends on how you initially deployed it
and on any subsequent changes.

Expand Down
105 changes: 53 additions & 52 deletions content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,21 @@ Client certificates generated by [kubeadm](/docs/reference/setup-tools/kubeadm/)
This page explains how to manage certificate renewals with kubeadm. It also covers other tasks related
to kubeadm certificate management.

The Kubernetes project recommends upgrading to the latest patch releases promptly, and
to ensure that you are running a supported minor release of Kubernetes.
Following this recommendation helps you to to stay secure.


## {{% heading "prerequisites" %}}


You should be familiar with [PKI certificates and requirements in Kubernetes](/docs/setup/best-practices/certificates/).

This guide covers the usage of the `openssl` command (used for manual certificate signing,
if you choose that approach), but you can use your preferred tools.

Some of the steps here use `sudo` for administrator access. You can use any equivalent tool.

<!-- steps -->

## Using custom certificates {#custom-certificates}
Expand Down Expand Up @@ -53,6 +63,9 @@ There are various ways to prepare the component credentials when using external
[PKI certificates and requirements](/docs/setup/best-practices/certificates/) includes information
on how to prepare all the required by kubeadm component credentials manually.

This guide covers the usage of the `openssl` command (used for manual certificate signing,
if you choose that approach), but you can use your preferred tools.

### Preparation of credentials by signing CSRs generated by kubeadm

kubeadm can [generate CSR files](#signing-csr) that you can sign manually with tools like
Expand Down Expand Up @@ -87,7 +100,11 @@ Once the credentials are prepared on all nodes, call `kubeadm init` and `kubeadm
join the cluster. kubeadm will use the existing kubeconfig and certificate files under `/etc/kubernetes/`
and its `pki` sub directory.

## Check certificate expiration
## Certificate expiry and management {#check-certificate-expiration}

{{< note >}}
`kubeadm` cannot manage certificates signed by an external CA.
{{< /note >}}

You can use the `check-expiration` subcommand to check when certificates expire:

Expand Down Expand Up @@ -123,20 +140,15 @@ by kubeadm (`admin.conf`, `controller-manager.conf` and `scheduler.conf`).
Additionally, kubeadm informs the user if the certificate is externally managed; in this case, the
user should take care of managing certificate renewal manually/using other tools.

{{< warning >}}
`kubeadm` cannot manage certificates signed by an external CA.
{{< /warning >}}

{{< note >}}
`kubelet.conf` is not included in the list above because kubeadm configures kubelet
The `kubelet.conf` configuration file is not included in the list above because kubeadm
configures kubelet
for [automatic certificate renewal](/docs/tasks/tls/certificate-rotation/)
with rotatable certificates under `/var/lib/kubelet/pki`.
To repair an expired kubelet client certificate see
[Kubelet client certificate rotation fails](/docs/setup/production-environment/tools/kubeadm/troubleshooting-kubeadm/#kubelet-client-cert).
{{< /note >}}

{{< warning >}}
On nodes created with `kubeadm init`, prior to kubeadm version 1.17, there is a
{{< note >}}
On nodes created with `kubeadm init` from versions prior to kubeadm version 1.17, there is a
[bug](https://github.com/kubernetes/kubeadm/issues/1753) where you manually have to modify the
contents of `kubelet.conf`. After `kubeadm init` finishes, you should update `kubelet.conf` to
point to the rotated kubelet client certificates, by replacing `client-certificate-data` and
Expand All @@ -146,7 +158,7 @@ point to the rotated kubelet client certificates, by replacing `client-certifica
client-certificate: /var/lib/kubelet/pki/kubelet-client-current.pem
client-key: /var/lib/kubelet/pki/kubelet-client-current.pem
```
{{< /warning >}}
{{< /note >}}
## Automatic certificate renewal
Expand All @@ -158,26 +170,24 @@ if you don't have specific requirements on certificate renewal and perform Kuber
upgrades regularly (less than 1 year in between each upgrade), kubeadm will take care of keeping
your cluster up to date and reasonably secure.
{{< note >}}
It is a best practice to upgrade your cluster frequently in order to stay secure.
{{< /note >}}
If you have more complex requirements for certificate renewal, you can opt out from the default
behavior by passing `--certificate-renewal=false` to `kubeadm upgrade apply` or to `kubeadm
upgrade node`.

{{< warning >}}
Prior to kubeadm version 1.17 there is a [bug](https://github.com/kubernetes/kubeadm/issues/1818)
where the default value for `--certificate-renewal` is `false` for the `kubeadm upgrade node`
command. In that case, you should explicitly set `--certificate-renewal=true`.
{{< /warning >}}

## Manual certificate renewal

You can renew your certificates manually at any time with the `kubeadm certs renew` command, with the appropriate command line options.
You can renew your certificates manually at any time with the `kubeadm certs renew` command,
with the appropriate command line options. If you are running cluster with a replicated control
plane, this command needs to be executed on all the control-plane nodes.

This command performs the renewal using CA (or front-proxy-CA) certificate and key stored in `/etc/kubernetes/pki`.

`kubeadm certs renew` uses the existing certificates as the authoritative source for attributes
(Common Name, Organization, subject alternative name) and does not rely on the `kubeadm-config`
ConfigMap.
Even so, the Kubernetes project recommends keeping the served certificate and the associated
values in that ConfigMap synchronized, to avoid any risk of confusion.

After running the command you should restart the control plane Pods. This is required since
dynamic certificate reload is currently not supported for all components and certificates.
[Static Pods](/docs/tasks/configure-pod-container/static-pod/) are managed by the local kubelet
Expand All @@ -188,33 +198,25 @@ The kubelet will terminate the Pod if it's no longer in the manifest directory.
You can then move the file back and after another `fileCheckFrequency` period, the kubelet will recreate
the Pod and the certificate renewal for the component can complete.

{{< warning >}}
If you are running an HA cluster, this command needs to be executed on all the control-plane nodes.
{{< /warning >}}

{{< note >}}
`certs renew` uses the existing certificates as the authoritative source for attributes (Common
Name, Organization, SAN, etc.) instead of the `kubeadm-config` ConfigMap. It is strongly recommended
to keep them both in sync.
{{< /note >}}

`kubeadm certs renew` can renew any specific certificate or, with the subcommand `all`, it can renew all of them, as shown below:
`kubeadm certs renew` can renew any specific certificate or, with the subcommand `all`, it can renew all of them:

```shell
# If you are running cluster with a replicated control plane, this command
# needs to be executed on all the control-plane nodes.
kubeadm certs renew all
```

{{< note >}}
### Copying the administrator certificate (optional) {#admin-certificate-copy}

Clusters built with kubeadm often copy the `admin.conf` certificate into
`$HOME/.kube/config`, as instructed in [Creating a cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/).
On such a system, to update the contents of `$HOME/.kube/config`
after renewing the `admin.conf`, you must run the following commands:
after renewing the `admin.conf`, you could run the following commands:

```shell
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
```
{{< /note >}}

## Renew certificates with the Kubernetes certificates API

Expand Down Expand Up @@ -430,16 +432,13 @@ as extra SANs and custom IP addresses must be stored in the same configuration
file and used for all relevant kubeadm commands by passing it as `--config`.

{{< note >}}
This guide will cover the usage of the `openssl` command for singing the CSRs,
but you can use your preferred tools.
{{< /note >}}
This guide uses the default Kubernetes directory `/etc/kubernetes`, which requires
a super user. If you are following this guide and are using directories that you can
write to (typically, this means running `kubeadm` with `--cert-dir` and `--kubeconfig-dir`)
then you can omit the `sudo` command).

{{< note >}}
This guide will use the default Kubernetes directory `/etc/kubernetes`, which requires
a super user. If you are following this guide with permissive directories
(by passing `--cert-dir` and `--kubeconfig-dir`) you can omit the `sudo` command).
But note that the resulted files must be copied to the `/etc/kubernetes` tree,
so that `kubeadm init` or `kubeadm join` will find them.
You must then copy the files that you produced over to within the `/etc/kubernetes`
directory so that `kubeadm init` or `kubeadm join` will find them.
{{< /note >}}

### Preparing CA and service account files
Expand All @@ -460,8 +459,9 @@ private keys) that kubeadm needs for a control plane node.

{{< note >}}
If you are using an external CA, you must generate the same files out of band and manually
copy them to the primary control plane node in `/etc/kubernetes`. Once all CSRs
are signed, you can delete the root CA key (`ca.key`) as noted in the
copy them to the primary control plane node in `/etc/kubernetes`.

Once all CSRs are signed, you can delete the root CA key (`ca.key`) as noted in the
[External CA mode](#external-ca-mode) section.
{{< /note >}}

Expand Down Expand Up @@ -500,9 +500,10 @@ That is because the active kube-controller-manager will be responsible
for signing new kubelet client certificates.

{{< note >}}
Processing the `kubelet.conf.csr` on the primary control plane node
(`kubeadm init`) is required, because that is considered the node that
bootstraps the cluster and a pre-populated `kubelet.conf` is needed.
You must process the `kubelet.conf.csr` file on the primary control plane node
(the host where you originally ran `kubeadm init`). This is because `kubeadm`
considers that as the node that bootstraps the cluster, and a pre-populated
`kubelet.conf` is needed.
{{< /note >}}

#### Control plane nodes
Expand Down Expand Up @@ -539,8 +540,8 @@ the steps for worker nodes entirely.

{{< note >}}
If you are using external CA and already have CA serial number files (`.srl`) for
`openssl` you can copy such files to a kubeadm node where CSRs will be processed.
`.srl` files to copy are `/etc/kubernetes/pki/ca.srl`,
`openssl`, you can copy such files to a kubeadm node where CSRs will be processed.
The `.srl` files to copy are `/etc/kubernetes/pki/ca.srl`,
`/etc/kubernetes/pki/front-proxy-ca.srl` and `/etc/kubernetes/pki/etcd/ca.srl`.
The files can be then moved to a new node where CSR files will be processed.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,10 @@ please refer to following pages instead:
- [Upgrading a kubeadm cluster from {{< skew currentVersionAddMinor -4 >}} to {{< skew currentVersionAddMinor -3 >}}](https://v{{< skew currentVersionAddMinor -3 "-" >}}.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/)
- [Upgrading a kubeadm cluster from {{< skew currentVersionAddMinor -5 >}} to {{< skew currentVersionAddMinor -4 >}}](https://v{{< skew currentVersionAddMinor -4 "-" >}}.docs.kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/)

The Kubernetes project recommends upgrading to the latest patch releases promptly, and
to ensure that you are running a supported minor release of Kubernetes.
Following this recommendation helps you to to stay secure.

The upgrade workflow at high level is the following:

1. Upgrade a primary control plane node.
Expand Down

0 comments on commit 7961595

Please sign in to comment.