Skip to content
This repository has been archived by the owner on May 12, 2021. It is now read-only.

Format update #124

Closed
wants to merge 1 commit into from
+64 −44
Closed

Format update #124

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 64 additions & 44 deletions how-to/how-to-use-k8s-with-cri-containerd-and-kata.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,65 +1,64 @@
# How to use Kata Containers and CRI (containerd plugin) with Kubernetes

This document describes how to set up a single-machine Kubernetes cluster.
The Kubernetes cluster will use the CRI containerd plugin and Kata Containers to launch untrusted workloads.
This document describes how to set up a single-machine Kubernetes cluster. The Kubernetes cluster will use the [CRI containerd plugin](https://github.com/containerd/cri) and [Kata Containers](https://katacontainers.io) to launch untrusted workloads.

## Requirements

## Requirements
- Kubernetes, kubelet, kubeadm
- cri-containerd
- Kata Containers

For information about the supported version of these components see
Kata Containers [versions.yaml](https://github.com/kata-containers/runtime/blob/master/versions.yaml) file.
Note|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the way you've made this stand out, but could you change it to use the standard formatting we use for notes? I'm afraid this hasn't historically been written down, but I've just raised the following to rectify that 😄: #126.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @OGtrilliams - I see your note is still in the original style, not formatted as per the guideline @jodh-intel detailed at https://github.com/kata-containers/documentation/blob/master/Documentation-Requirements.md#notes - could you make that small change please?

----------------- |
|For information about the supported versions of these components, see the Kata Containers [versions.yaml](https://github.com/kata-containers/runtime/blob/master/versions.yaml) file. |


## Install containerd(with CRI plugin enabled)

Follow the instructions from [CRI installation guide](http://github.com/containerd/cri/blob/master/docs/installation.md)

<!---
```bash
# Check if containerd is installed
$ command -v containerd
```
--->

## Install Kata Containers
## Install Kata Containers

Follow the instructions to [install Kata](https://github.com/kata-containers/documentation/blob/master/install/README.md).

<!---
```bash
# Check if kata-runtime is installed
$ command -v kata-runtime
# Check kata is well configured
$ kata-runtime kata-env
```
--->

## Install Kubernetes
Install Kubernetes in your host. See kubeadm [installation](https://kubernetes.io/docs/tasks/tools/install-kubeadm/)
<!---
Install Kubernetes in your host. See [kubeadm installation](https://kubernetes.io/docs/setup/independent/install-kubeadm/)

```bash
# Check if kubadm is installed
$ command -v kubeadm
```
--->

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link on line 39 goes to 404. Also, why are the code blocks commented out?

### Configure containerd to use Kata Containers
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why this is a sub-heading of Install Kubernetes.


The CRI containerd plugin support configuration for two runtime types.
The CRI containerd plugin supports configuration for two runtime types.

- Default runtime: A runtime that is used by default to run workloads.
- Untrusted workload runtime: A runtime that will be used run untrusted workloads.
- **Default runtime:** A runtime that is used by default to run workloads.
- **Untrusted workload runtime:** A runtime that will be used to run untrusted workloads.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be useful to tell the user why they would use the untrusted workload runtime with the CRI plugin. Also, it might be worth explicitly stating that no further configuration is needed to use the CRI plugin with the default runtime if that is the case.

#### Define the Kata runtime as `untrusted_workload_runtime`
#### Define the Kata runtime as `untrusted_workload_runtime`

Configure the Kata runtime for untrusted workload with the [config option](https://github.com/containerd/cri/blob/v1.0.0-rc.0/docs/config.md)
`plugins.cri.containerd.untrusted_workload_runtime`.
Configure the Kata runtime for untrusted workloads with the [config option](https://github.com/containerd/cri/blob/v1.0.0-rc.0/docs/config.md) `plugins.cri.containerd.untrusted_workload_runtime`.

Unless configured otherwise, the default runtime is set to `runc`.

```bash
# Configure containerd to use Kata as untrusted_workload_runtime
$ sudo mkdir -p /etc/containerd/
```
```bash
$ cat << EOT | sudo tee /etc/containerd/config.toml
[plugins]
[plugins.cri.containerd]
Expand All @@ -71,23 +70,33 @@ EOT

### Configure Kubelet to use containerd

In order to allow kubelet use containerd (using CRI interface) configure the service to
point to containerd socket.





In order to allow kubelet to use containerd (using CRI interface), configure the service to point to the `containerd` socket.

- Configure k8s to use containerd

```bash
# Configure k8s to use containerd
$ sudo mkdir -p /etc/systemd/system/kubelet.service.d/
```
```bash
$ cat << EOF | sudo tee /etc/systemd/system/kubelet.service.d/0-containerd.conf
[Service]
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock"
EOF
```
```bash
$ sudo systemctl daemon-reload
```

### Optional: Configure proxy

If you are behind a proxy this script will configure your proxy for docker
kubelet and containerd.

If you are behind a proxy, use the following script to configure your proxy for docker, kubelet, and containerd:


```bash
# Set proxys
Expand All @@ -109,20 +118,30 @@ Environment="HTTPS_PROXY=${https_proxy}"
Environment="NO_PROXY=${no_proxy}"
EOT
done
```
```bash
$ sudo systemctl daemon-reload
```

### Start Kubernetes with kubeadm
### Start Kubernetes with `kubeadm`


- Make sure containerd is up and running

```bash
# Mark sure containerd is up and running
$ sudo systemctl restart containerd
$ sudo systemctl status containerd
```

- Prevent conflicts of docker iptables rules & k8s pod communication

# Prevent docker iptables rules conflict with k8s pod communication
```bash
$ sudo iptables -P FORWARD ACCEPT
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm an iptables numpty - is this basically disabling a bunch of iptables functionality globally? If so, maybe we want to put some sort of note/warning in here to tell users this is a dev/test measure, and for any sort of deployment they might want to do something more 'robust' ?

```

# Start cluster using kubeadm
- Start cluster using `kubeadm`

```bash
$ sudo kubeadm init --skip-preflight-checks \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we merely skipping pre-flight tests to save some time during development?

--cri-socket /run/containerd/containerd.sock --pod-network-cidr=10.244.0.0/16

Expand All @@ -132,22 +151,19 @@ $ sudo -E kubectl get nodes
$ sudo -E kubectl get pods
```

### Install a pod network
Install a pod network plugin is needed to allow pods communicate with each other.
### Install a Pod Network

A pod network plugin is needed to allow pods to communicate with each other.

Install the `flannel` plugin by following the [Using kubeadm to Create a Cluster](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#instructions) guide, starting from the **Installing a pod network** section.

Install flannel plugging, by following the instructions in the section *Installing a pod network*
from [Using kubeadm to Create a Cluster ](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/)
guide.

<!---
```bash
# Install a pod network using flannel
# There is not a programmatic way to know last what flannel commit use
# See https://github.com/coreos/flannel/issues/995
$ sudo -E kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
```
--->


```bash
# wait for pod network
Expand All @@ -166,19 +182,18 @@ $ sudo -E kubectl get pods --all-namespaces | grep dns | grep Running && echo "O

### Allow run pods in master node

By default, the cluster will not schedule pods in the master node to allow that run:
By default, the cluster will not schedule pods in the master node. To enable master node scheduling, run:

```bash
# allow master node run pods
# allow master node to run pods
$ sudo -E kubectl taint nodes --all node-role.kubernetes.io/master-
```


### Create a unstrusted pod using Kata Containers
### Create an unstrusted pod using Kata Containers

By default, all pods are created with the default runtime configured in CRI containerd plugin. If a pod has the `io.kubernetes.cri.untrusted-workload` annotation set to `"true"`, the CRI plugin runs the pod with the [Kata Containers runtime](https://github.com/kata-containers/runtime/blob/master/README.md).

By default, all pods are created with the default runtime configured in CRI containerd plugin.
If a pod has the `io.kubernetes.cri.untrusted-workload annotation` set as
`"true"`, the CRI plugin will run the pod with the Kata Containers runtime.

```bash
# Create untrusted pod configuration
Expand All @@ -195,19 +210,24 @@ spec:
image: nginx
EOT
```

```bash
# Create untrusted pod
$ sudo -E kubectl apply -f nginx-untrusted.yaml

```
```bash
# Check pod is running
$ sudo -E kubectl get pods
```

```bash
# Check qemu is running
$ ps aux | grep qemu
```
### Delete created pod

```bash
### Delete created pod
# Delete pod
$ sudo -E kubectl delete -f nginx-untrusted.yaml
```