Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

doc: update vmm docs for isulad+stratovirt #72

Merged
merged 1 commit into from
Aug 30, 2023
Merged

doc: update vmm docs for isulad+stratovirt #72

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/vmm/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Architecture
Kuasar-sandboxer is a sandboxer plugin of containerd. a sandboxer is a component of containerd for container sandbox lifecycle management. A sandbox should provide a set of task API to containerd for container lifecycle management. the `containerd-task-kuasar` the PID 1 process running in the vm launched by vmm-sandboxer, it provides task API with the vsock connection.
Kuasar-sandboxer is a sandboxer plugin of containerd. a sandboxer is a component of containerd for container sandbox lifecycle management. A sandbox should provide a set of task API to containerd for container lifecycle management. the `vmm-task` the PID 1 process running in the vm launched by vmm-sandboxer, it provides task API with the vsock connection.
![](images/arch.png)

# Installation Guide

## Prerequisites
kuasar should be running on bare metal of x86_64 arch, HostOS should be linux with of 4.8 or higher, with hypervisor installed(qemu support currently, and cloud-hypervisor will be supported soon), Containerd with CRI plugin is also required. rust toolchains is required for compiling the source.
kuasar should be running on bare metal of x86_64 arch, HostOS should be linux of 4.8 or higher, with hypervisor installed(qemu/stratovirt/cloud-hypervisor supported), Containerd with CRI plugin is also required. rust toolchains is required for compiling the source.

## Building from source

Expand Down Expand Up @@ -100,7 +100,7 @@ OPTIONS='--listen /run/vmm-sandboxer.sock --dir /run/kuasar-vmm'

## Get kuasar-vmm service log

Since `vmm-sandboxer` daemon process is run as a `kuasar-vmm` systemd service, vmm-sandboxer's stdout/stderr outputs will be collected by systemd-journald journal service.
Since `vmm-sandboxer` daemon process is running as a `kuasar-vmm` systemd service, vmm-sandboxer's stdout/stderr outputs will be collected by systemd-journald journal service.

So you can use the `journalctl` command to get vmm-sandboxer process log:
```bash
Expand Down Expand Up @@ -144,4 +144,4 @@ $ ncat --vsock 395568061 1025

# Note

Please note that this guide only teach you how to build kuasar from source code, if you want to run the kuasar, cloud hypervisor and virtiofsd are also needed!
Please note that this guide only teach you how to build kuasar from source code, if you want to run the kuasar, hypervisor and virtiofsd are also needed!
93 changes: 43 additions & 50 deletions docs/vmm/how-to-run-kuasar-with-isulad-and-stratovirt.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
$ yum install stratovirt
```

- If you use another Linux distribution OS, you can build the StratoVirt from the source and install it: [Build StratoVirt](https://gitee.com/openeuler/stratovirt/blob/master/README.md#build-stratovirt)
- If you use another Linux distribution OS, you can build the stratovirt from the source and install it: [Build StratoVirt](https://gitee.com/openeuler/stratovirt/blob/master/README.md#build-stratovirt)

- After you build or install the stratovirt package, you can find the following important binary file in your sever:
```bash
Expand All @@ -33,7 +33,7 @@

Kuasar use `docker` or `containerd` container engine to build guest os initrd image, so you need to **make sure `docker` or `containerd` is correctly installed and can pull the image from the dockerhub registries**.

> Tips: `make vmm` build command will download the Rust and Golang packages from the internet network, so you need to provide the `http_proxy` and `https_proxy` environments for the `make all` command.
> Tips: `make vmm` build command will download the Rust and Golang packages from the internet, so you need to provide the `http_proxy` and `https_proxy` environments for the `make all` command.
>
> If a self-signed certificate is used in the `make all` build command execution environment, you may encounter SSL issues with downloading resources from https URL failed. Therefore, you need to provide a CA-signed certificate and copy it into the root directory of the Kuasar project, then rename it as "proxy.crt". In this way, our build script will use the "proxy.crt" certificate to access the https URLs of Rust and Golang installation packages.

Expand Down Expand Up @@ -64,45 +64,29 @@ After installation, you will find the required files in the specified path

## Build and configure iSulad

[iSulad](https://gitee.com/openeuler/iSulad) supports Kuasar with its dev-sandbox branch at the moment. For building iSulad from scratch, please refer to [iSulad build guide](https://gitee.com/openeuler/iSulad/blob/master/docs/build_docs/guide/build_guide.md). Here we only emphasize the difference of the building steps.
[iSulad](https://gitee.com/openeuler/iSulad) supports Kuasar with its master branch at the moment. For building iSulad from scratch, please refer to [iSulad build guide](https://gitee.com/openeuler/iSulad/blob/master/docs/build_docs/guide/build_guide.md). Here we only emphasize the difference of the building steps.

### Build LCR

```bash
$ git clone https://gitee.com/openeuler/lcr.git

$ cd lcr

$ git checkout dev-sandbox

$ mkdir build

$ cd build

$ sudo -E cmake ..

$ sudo -E make -j $(nproc)

$ sudo -E make install
```

### Build iSulad

```bash
$ git clone https://gitee.com/openeuler/iSulad.git

$ cd iSulad

$ git checkout dev-sandbox

$ mkdir build

$ cd build

$ sudo -E cmake .. -D ENABLE_SANDBOX=ON -D ENABLE_SHIM_V2=ON

$ sudo -E cmake .. -DENABLE_CRI_API_V1=ON -DENABLE_SHIM_V2=ON -DENABLE_SANDBOXER=ON
$ sudo make -j

$ sudo -E make install
```

Expand All @@ -111,11 +95,10 @@ Add the following configuration in the iSulad configuration file `/etc/isulad/da
```json
...
"default-sandboxer": "vmm",
"sandboxers": {
"cri-sandboxers": {
"vmm": {
"address": "/run/vmm-sandboxer.sock",
"controller": "proxy",
"protocol": "grpc"
"name": "vmm",
"address": "/run/vmm-sandboxer.sock"
}
},
"cri-runtimes": {
Expand All @@ -128,17 +111,13 @@ Add the following configuration in the iSulad configuration file `/etc/isulad/da

### Build containerd

Sine some code have not been merged into the upstream containerd community, so you need to manually compile the containerd source code in the [kuasar-io/containerd](https://github.com/kuasar-io/containerd.git)
.
Sine some code have not been merged into the upstream containerd community, so you need to manually compile the containerd source code in the [kuasar-io/containerd](https://github.com/kuasar-io/containerd.git).

git clone the codes of containerd fork version from kuasar repository.
```bash
$ git clone https://github.com/kuasar-io/containerd.git

$ cd containerd

$ make bin/containerd

$ install bin/containerd /usr/bin/containerd
```

Expand Down Expand Up @@ -177,12 +156,12 @@ tar -zxvf cni-plugins-linux-arm64-v1.2.0.tgz -C /opt/cni/bin/

```bash
VERSION="v1.15.0" # check latest version in /releases page
wget https://github.com/kubernetes-sigs/cri-tools/releases/download/$VERSION/crictl-$VERSION-linux-amd64.tar.gz
sudo tar zxvf crictl-$VERSION-linux-amd64.tar.gz -C /usr/local/bin
rm -f crictl-$VERSION-linux-amd64.tar.gz
wget https://github.com/kubernetes-sigs/cri-tools/releases/download/$VERSION/crictl-$VERSION-linux-arm64.tar.gz
sudo tar zxvf crictl-$VERSION-linux-arm64.tar.gz -C /usr/local/bin
rm -f crictl-$VERSION-linux-arm64.tar.gz
```

create and the crictl config file in the `/etc/crictl.yaml`
create the crictl config file in the `/etc/crictl.yaml`
```bash
cat /etc/crictl.yaml
# isulad container engine configuraiton
Expand All @@ -202,21 +181,36 @@ timeout: 10
The default config file `/var/lib/kuasar/config_stratovirt.toml` for stratovirt vmm-sandboxer:
```toml
[sandbox]

[hypervisor]
path = "/usr/bin/stratovirt"
machine_type = "virt,mem-share=on"
kernel_path = "/var/lib/kuasar/vmlinux.bin"
image_path = ""
initrd_path = "/var/lib/kuasar/kuasar.initrd"
kernel_params = "task.log_level=debug task.sharefs_type=virtiofs"
vcpus = 1
memory_in_mb = 1024
block_device_driver = "virtio-blk"
# set kuasar log level, (default: info)
log_level = "info"

[hypervisor]
# set stratovirt binary path, (default: stratovirt)
path = "/usr/bin/stratovirt"
# set the type of the analog chip, "virt" for ARM architecture and "q35" for x86 architecture, (default: virt)
machine_type = "virt,mem-share=on"
# set guest kernel path, (default: /var/lib/kuasar/vmlinux.bin)
kernel_path = "/var/lib/kuasar/vmlinux.bin"
# set guest image path, (default: "")
image_path = ""
# set guest initrd path, select either image or image, (default: "")
initrd_path = "/var/lib/kuasar/kuasar.initrd"
# set parameters of the guest kernel, (default: "")
kernel_params = "task.log_level=debug task.sharefs_type=virtiofs"
# set number of vcpus for each sandbox, (default: 1)
vcpus = 1
# set memory size for each sandbox, (default: 1024)
memory_in_mb = 1024
# set the drivers of block devices, (default: virtio-blk)
block_device_driver = "virtio-blk"
# set whether to enable debug mode, (default: false)
debug = true
# enable VM RAM pre-allocation, (default: false)
enable_mem_prealloc = false

[hypervisor.virtiofsd_conf]
path = "/usr/bin/vhost_user_fs
[hypervisor.virtiofsd_conf]
# set vhost_user_fs path, (default: /usr/bin/vhost_user_fs)
path = "/usr/bin/vhost_user_fs"
```

### Start containerd process
Expand All @@ -226,11 +220,10 @@ The default config file `/var/lib/kuasar/config_stratovirt.toml` for stratovirt
$ ENABLE_CRI_SANDBOXES=1 ./bin/containerd
```

### Start StratoVirt vmm-sandboxer process
### Run kuasar-vmm service

```bash
# TODO: create a vmm-sandboxer systemd service
$ RUST_LOG=debug ./bin/vmm-sandboxer --listen /run/vmm-sandboxer.sock --dir /kuasar
$ systemctl start kuasar-vmm
```

### Run pod sandbox with config file
Expand Down
Loading