Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: Add some descriptions for runc sandbox and microvm sandbox #101

Merged
merged 1 commit into from
Dec 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 24 additions & 6 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,19 @@ jobs:
name: wasm-sandboxer ${{ matrix.features }}
path: bin/wasm-sandboxer

runc:
runs-on: ubuntu-22.04
timeout-minutes: 10
steps:
- uses: actions/checkout@v3
- name: Build runc
run: make runc
- name: Upload Artifacts
uses: actions/upload-artifact@v3
with:
name: runc
path: bin/runc-sandboxer

containerd:
runs-on: ubuntu-22.04
timeout-minutes: 30
Expand All @@ -120,6 +133,7 @@ jobs:
- kernel
- quark
- wasm
- runc
- containerd
runs-on: ubuntu-22.04
steps:
Expand All @@ -129,20 +143,24 @@ jobs:
releasever=${{ github.ref }}
releasever="${releasever#refs/tags/}"
bash ./scripts/build/cargo-vendor.sh
mkdir -p /tmp/kuasar-$releasever
cp -r ./* /tmp/kuasar-$releasever
dir="kuasar-${releasever}-vensor"
mkdir -p /tmp/${dir}
cp -r ./* /tmp/${dir}
sudo -E chown -R root:root /tmp/${dir}
mkdir _release
tar -czvf _release/kuasar-$releasever-vendor.tar.gz -C /tmp/ kuasar-$releasever
sudo -E tar -czf _release/${dir}.tar.gz -C /tmp/ ${dir}
- uses: actions/download-artifact@v3
with:
path: _artifacts
- name: Package binaries
run: |
releasever=${{ github.ref }}
releasever="${releasever#refs/tags/}"
mkdir _dist
find _artifacts -type f | xargs -I {} cp {} _dist/
tar -czvf _release/kuasar-$releasever-linux-amd64.tar.gz -C _dist .
dir="kuasar-${releasever}-linux-amd64"
mkdir -p ${dir}
find _artifacts -type f | xargs -I {} cp {} ${dir}/
sudo -E chown -R root:root ${dir}
sudo -E tar -czf _release/${dir}.tar.gz ${dir}
- name: Update Release
uses: softprops/action-gh-release@v1
with:
Expand Down
39 changes: 22 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,18 +20,18 @@ Kuasar is an efficient container runtime that provides cloud-native, all-scenari

# Supported Sandboxes

| Sandboxer | Sandbox | Status |
|------------|------------------|--------------------|
| MicroVM | Cloud Hypervisor | Supported |
| | QEMU | Supported |
| | Firecracker | Planned in 2024 |
| | StratoVirt | Supported |
| Wasm | WasmEdge | Supported |
| | Wasmtime | Supported |
| | Wasmer | Planned in 2024 |
| App Kernel | gVisor | Planned in 2024 |
| | Quark | Supported |
| runC | runC | Planned in 2023 H2 |
| Sandboxer | Sandbox | Status |
|------------|------------------|-----------------|
| MicroVM | Cloud Hypervisor | Supported |
| | QEMU | Supported |
| | Firecracker | Planned in 2024 |
| | StratoVirt | Supported |
| Wasm | WasmEdge | Supported |
| | Wasmtime | Supported |
| | Wasmer | Planned in 2024 |
| App Kernel | gVisor | Planned in 2024 |
| | Quark | Supported |
| runC | runC | Supported |
# Why Kuasar?

In the container world, a sandbox is a technique used to separate container processes from each other, and from the operating system itself. After the introduction of the [Sandbox API](https://github.com/containerd/containerd/issues/4131), sandbox has become the first-class citizen in containerd. With more and more sandbox techniques available in the container world, a management service called "sandboxer" is expected to be proposed.
Expand All @@ -57,7 +57,7 @@ Additionally, Kuasar is also a platform under active development, and we welcome

## MicroVM Sandboxer

In the microVM sandbox scenario, the VM process provides complete virtual machines and Linux kernels based on open-source VMMs such as [Cloud Hypervisor](https://www.cloudhypervisor.org/), [StratoVirt](https://gitee.com/openeuler/stratovirt), [Firecracker](https://firecracker-microvm.github.io/) and [QEMU](https://www.qemu.org/). Hence, the `vmm-sandboxer` of MicroVM sandboxer is responsible for launching VMs and calling APIs, and the `vmm-task`, as the init process in VMs, plays the role of running container processes. The container IO can be exported via vsock or uds.
In the microVM sandbox scenario, the VM process provides complete virtual machines and Linux kernels based on open-source VMMs such as [Cloud Hypervisor](https://www.cloudhypervisor.org/), [StratoVirt](https://gitee.com/openeuler/stratovirt), [Firecracker](https://firecracker-microvm.github.io/) and [QEMU](https://www.qemu.org/). **All of these vm must be running on virtualization-enabled node, otherwise, it won't work!**. Hence, the `vmm-sandboxer` of MicroVM sandboxer is responsible for launching VMs and calling APIs, and the `vmm-task`, as the init process in VMs, plays the role of running container processes. The container IO can be exported via vsock or uds.

The microVM sandboxer avoids the necessity of running shim process on the host, bringing about a cleaner and more manageable architecture with only one process per pod.

Expand All @@ -82,7 +82,11 @@ The `quark-sandboxer` of app kernel sandboxer starts `Qvisor` and an app kernel
The wasm sandbox, such as [WasmEdge](https://wasmedge.org/) or [Wasmtime](https://wasmtime.dev/), is incredibly lightweight, but it may have constraints for some applications at present. The `wasm-sandboxer` and `wasm-task` launch containers within a WebAssembly runtime. Whenever containerd needs to start a container in the sandbox, the `wasm-task` will fork a new process, start a new WasmEdge runtime, and run the Wasm code inside it. All containers within the same pod will share the same Namespace/Cgroup resources with the `wasm-task` process.
![wasm](docs/images/wasm-arch.png)

*Please note that only WasmEdge is currently supported.*
## Runc Sandboxer

Except secure containers, Kuasar also has provide the ability for [runC](https://github.com/opencontainers/runc) containers. In order to generate a seperate namespace, a slight process is created by the `runc-sandboxer` through double folked and then becomes the PID 1. Based on this namespace, the `runc-task` can create the container process and join the namespace. If the container need a private namespace, it will unshare a new namespace for itself.

![wasm](docs/images/runc-arch.png)

# Performance

Expand All @@ -106,7 +110,7 @@ Please also note that Quark requires a Linux kernel version >= 5.15.

### 2. Sandbox

+ MicroVM: To launch a microVM-based sandbox, a hypervisor must be installed on the host.
+ MicroVM: To launch a microVM-based sandbox, a hypervisor must be installed on the **virtualization-enabled** host.
+ It is recommended to install Cloud Hypervisor by default. You can find Cloud Hypervisor installation instructions [here](https://github.com/cloud-hypervisor/cloud-hypervisor/blob/main/docs/building.md).
+ If you want to run kuasar with iSulad container engine and StratoVirt hypervisor, you can refer to this guide [how-to-run-kuasar-with-isulad-and-stratovirt](docs/vmm/how-to-run-kuasar-with-isulad-and-stratovirt.md).
+ Quark: To use Quark, please refer to the installation instructions [here](docs/quark/README.md).
Expand Down Expand Up @@ -154,14 +158,15 @@ Launch the sandboxers by the following commands:
+ For vmm: `nohup vmm-sandboxer --listen /run/vmm-sandboxer.sock --dir /run/kuasar-vmm &`
+ For quark: `nohup quark-sandboxer --listen /run/quark-sandboxer.sock --dir /var/lib/kuasar-quark &`
+ For wasm: `nohup wasm-sandboxer --listen /run/wasm-sandboxer.sock --dir /run/kuasar-wasm &`
+ For runc: `nohup runc-sandboxer --listen /run/runc-sandboxer.sock --dir /run/kuasar-runc &`

## Start Container

Since Kuasar is a low-level container runtime, all interactions should be done via CRI in containerd, such as crictl or Kubernetes. We use crictl as examples:

+ For vmm and quark, run the following scripts:
+ For vmm, quark or runc, run the following scripts:

`examples/run_example_container.sh vmm` or `examples/run_example_container.sh quark`
`examples/run_example_container.sh vmm`, `examples/run_example_container.sh quark` or `examples/run_example_container.sh runc`

+ For wasm: Wasm container needs its own container image so our script has to build and import the container image at first.

Expand Down
Binary file added docs/images/runc-arch.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
6 changes: 6 additions & 0 deletions scripts/build/build-containerd.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ version = 2
disable_apparmor = true

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
runtime_type = "io.containerd.runc.v2"
sandboxer = "runc"

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.vmm]
runtime_type = "io.containerd.kuasar.v1"
Expand All @@ -50,4 +52,8 @@ address = "/run/quark-sandboxer.sock"
[proxy_plugins.wasm]
type = "sandbox"
address = "/run/wasm-sandboxer.sock"

[proxy_plugins.runc]
type = "sandbox"
address = "/run/runc-sandboxer.sock"
EOF
1 change: 1 addition & 0 deletions scripts/build/cargo-vendor.sh
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ directories=(
"vmm/task"
"vmm/common"
"wasm"
"runc"
)

for dir in "${directories[@]}"; do
Expand Down
Loading