Add `LinuxContainerExecutor` to Wasmtime and WasmEdge shims #156

Mossaka · 2023-06-23T18:50:20Z

This PR adds a LinuxContainerExecutor to both the Wasmtime and Wasmedge shims. It enables the shims to be able to execute containers side by side with wasms.

Some notable changes

It adds a new libcontainer_instance module to containerd-shim-wasm which provides a linux container executor implementation that is shared by both wasmtime and wasmedge shims.
It adds this executor to both the shims.
It adds nginx image to integration tests.

jsturtevant · 2023-06-23T19:16:21Z

It enables the wasmtime shim to be able to execute linux container if the wasm binary was not found on the rootfs.

We may need to revisit this since #147 would mean the wasm binary would definitely not be in the rootfs

Mossaka · 2023-06-23T19:54:43Z

It enables the wasmtime shim to be able to execute linux container if the wasm binary was not found on the rootfs.

We may need to revisit this since #147 would mean the wasm binary would definitely not be in the rootfs

Yeah we can revisit this once #147 is merged.

crates/containerd-shim-wasmtime/src/instance.rs

This commit implements can_handle in wasmtime executor. It checks if the entrypoint binary has wasm as the path extension. If not, the wasmtime executor will not be used. It adds a default executor to the wasmtime shim as a backup so it enables running wasm and containers side-by-side. Add wat extension support in can_handle Signed-off-by: jiaxiao zhou <jiazho@microsoft.com>

this commit adds a python container in the test directory. It also adds a few commands in the makefile to build the python flask app into a container, load into the kind cluster for testing. Eventually the wasmtime tests will have both python container and a wasm contaienr running in a pod. Signed-off-by: jiaxiao zhou <jiazho@microsoft.com>

Signed-off-by: jiaxiao zhou <jiazho@microsoft.com>

The issue says "WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load" Signed-off-by: jiaxiao zhou <jiazho@microsoft.com>

Mossaka · 2023-07-13T16:59:09Z

Could I please have some review comments on this PR? 🙏🙏

crates/containerd-shim-wasmtime/src/executor.rs

crates/containerd-shim-wasmtime/src/instance.rs

Makefile

test/py-flask/Dockerfile

arschles

just one suggestion. otherwise LGTM

crates/containerd-shim-wasmtime/src/instance.rs

crates/containerd-shim-wasmtime/src/executors/container.rs

Signed-off-by: jiaxiao zhou <jiazho@microsoft.com>

cpuguy83 · 2023-08-10T03:15:41Z

Looking through youki a bit, I think can_handle is not part of the executor interface and instead it is expected that we call can_handle from within exec() and return an appropriate error if it can't be used.

0xE282B0 · 2023-08-10T07:02:00Z

@cpuguy83, yes that was refactored lately. There is #209 to adapt to the new interface.

Mossaka · 2023-08-10T16:09:21Z

I think can_handle is not part of the executor interface and instead it is expected that we call can_handle from within exec() and return an appropriate error if it can't be used.

Yeah, as @0xE282B0 mentioned, this was the latest breaking change post v0.1.0 of libcontainer and we are still depending on the released version so we will deal with the refactoring later at #209.

Signed-off-by: jiaxiao zhou <jiazho@microsoft.com>

Mossaka · 2023-08-14T22:32:18Z

I've updated this PR to add linux container executors to both Wasmtime and WasmEdge shims.

this commits adds the default executor to the wasmedge shim similar to how wasmtime shim gets it. It made an API change to the containerd-shim-wasm crate by moving default executor logic from wasmtime shim upstream for sharing code. Signed-off-by: jiaxiao zhou <jiazho@microsoft.com>

Mossaka · 2023-08-14T23:29:36Z

Hey thanks everyone for reviewing this PR! 🙌🥂 I realized it's very difficult to squash commits before merge so I did a squash merge all together to make main branch clean. We can set a policy to only allow squash merge, wdyt? @jsturtevant

Signed-off-by: James Sturtevant <jstur@microsoft.com>

Mossaka force-pushed the issue64 branch 3 times, most recently from 4ff9d3c to c9dee9f Compare June 23, 2023 23:30

Mossaka commented Jun 27, 2023

View reviewed changes

crates/containerd-shim-wasmtime/src/instance.rs Outdated Show resolved Hide resolved

Mossaka force-pushed the issue64 branch 4 times, most recently from 7cb3119 to 30456c3 Compare July 10, 2023 22:29

Mossaka requested review from cpuguy83, devigned, ipuustin and utam0k July 10, 2023 23:01

Mossaka force-pushed the issue64 branch from 30456c3 to 93dd8bb Compare July 11, 2023 21:40

Mossaka force-pushed the issue64 branch from 93dd8bb to 8b49b0c Compare July 11, 2023 22:28

Mossaka added 2 commits July 12, 2023 20:46

feat: add docker buildx to CI

bf71029

Signed-off-by: jiaxiao zhou <jiazho@microsoft.com>

Mossaka force-pushed the issue64 branch from 8c11adb to 2827e2a Compare July 13, 2023 03:45

Mossaka closed this Jul 13, 2023

Mossaka reopened this Jul 13, 2023