Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run tests with spin-registry-push as well #90

Merged
merged 11 commits into from
May 15, 2024
Merged

Run tests with spin-registry-push as well #90

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
2414b97
update wasm path in test apps and push to local registry
rajatjindal Apr 19, 2024
fd08081
do spin build and push for test apps
rajatjindal Apr 23, 2024
27b4d6f
setup rust/tinygo for building spin apps
rajatjindal Apr 22, 2024
2aa6846
setup spin for running spin build/registry push
rajatjindal Apr 22, 2024
be84b0a
restructure to run both spin-registry-push and docker-build-push tests
rajatjindal Apr 22, 2024
ddffad4
use random forward port for redis
rajatjindal Apr 23, 2024
181f6f8
address review feedback
rajatjindal Apr 29, 2024
7e234fd
use default rust version from toolchain config
rajatjindal May 1, 2024
a3ceefe
rename test-workloads-delete to pod-terminates-test
rajatjindal May 6, 2024
14338b4
restructure to use common-scripts and reduce Makefile complexity
rajatjindal May 6, 2024
4e52d0e
add missing teardown-workloads.sh makefile target
rajatjindal May 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,10 @@ jobs:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
- uses: azure/setup-kubectl@v4
- uses: fermyon/actions/spin/setup@v1
rajatjindal marked this conversation as resolved.
Show resolved Hide resolved
with:
version: "v2.4.2"

- name: Setup build env
run: |
./scripts/setup-linux.sh
Expand Down
33 changes: 25 additions & 8 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,22 +34,39 @@ move-bins:
up:
./scripts/up.sh

./PHONY: teardown-workloads
teardown-workloads:
./scripts/teardown-workloads.sh

./PHONY: pod-status-check
pod-status-check:
./scripts/pod-status-check.sh

./PHONY: workloads
workloads:
./scripts/workloads.sh
./PHONY: deploy-workloads-pushed-using-docker-build-push
deploy-workloads-pushed-using-docker-build-push:
./scripts/deploy-workloads.sh "workloads-pushed-using-docker-build-push"

./PHONY: deploy-workloads-pushed-using-spin-registry-push
deploy-workloads-pushed-using-spin-registry-push:
./scripts/deploy-workloads.sh "workloads-pushed-using-spin-registry-push"

./PHONY: test-workloads-delete
test-workloads-delete:
./scripts/workloads-delete.sh
./PHONY: pod-terminates-test
pod-terminates-test:
./scripts/pod-terminates-test.sh

.PHONY: integration-tests
integration-tests: check-bins move-bins up pod-status-check workloads test-workloads-delete
cargo test -p containerd-shim-spin-tests -- --nocapture
integration-tests: prepare-cluster-and-images integration-docker-build-push-tests integration-spin-registry-push-tests

.PHONY: integration-docker-build-push-tests
integration-docker-build-push-tests:
./scripts/run-integration-tests.sh "workloads-pushed-using-docker-build-push"

.PHONY: integration-spin-registry-push-tests pod-terminates-test
integration-spin-registry-push-tests:
./scripts/run-integration-tests.sh "workloads-pushed-using-spin-registry-push"

.PHONY: prepare-cluster-and-images
prepare-cluster-and-images: check-bins move-bins up pod-status-check
.PHONY: tests/collect-debug-logs
tests/collect-debug-logs:
./scripts/collect-debug-logs.sh 2>&1
Expand Down
8 changes: 4 additions & 4 deletions images/spin-inbound-redis/Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion images/spin-inbound-redis/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@ COPY . .
RUN rustup target add wasm32-wasi && cargo build --target wasm32-wasi --release

FROM scratch
COPY --from=build /opt/build/target/wasm32-wasi/release/spin_inbound_redis.wasm .
COPY --from=build /opt/build/target/wasm32-wasi/release/spin_inbound_redis.wasm ./target/wasm32-wasi/release/spin_inbound_redis.wasm
COPY --from=build /opt/build/spin.toml .
5 changes: 4 additions & 1 deletion images/spin-inbound-redis/spin.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,12 @@ address = "{{ redis_address }}"
component = "hello"

[component.hello]
source = "spin_inbound_redis.wasm"
source = "target/wasm32-wasi/release/spin_inbound_redis.wasm"
allowed_outbound_hosts = ["redis://*:*"]

[component.hello.build]
command = "cargo build --target wasm32-wasi --release"

[component.hello.variables]
redis_address = "{{ redis_address }}"
redis_channel = "{{ redis_channel }}"
Expand Down
4 changes: 2 additions & 2 deletions images/spin-multi-trigger-app/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,6 @@ WORKDIR /opt/build/spin-redis-trigger
RUN cargo build --target wasm32-wasi --release

FROM scratch
COPY --from=build /opt/build/spin-http-trigger/target/wasm32-wasi/release/spin_http_trigger.wasm .
COPY --from=build /opt/build/spin-redis-trigger/target/wasm32-wasi/release/spin_redis_trigger.wasm .
COPY --from=build /opt/build/spin-http-trigger/target/wasm32-wasi/release/spin_http_trigger.wasm spin-http-trigger/target/wasm32-wasi/release/spin_http_trigger.wasm
COPY --from=build /opt/build/spin-redis-trigger/target/wasm32-wasi/release/spin_redis_trigger.wasm spin-redis-trigger/target/wasm32-wasi/release/spin_redis_trigger.wasm
COPY --from=build /opt/build/spin.toml .
16 changes: 8 additions & 8 deletions images/spin-multi-trigger-app/spin-http-trigger/Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

18 changes: 9 additions & 9 deletions images/spin-multi-trigger-app/spin-redis-trigger/Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions images/spin-multi-trigger-app/spin.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@ route = "/..."
component = "spin-http-trigger"

[component.spin-http-trigger]
source = "spin_http_trigger.wasm"
source = "spin-http-trigger/target/wasm32-wasi/release/spin_http_trigger.wasm"
allowed_outbound_hosts = []
[component.spin-http-trigger.build]
command = "cargo build --target wasm32-wasi --release"
command = "cargo build --target wasm32-wasi --release --manifest-path spin-http-trigger/Cargo.toml"
watch = ["src/**/*.rs", "Cargo.toml"]

[application.trigger.redis]
Expand All @@ -25,7 +25,7 @@ channel = "testchannel"
component = "spin-redis-trigger"

[component.spin-redis-trigger]
source = "spin_redis_trigger.wasm"
source = "spin-redis-trigger/target/wasm32-wasi/release/spin_redis_trigger.wasm"
allowed_outbound_hosts = ["redis://redis-service.default.svc.cluster.local:6379"]
[component.spin-redis-trigger.build]
command = "cargo build --target wasm32-wasi --release"
command = "cargo build --target wasm32-wasi --release --manifest-path spin-redis-trigger/Cargo.toml"
8 changes: 4 additions & 4 deletions images/spin-outbound-redis/Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion images/spin-outbound-redis/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,5 @@ COPY . .
RUN rustup target add wasm32-wasi && cargo build --target wasm32-wasi --release

FROM scratch
COPY --from=build /opt/build/target/wasm32-wasi/release/spin_outbound_redis.wasm .
COPY --from=build /opt/build/target/wasm32-wasi/release/spin_outbound_redis.wasm ./target/wasm32-wasi/release/spin_outbound_redis.wasm
COPY --from=build /opt/build/spin.toml .
2 changes: 1 addition & 1 deletion images/spin-outbound-redis/spin.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ route = "hello"
component = "hello"

[component.hello]
source = "spin_outbound_redis.wasm"
source = "target/wasm32-wasi/release/spin_outbound_redis.wasm"
allowed_outbound_hosts = ["redis://*:*"]

[component.hello.variables]
Expand Down
Binary file removed images/spin-outbound-redis/spin_outbound_redis.wasm
View file
Open in desktop
Binary file not shown.
4 changes: 2 additions & 2 deletions images/spin/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,6 @@ RUN curl -LO https://github.com/tinygo-org/tinygo/releases/download/v0.30.0/tiny
RUN cd go-hello && tinygo build -target=wasi -o spin_go_hello.wasm main.go

FROM scratch
COPY --from=build /opt/build/target/wasm32-wasi/release/spin_rust_hello.wasm .
COPY --from=build /opt/build/target/wasm32-wasi/release/spin_rust_hello.wasm ./target/wasm32-wasi/release/spin_rust_hello.wasm
COPY --from=build /opt/build/spin.toml .
COPY --from=build-go /opt/build/go-hello/spin_go_hello.wasm .
COPY --from=build-go /opt/build/go-hello/spin_go_hello.wasm ./go-hello/spin_go_hello.wasm
11 changes: 9 additions & 2 deletions images/spin/spin.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,14 @@ route = "/go-hello"
component = "go-hello"

[component.hello]
source = "spin_rust_hello.wasm"
source = "target/wasm32-wasi/release/spin_rust_hello.wasm"

[component.hello.build]
command = "cargo build --target wasm32-wasi --release"
watch = ["src/**/*.rs", "Cargo.toml"]

[component.go-hello]
source = "spin_go_hello.wasm"
source = "go-hello/spin_go_hello.wasm"

[component.go-hello.build]
command = "cd go-hello && tinygo build -target=wasi -o spin_go_hello.wasm main.go && cd -"
11 changes: 9 additions & 2 deletions scripts/workloads.sh → scripts/deploy-workloads.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,15 @@ fi

# apply the workloads
echo ">>> apply workloads"
kubectl apply -f tests/workloads

kubectl apply -f tests/workloads-common

if [ "$1" == "workloads-pushed-using-spin-registry-push" ]; then
echo "deploying spin apps pushed to registry using 'spin registry push' command"
kubectl apply -f tests/workloads-pushed-using-spin-registry-push
else
echo "deploying spin apps pushed to registry using 'docker build && k3d image import' command"
kubectl apply -f tests/workloads-pushed-using-docker-build-push
fi

# wait for all the pods to be ready
kubectl wait --for=condition=ready --timeout=50s pod --all
Expand Down
2 changes: 1 addition & 1 deletion scripts/down.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ teardown_test() {
docker rmi k3d-shim-test

# remote test folder
rm -r ./test
rm -r ./test || echo "test folder already deleted"

# delete binaries
rm -r "$bin_path"
Expand Down
0 scripts/workloads-delete.sh → scripts/pod-terminates-test.sh
View file
Open in desktop
File renamed without changes.
21 changes: 21 additions & 0 deletions scripts/run-integration-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
#!/bin/bash

set -euo pipefail

## Deploy workloads into k3d cluster
if [ "$1" == "workloads-pushed-using-spin-registry-push" ]; then
make deploy-workloads-pushed-using-spin-registry-push
else
make deploy-workloads-pushed-using-docker-build-push
fi



## Verify pods can be terminated successfully
make pod-terminates-test

## Run integration tests
cargo test -p containerd-shim-spin-tests -- --nocapture

## tests done, cleanup workloads for next test
make teardown-workloads
8 changes: 8 additions & 0 deletions scripts/setup-linux.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,11 @@
sudo apt -y update
sudo apt-get install -y protobuf-compiler libseccomp-dev

echo "setting up rust"
sudo rustup toolchain install --component clippy --component rustfmt --no-self-update
sudo rustup target add wasm32-wasi && sudo rustup target add wasm32-unknown-unknown

## setup tinygo. required for building test spin app
echo "setting up tinygo"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is TinyGo needed? I thought Spin apps are built through Dockerfile which already has TinyGo.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @Mossaka, we need tinygo for building this app: https://github.com/spinkube/containerd-shim-spin/tree/main/images/spin/go-hello

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are using Docker to build images and the Dockerfile in images/spin will install TinyGo. Am I missing anything?

Copy link
Member Author

@rajatjindal rajatjindal May 2, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh apologies for the confusion.

Now that we need to run tests with "spin registry push" version as well, we push the apps in following two ways:

  • docker build && k3d import image (was already part of up.sh)
  • spin build && spin registry push <artifact> (added to up.sh). This runs on the runner machine, and therefore needs tinygo installed.

does that answer your question?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, that makes sense. Maybe as follow-up I wonder if we could run docker build and then extract the component binary out and use it as the artifact for spin registry push. (this will reduce the amount of deps we need to install on the host machine).

wget https://github.com/tinygo-org/tinygo/releases/download/v0.30.0/tinygo_0.30.0_amd64.deb
sudo dpkg -i tinygo_0.30.0_amd64.deb
8 changes: 8 additions & 0 deletions scripts/teardown-workloads.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/bash

set -euo pipefail

kubectl delete -f tests/workloads-common --wait --timeout 60s --ignore-not-found=true
kubectl delete -f tests/workloads-pushed-using-docker-build-push --wait --timeout 60s --ignore-not-found=true
kubectl delete -f tests/workloads-pushed-using-spin-registry-push --wait --timeout 60s --ignore-not-found=true
kubectl wait pod --for=delete -l app=wasm-spin -l app=spin-keyvalue -l app=spin-outbound-redis -l app=spin-multi-trigger-app --timeout 60s
10 changes: 9 additions & 1 deletion scripts/up.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@ IMAGES=("spin-hello-world" "spin-keyvalue" "spin-outbound-redis" "spin-multi-tri
# build the Docker image for the k3d cluster
docker build -t k3d-shim-test "$dockerfile_path"

k3d cluster create "$cluster_name" --image k3d-shim-test --api-port 6551 -p '8082:80@loadbalancer' --agents 2
k3d cluster create "$cluster_name" \
--image k3d-shim-test --api-port 6551 -p '8082:80@loadbalancer' --agents 2 \
--registry-create test-registry:0.0.0.0:5000

kubectl wait --for=condition=ready node --all --timeout=120s

Expand All @@ -22,6 +24,12 @@ for i in "${!DOCKER_IMAGES[@]}"; do
mkdir -p "${OUT_DIRS[$i]}"
docker save -o "${OUT_DIRS[$i]}/img.tar" "${IMAGES[$i]}:latest"
k3d image import "${OUT_DIRS[$i]}/img.tar" -c "$cluster_name"

## also do spin builds and spin registry push
## images pushed as localhost:5000/<namespace>/<app>:<version>
## can be pulled as registry:5000/<namespace>/<app>:<version> from within k3d cluster
spin build -f "./images/${DOCKER_IMAGES[$i]}/spin.toml"
spin registry push "localhost:5000/spin-registry-push/${IMAGES[$i]}:latest" -f "./images/${DOCKER_IMAGES[$i]}/spin.toml" -k
done

sleep 5
Expand Down
Loading
Loading