Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update gvisor-config.toml #14854

Closed
wants to merge 2 commits into from
Closed

Conversation

fvoznika
Copy link
Contributor

containerd config.toml file has been updated to the version=2 format. Update the gVisor replacement copy accordingly.

Fixes google/gvisor#7877

containerd `config.toml` file has been updated to the `version=2` format. Update the gVisor replacement copy accordingly.
@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Aug 24, 2022

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: fvoznika / name: Fabricio Voznika (a7a73cf)

@k8s-ci-robot k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Aug 24, 2022
@k8s-ci-robot
Copy link
Contributor

Welcome @fvoznika!

It looks like this is your first PR to kubernetes/minikube 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/minikube has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot
Copy link
Contributor

Hi @fvoznika. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 24, 2022
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Aug 24, 2022
@minikube-bot
Copy link
Collaborator

Can one of the admins verify this patch?

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: fvoznika, medyagh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 31, 2022
@medyagh
Copy link
Member

medyagh commented Aug 31, 2022

fvoznika do you mind pasting the output of enalbing gvisor addon before/after this PR ? so we know this works?

@fvoznika
Copy link
Contributor Author

fvoznika commented Sep 1, 2022

I don't have the before at the moment, but it would fail and restart right after: Restarting containerd.... As you can see, it completes the installation now:

fvoznika@minikube:~/git/minikube/out$ minikube addons enable gvisor
💡  gvisor is an addon maintained by Google. For any concerns contact minikube on GitHub.
You can view the list of minikube maintainers at: https://github.com/kubernetes/minikube/blob/master/OWNERS
    ▪ Using image gcr.io/k8s-minikube/gvisor-addon:3
🔎  Verifying gvisor addon...
🌟  The 'gvisor' addon is enabled

fvoznika@minikube:~/git/minikube/out$ ./minikube kubectl -- get pod/gvisor -n kube-system
NAME                               READY   STATUS    RESTARTS   AGE
gvisor                             1/1     Running   0          25s

fvoznika@minikube:~/git/minikube/out$ ./minikube kubectl -- logs pod/gvisor -n kube-system
2022/09/01 22:58:19 Storing default config.toml at /tmp/config.toml
2022/09/01 22:58:19 Copying gvisor-config.toml asset to /node/etc/containerd/config.toml
2022/09/01 22:58:19 gvisor-config.toml asset path: /tmp/gvisor/gvisor-config.toml
2022/09/01 22:58:19 creating /node/etc/containerd/config.toml
2022/09/01 22:58:19 restartContainerd black magic happening
2022/09/01 22:58:19 Stopping rpc-statd.service...
2022/09/01 22:58:20 Restarting containerd...
2022/09/01 22:58:20 Starting rpc-statd...
2022/09/01 22:58:20 containerd restart complete
2022/09/01 22:58:20 gvisor successfully enabled in cluster

@spowelljr
Copy link
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 7, 2022
@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 14854) |
+----------------+----------+---------------------+
| minikube start | 55.7s    | 55.9s               |
| enable ingress | 28.2s    | 28.1s               |
+----------------+----------+---------------------+

Times for minikube start: 55.8s 54.0s 56.1s 56.9s 55.6s
Times for minikube (PR 14854) start: 56.6s 56.1s 55.6s 54.6s 56.4s

Times for minikube ingress: 24.1s 29.2s 29.2s 29.1s 29.6s
Times for minikube (PR 14854) ingress: 28.6s 25.6s 29.1s 29.1s 28.1s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 14854) |
+----------------+----------+---------------------+
| minikube start | 28.4s    | 28.2s               |
| enable ingress | 23.6s    | 23.8s               |
+----------------+----------+---------------------+

Times for minikube ingress: 26.0s 22.5s 23.5s 23.0s 23.0s
Times for minikube (PR 14854) ingress: 23.5s 26.5s 23.5s 22.5s 23.0s

Times for minikube start: 28.3s 28.9s 29.2s 27.9s 27.6s
Times for minikube (PR 14854) start: 27.7s 28.3s 29.1s 28.0s 27.8s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 14854) |
+----------------+----------+---------------------+
| minikube start | 26.2s    | 24.2s               |
| enable ingress | 27.1s    | 27.0s               |
+----------------+----------+---------------------+

Times for minikube ingress: 27.5s 27.0s 27.0s 27.0s 27.0s
Times for minikube (PR 14854) ingress: 27.0s 27.0s 27.0s 27.0s 27.0s

Times for minikube start: 24.4s 34.9s 24.0s 23.6s 24.3s
Times for minikube (PR 14854) start: 24.3s 24.5s 23.1s 24.5s 24.8s

@minikube-pr-bot
Copy link

These are the flake rates of all failed tests.

Environment Failed Tests Flake Rate (%)
KVM_Linux TestGvisorAddon (gopogh) 0.59 (chart)
KVM_Linux TestSkaffold (gopogh) 0.59 (chart)
Docker_Windows TestPause/serial/SecondStartNoReconfiguration (gopogh) 30.00 (chart)
Docker_Windows TestNetworkPlugins/group/enable-default-cni/DNS (gopogh) 39.24 (chart)
Hyper-V_Windows TestPause/serial/SecondStartNoReconfiguration (gopogh) 47.95 (chart)
Docker_Windows TestNoKubernetes/serial/StartNoArgs (gopogh) 50.33 (chart)
Docker_Windows TestStartStop/group/newest-cni/serial/Pause (gopogh) 65.81 (chart)
Docker_Windows TestNetworkPlugins/group/bridge/DNS (gopogh) 68.75 (chart)
Docker_Linux_containerd TestNetworkPlugins/group/calico/Start (gopogh) 69.92 (chart)
Docker_Linux_containerd TestNetworkPlugins/group/enable-default-cni/DNS (gopogh) 77.54 (chart)
Docker_Linux_containerd TestNetworkPlugins/group/bridge/DNS (gopogh) 80.29 (chart)
Docker_Windows TestNetworkPlugins/group/calico/Start (gopogh) 93.55 (chart)
KVM_Linux TestMultiNode/serial/ValidateNameConflict (gopogh) 96.25 (chart)
Docker_macOS TestDownloadOnly/v1.16.0/preload-exists (gopogh) 98.79 (chart)
Hyper-V_Windows TestNoKubernetes/serial/StartWithStopK8s (gopogh) 99.02 (chart)
Docker_Linux_containerd TestKubernetesUpgrade (gopogh) 100.00 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressAddonActivation (gopogh) 100.00 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressAddons (gopogh) 100.00 (chart)
Docker_macOS TestIngressAddonLegacy/serial/ValidateIngressDNSAddonActivation (gopogh) 100.00 (chart)
Docker_macOS TestIngressAddonLegacy/StartLegacyK8sCluster (gopogh) 100.00 (chart)
Docker_macOS TestKubernetesUpgrade (gopogh) 100.00 (chart)
Docker_macOS TestMissingContainerUpgrade (gopogh) 100.00 (chart)
Docker_macOS TestNetworkPlugins/group/kubenet/HairPin (gopogh) 100.00 (chart)
Docker_macOS TestPause/serial/VerifyStatus (gopogh) 100.00 (chart)
Docker_macOS TestPreload (gopogh) 100.00 (chart)
Docker_macOS TestRunningBinaryUpgrade (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/default-k8s-different-port/serial/Pause (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/embed-certs/serial/Pause (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/newest-cni/serial/Pause (gopogh) 100.00 (chart)
Docker_macOS TestStartStop/group/no-preload/serial/Pause (gopogh) 100.00 (chart)
More tests... Continued...

Too many tests failed - See test logs for more details.

To see the flake rates of all tests by environment, click here.

@spowelljr
Copy link
Member

Hi @fvoznika, I see that the GvisorAddon test is failing with:
network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized

@spowelljr spowelljr added the do-not-merge/failing-test Indicates that a PR should not merge because it has a failing test label Sep 12, 2022
@fvoznika
Copy link
Contributor Author

fvoznika commented Sep 21, 2022

Ah, I didn't think that containerd.runtimes.untrusted was still being used. I updated the file to include it...

@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 14854) |
+----------------+----------+---------------------+
| minikube start | 56.3s    | 56.2s               |
| enable ingress | 27.1s    | 26.7s               |
+----------------+----------+---------------------+

Times for minikube start: 54.7s 63.5s 53.5s 54.2s 55.4s
Times for minikube (PR 14854) start: 55.7s 55.8s 56.3s 55.8s 57.3s

Times for minikube (PR 14854) ingress: 25.1s 29.7s 25.7s 28.1s 24.7s
Times for minikube ingress: 29.2s 31.2s 25.1s 25.2s 24.6s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 14854) |
+----------------+----------+---------------------+
| minikube start | 28.3s    | 28.0s               |
| enable ingress | 24.4s    | 22.7s               |
+----------------+----------+---------------------+

Times for minikube start: 29.5s 26.5s 28.8s 30.0s 26.9s
Times for minikube (PR 14854) start: 30.8s 26.7s 27.5s 27.7s 27.3s

Times for minikube ingress: 24.5s 28.0s 23.0s 24.0s 22.5s
Times for minikube (PR 14854) ingress: 22.0s 22.5s 22.5s 23.0s 23.5s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 14854) |
+----------------+----------+---------------------+
| minikube start | 23.3s    | 22.8s               |
| enable ingress | 27.2s    | 27.1s               |
+----------------+----------+---------------------+

Times for minikube start: 23.1s 22.3s 25.8s 22.7s 22.5s
Times for minikube (PR 14854) start: 22.3s 22.9s 22.8s 22.7s 23.6s

Times for minikube ingress: 27.4s 27.0s 27.5s 27.0s 27.0s
Times for minikube (PR 14854) ingress: 27.0s 27.5s 27.0s 27.0s 27.0s

@minikube-pr-bot
Copy link

These are the flake rates of all failed tests.

Environment Failed Tests Flake Rate (%)
KVM_Linux TestGvisorAddon (gopogh) 0.65 (chart)
Docker_Windows TestPause/serial/VerifyDeletedResources (gopogh) 2.17 (chart)
Docker_Linux_containerd TestPause/serial/Start (gopogh) 21.94 (chart)
Docker_Linux_containerd TestOffline (gopogh) 25.81 (chart)
Docker_Linux_containerd TestAddons/Setup (gopogh) 27.74 (chart)
Docker_Linux_containerd TestFunctional/parallel/MountCmd/any-port (gopogh) 27.74 (chart)
Docker_Linux_containerd TestFunctional/parallel/MySQL (gopogh) 27.74 (chart)
Docker_Linux_containerd TestFunctional/parallel/PersistentVolumeClaim (gopogh) 27.74 (chart)
Docker_Linux_containerd TestFunctional/parallel/ServiceCmd (gopogh) 27.74 (chart)
Docker_Linux_containerd TestFunctional/parallel/ServiceCmdConnect (gopogh) 27.74 (chart)
Docker_Linux_containerd TestFunctional/parallel/TunnelCmd/serial/AccessDirect (gopogh) 27.74 (chart)
Docker_Linux_containerd TestFunctional/parallel/TunnelCmd/serial/WaitService/Setup (gopogh) 27.74 (chart)
Docker_Linux_containerd TestFunctional/serial/ExtraConfig (gopogh) 27.74 (chart)
Docker_Linux_containerd TestFunctional/serial/SoftStart (gopogh) 27.74 (chart)
Docker_Linux_containerd TestFunctional/serial/StartWithProxy (gopogh) 27.74 (chart)
Docker_Linux_containerd TestIngressAddonLegacy/serial/ValidateIngressAddonActivation (gopogh) 27.74 (chart)
Docker_Linux_containerd TestIngressAddonLegacy/StartLegacyK8sCluster (gopogh) 27.74 (chart)
Docker_Linux_containerd TestJSONOutput/start/Command (gopogh) 27.74 (chart)
Docker_Linux_containerd TestMultiNode/serial/AddNode (gopogh) 27.74 (chart)
Docker_Linux_containerd TestMultiNode/serial/DeployApp2Nodes (gopogh) 27.74 (chart)
Docker_Linux_containerd TestMultiNode/serial/FreshStart2Nodes (gopogh) 27.74 (chart)
Docker_Linux_containerd TestMultiNode/serial/PingHostFrom2Pods (gopogh) 27.74 (chart)
Docker_Linux_containerd TestMultiNode/serial/ProfileList (gopogh) 27.74 (chart)
Docker_Linux_containerd TestMultiNode/serial/StartAfterStop (gopogh) 27.74 (chart)
Docker_Linux_containerd TestMultiNode/serial/StopNode (gopogh) 27.74 (chart)
Docker_Linux_containerd TestFunctional/parallel/DashboardCmd (gopogh) 28.39 (chart)
Docker_Linux_containerd TestMultiNode/serial/DeleteNode (gopogh) 28.39 (chart)
Docker_Linux_containerd TestMultiNode/serial/RestartKeepsNodes (gopogh) 28.39 (chart)
Docker_Linux_containerd TestPreload (gopogh) 28.39 (chart)
Docker_Linux_containerd TestMultiNode/serial/RestartMultiNode (gopogh) 29.03 (chart)
More tests... Continued...

Too many tests failed - See test logs for more details.

To see the flake rates of all tests by environment, click here.

@fvoznika
Copy link
Contributor Author

Untrusted runtime was removed in containerd 1.4 (containerd/cri#1277). I'm working on a better fix for gvisor-config.toml and will remote the untrusted test as part of that. Let's forget this PR for now and I'll send the other change soon.

@fvoznika fvoznika closed this Sep 21, 2022
@fvoznika
Copy link
Contributor Author

Please see #14996

@fvoznika fvoznika deleted the patch-1 branch September 21, 2022 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/failing-test Indicates that a PR should not merge because it has a failing test ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot get gVisor working on minikube
6 participants