Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Using podman run with container runtime crio #7709

Closed
kameshsampath opened this issue Apr 16, 2020 · 14 comments
Closed

Using podman run with container runtime crio #7709

kameshsampath opened this issue Apr 16, 2020 · 14 comments
Labels
co/runtime/crio CRIO related issues kind/documentation Categorizes issue or PR as related to documentation. kind/feature Categorizes issue or PR as related to a new feature. kind/support Categorizes issue or PR as a support question. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete.
Milestone
v1.10.0

Comments

@kameshsampath
Copy link

kameshsampath commented Apr 16, 2020 •
edited
Loading

OS: macOS Catalina

podman version:

Client:
Version:            1.6.3-dev
RemoteAPI Version:  1
Go Version:         go1.12.10
Git Commit:         6c6e78374f5be949d11a8608080c96e2d22ca872
Built:              Wed Oct 30 00:38:11 2019
OS/Arch:            darwin/amd64

Service:
Version:            1.8.2
RemoteAPI Version:  1
Go Version:         go1.11.13
Git Commit:         028e3317eb1494b9b2acba4a0a295df80fae66cc
Built:              Fri Mar 27 00:14:28 2020
OS/Arch:            linux/amd64
``
minikube version:

miniikube version: v1.9.2
commit: 93af9c1


**Steps to reproduce the issue:** 

1.

./minikube start -p podman-node
--container-runtime=cri-o
--network-plugin=cni
--enable-default-cni

2.  `minikube -p podman-node podman-env `
3.  `podman run -it --rm busybox /bin/sh `

The command fails with error

Trying to pull docker.io/library/busybox...
Getting image source signatures
Copying blob sha256:e2334dd9fee4b77e48a8f2d793904118a3acf26f1f2e72a3d79c6cae993e07f0
Copying config sha256:be5888e67be651f1fbb59006f0fd791b44ed3fceaa6323ab4e37d5928874345a
Writing manifest to image destination
Storing signatures
time="2020-04-16T06:10:09Z" level=error msg="error configuring network namespace for container 7fddc83c2a0dd031fab5c084235ed63b73322eb269706ee2ca9c66e726fe1840: Missing CNI default network"



**Full output of `minikube start` command used, if not already included:**

./minikube start -p podman-node
--container-runtime=cri-o
--network-plugin=cni
--enable-default-cni
😄 [podman-node] minikube v1.9.2 on Darwin 10.15.4
▪ MINIKUBE_HOME=/Users/kameshs/MyLabs/minikube/podman/
✨ Automatically selected the hyperkit driver
💾 Downloading driver docker-machine-driver-hyperkit:
> docker-machine-driver-hyperkit.sha256: 65 B / 65 B [---] 100.00% ? p/s 0s
> docker-machine-driver-hyperkit: 10.90 MiB / 10.90 MiB 100.00% 3.03 MiB p
🔑 The 'hyperkit' driver requires elevated permissions. The following commands will be executed:

$ sudo chown root:wheel /Users/kameshs/MyLabs/minikube/podman/.minikube/bin/docker-machine-driver-hyperkit
$ sudo chmod u+s /Users/kameshs/MyLabs/minikube/podman/.minikube/bin/docker-machine-driver-hyperkit

Password:
💿 Downloading VM boot image ...
> minikube-v1.9.0.iso.sha256: 65 B / 65 B [--------------] 100.00% ? p/s 0s
> minikube-v1.9.0.iso: 174.93 MiB / 174.93 MiB [] 100.00% 13.09 MiB p/s 14s
👍 Starting control plane node m01 in cluster podman-node
🔥 Creating hyperkit VM (CPUs=2, Memory=6000MB, Disk=20000MB) ...
🎁 Preparing Kubernetes v1.18.0 on CRI-O 1.17.1 ...
> kubectl.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
> kubelet.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
> kubeadm.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
> kubeadm: 37.96 MiB / 37.96 MiB [---------------] 100.00% 12.88 MiB p/s 3s
> kubectl: 41.98 MiB / 41.98 MiB [----------------] 100.00% 8.90 MiB p/s 5s
> kubelet: 108.01 MiB / 108.01 MiB [-------------] 100.00% 13.39 MiB p/s 9s
🌟 Enabling addons: default-storageclass, storage-provisioner
🏄 Done! kubectl is now configured to use "podman-node"


https://gist.github.com/kameshsampath/4d8d54f520697735ee31c3a0f986c6e2
@medyagh
Copy link
Member

medyagh commented Apr 16, 2020

I personally have not tried podman driver with a remote Podman but I belive u needs to set ENV

@medyagh medyagh added co/podman-driver podman driver issues kind/support Categorizes issue or PR as a support question. labels Apr 16, 2020
@kameshsampath
Copy link
Author

kameshsampath commented Apr 16, 2020 via email

@rhatdan
Copy link

rhatdan commented Apr 16, 2020

@mheon PTAL
This looks like something is setup incorrectly on the server for CNI configuration?

@mheon
Copy link

mheon commented Apr 16, 2020

I suspect that libpod.conf has podman set as the default network name, but there is no CNI network by the name of podman available.

@afbjorklund
Copy link
Collaborator

afbjorklund commented Apr 17, 2020 •
edited
Loading

This is a known shortcoming with the current minikube.iso, we didn't include a podman bridge config because there were some conflicts with the CNI version at the time and we would rather have a working cri-o... We're only using podman to load images, so there was not really any use for podman run. So the feature was omitted from the release 1.8.0, and later not added for 1.9.0 partly because we forgot about it and partly because there are not many using it. Thank you for the reminder, that it might be useful.

We need to install /etc/cni/net.d/87-podman-bridge.conflist with the podman package.

Upgraded CNI in: #6664

Similar earlier issue: #4406

We have upgraded CNI now, so a future version could bring back support for networking.
As a workaround you could run it with --network none or --network host meanwhile ?

Basically the "supported" commands are sudo podman load and sudo podman build.
Most other things would rather use kubectl (or sudo crictl) instead. Including running...

See also #6664 (comment)

What will you be using it for, besides busybox ?

@afbjorklund afbjorklund added co/runtime/crio CRIO related issues kind/feature Categorizes issue or PR as related to a new feature. priority/backlog Higher priority than priority/awaiting-more-evidence. kind/documentation Categorizes issue or PR as related to documentation. and removed co/podman-driver podman driver issues labels Apr 17, 2020
@afbjorklund
Copy link
Collaborator

We could document this better, in the meantime:

$ podman-remote run -it --rm busybox /bin/sh
time="2020-04-17T18:53:41Z" level=error msg="error configuring network namespace for container a5d30bd00be357c19970432e39587db33795ea307b629b67b474a3aa79c59561: Missing CNI default network"
$ podman-remote run -it --rm --network host busybox /bin/sh
/ # exit
exit

It could also use the minikube default CNI network:

$ podman-remote run --rm --network rkt.kubernetes.io busybox ifconfig eth0
eth0      Link encap:Ethernet  HWaddr DA:EA:8C:B2:13:A2  
          inet addr:10.1.0.10  Bcast:10.1.255.255  Mask:255.255.0.0
          inet6 addr: fe80::d8ea:8cff:feb2:13a2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1460  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:222 (222.0 B)

@afbjorklund afbjorklund mentioned this issue Apr 17, 2020
Merged
@kameshsampath
Copy link
Author

thanks @afbjorklund

What will you be using it for, besides busybox ?

I just started with busybox to test, but few thigns that I plan to try are

  • podman build from host and use the image in kubectl deployments
  • ideally using minikube podman-machine instead of docker for mac/windows

@kameshsampath
Copy link
Author

I just tried uninstalling Docker for Mac and aliased docker=podman and it runs super cool. If we happen the default network on in next version it will be super nice as user does not need add the network option.

@afbjorklund
Copy link
Collaborator

afbjorklund commented Apr 18, 2020 •
edited
Loading

  • ideally using minikube podman-machine instead of docker for mac/windows

OK, so basically cut down on the number of VMs used. That is a supported use case indeed.

Minikube normally does provide the docker daemon for this (instead of using containerd), so we could provide a podman alternative (when running with crio) for the users without a local possibility.

The other option would be to run one minikube with crio and one podman-machine, but that has the same downside as minikube with containerd and one docker-machine... More virtual machines.

Another thing is Docker Desktop and reusing an existing VM, but there is no Podman Desktop.

If we happen the default network on in next version it will be super nice as user does not need add the network option.

I have added the PR to add the missing file to the minikube.iso image for the VM

If you could confirm that adding /etc/cni/net.d/87-podman-bridge.conflist fixes it, it would be great!

https://github.com/containers/libpod/blob/v1.8.2/cni/87-podman-bridge.conflist

It should be enough to copy this file to the running machine, since there is no daemon to restart...

@kameshsampath
Copy link
Author

  • ideally using minikube podman-machine instead of docker for mac/windows

OK, so basically cut down on the number of VMs used. That is a supported use case indeed.
Right and wish you have one enviroment for seamless working between Kubernetes and simple container builds/runs. I also personally like the concept of pod in podman

Minikube normally does provide the docker daemon for this (instead of using containerd), so we could provide a podman alternative (when running with crio) for the users without a local possibility.

what I have observed atleast in mac is that crio seem to consume less cpu/memory compared to Docker, this will also help me to use tools like Buildah and Kaniko more naturally.

The other option would be to run one minikube with crio and one podman-machine, but that has the same downside as minikube with containerd and one docker-machine... More virtual machines.

Yes that is what I tried earlier and as you observed two vm' so dropped that paln

Another thing is Docker Desktop and reusing an existing VM, but there is no Podman Desktop.

If we happen the default network on in next version it will be super nice as user does not need add the network option.

I have added the PR to add the missing file to the minikube.iso image for the VM

If you could confirm that adding /etc/cni/net.d/87-podman-bridge.conflist fixes it, it would be great!

https://github.com/containers/libpod/blob/v1.8.2/cni/87-podman-bridge.conflist

It should be enough to copy this file to the running machine, since there is no daemon to restart...

@afbjorklund that works ! I added the 87-podman-bridge.conflist to the /etc/cni/net.d and running the same podman run -it --rm busybox /bin/sh works now without adding the --net=host option

@tstromberg tstromberg added priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. and removed priority/backlog Higher priority than priority/awaiting-more-evidence. labels Apr 20, 2020
@priyawadhwa
Copy link

@kameshsampath would you say your issue has been resolved? If so, it seems like the only thing left to do is document this.

@kameshsampath
Copy link
Author

kameshsampath commented Apr 20, 2020 via email

@afbjorklund afbjorklund added this to the v1.10.0 milestone Apr 21, 2020
@afbjorklund
Copy link
Collaborator

Should be in the v1.10.0-beta.0 already.

@medyagh medyagh modified the milestones: v1.10.0, v1.11.0 May 4, 2020
@afbjorklund afbjorklund modified the milestones: v1.11.0, v1.10.0 May 12, 2020
@afbjorklund
Copy link
Collaborator

Available in v1.10.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
co/runtime/crio CRIO related issues kind/documentation Categorizes issue or PR as related to documentation. kind/feature Categorizes issue or PR as related to a new feature. kind/support Categorizes issue or PR as a support question. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete.
Projects
None yet
Milestone
v1.10.0
Development

No branches or pull requests
7 participants
@tstromberg @kameshsampath @rhatdan @medyagh @mheon @afbjorklund @priyawadhwa