Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

podman container not responding after reload firewalld #8048

Closed
Masber opened this issue Oct 16, 2020 · 1 comment
Closed

podman container not responding after reload firewalld #8048

Masber opened this issue Oct 16, 2020 · 1 comment
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@Masber
Copy link

Masber commented Oct 16, 2020

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

podman container network breaks after reload firewalld

Steps to reproduce the issue:

  1. Run a continer
    $ podman run --cap-add=IPC_LOCK -d -e "VAULT_ADDR=http://148.187.80.29:8200" -p 8200:8200 -v "/var/local/vault:/vault" -v "/etc/vault.d/:/vault/config/" --name="hashicorp-vault" vault:latest server

  2. Test connectivity to service
    $ curl localhost:8200/v1/sys/seal-status
    {"type":"shamir","initialized":false,"sealed":true,"t":0,"n":0,"progress":0,"nonce":"","version":"","migration":false,"recovery_seal":false,"storage_type":"file"}

  3. Reload firewalld
    $ firewall-cmd --reload
    success

  4. Test connectivity again --> this step should not work and the session should hang/freeze
    $ curl localhost:8200/v1/sys/seal-status
    ^C

  5. Delete container
    $ podman rm -f hashicorp-vault
    ERRO[0000] Error deleting network: running [/usr/sbin/iptables -t nat -D POSTROUTING -s 10.88.0.96 -j CNI-f1d9f8791753a895cccae697 -m comment --comment name: "podman" id: "536f01e839281baedbdb0664585b2c80fb0d18164b86fc50e687725636f0c51c" --wait]: exit status 2: iptables v1.8.4 (nf_tables): Chain 'CNI-f1d9f8791753a895cccae697' does not exist
    Try iptables -h' or 'iptables --help' for more information. ERRO[0000] Error while removing pod from CNI network "podman": running [/usr/sbin/iptables -t nat -D POSTROUTING -s 10.88.0.96 -j CNI-f1d9f8791753a895cccae697 -m comment --comment name: "podman" id: "536f01e839281baedbdb0664585b2c80fb0d18164b86fc50e687725636f0c51c" --wait]: exit status 2: iptables v1.8.4 (nf_tables): Chain 'CNI-f1d9f8791753a895cccae697' does not exist Try iptables -h' or 'iptables --help' for more information.
    ERRO[0000] unable to cleanup network for container 536f01e839281baedbdb0664585b2c80fb0d18164b86fc50e687725636f0c51c: "error tearing down CNI namespace configuration for container 536f01e839281baedbdb0664585b2c80fb0d18164b86fc50e687725636f0c51c: running [/usr/sbin/iptables -t nat -D POSTROUTING -s 10.88.0.96 -j CNI-f1d9f8791753a895cccae697 -m comment --comment name: "podman" id: "536f01e839281baedbdb0664585b2c80fb0d18164b86fc50e687725636f0c51c" --wait]: exit status 2: iptables v1.8.4 (nf_tables): Chain 'CNI-f1d9f8791753a895cccae697' does not exist\nTry `iptables -h' or 'iptables --help' for more information.\n"
    536f01e839281baedbdb0664585b2c80fb0d18164b86fc50e687725636f0c51c

Describe the results you received:

firewalld reload configuration breaks podman network. In my case the console freezes when trying to access a REST service running in podman

Describe the results you expected:

I expect to receive a response from the service

Additional information you deem important (e.g. issue happens only occasionally):

I believe firewall-cmd --reload breaks the network configuration with podman. I can see podman throws an error trying to delete the network after the connectivity stops

Output of podman version:

ERRO[0000] Error deleting network: running [/usr/sbin/iptables -t nat -D POSTROUTING -s 10.88.0.96 -j CNI-f1d9f8791753a895cccae697 -m comment --comment name: "podman" id: "536f01e839281baedbdb0664585b2c80fb0d18164b86fc50e687725636f0c51c" --wait]: exit status 2: iptables v1.8.4 (nf_tables): Chain 'CNI-f1d9f8791753a895cccae697' does not exist
Try `iptables -h' or 'iptables --help' for more information. 
ERRO[0000] Error while removing pod from CNI network "podman": running [/usr/sbin/iptables -t nat -D POSTROUTING -s 10.88.0.96 -j CNI-f1d9f8791753a895cccae697 -m comment --comment name: "podman" id: "536f01e839281baedbdb0664585b2c80fb0d18164b86fc50e687725636f0c51c" --wait]: exit status 2: iptables v1.8.4 (nf_tables): Chain 'CNI-f1d9f8791753a895cccae697' does not exist
Try `iptables -h' or 'iptables --help' for more information. 
ERRO[0000] unable to cleanup network for container 536f01e839281baedbdb0664585b2c80fb0d18164b86fc50e687725636f0c51c: "error tearing down CNI namespace configuration for container 536f01e839281baedbdb0664585b2c80fb0d18164b86fc50e687725636f0c51c: running [/usr/sbin/iptables -t nat -D POSTROUTING -s 10.88.0.96 -j CNI-f1d9f8791753a895cccae697 -m comment --comment name: \"podman\" id: \"536f01e839281baedbdb0664585b2c80fb0d18164b86fc50e687725636f0c51c\" --wait]: exit status 2: iptables v1.8.4 (nf_tables): Chain 'CNI-f1d9f8791753a895cccae697' does not exist\nTry `iptables -h' or 'iptables --help' for more information.\n" 
536f01e839281baedbdb0664585b2c80fb0d18164b86fc50e687725636f0c51c

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.16.1
  cgroupManager: systemd
  cgroupVersion: v1
  conmon:
    package: conmon-2.0.21-1.el8.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.21, commit: fa5f92225c4c95759d10846106c1ebd325966f91-dirty'
  cpus: 1
  distribution:
    distribution: '"centos"'
    version: "8"
  eventLogger: journald
  hostname: hashicorp-vault
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 4.18.0-193.14.2.el8_2.x86_64
  linkmode: dynamic
  memFree: 2653114368
  memTotal: 4102627328
  ociRuntime:
    name: runc
    package: runc-1.0.0-65.rc10.module_el8.2.0+305+5e198a41.x86_64
    path: /usr/bin/runc
    version: 'runc version spec: 1.0.1-dev'
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  rootless: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 98h 33m 42.33s (Approximately 4.08 days)
registries:
  search:
  - registry.access.redhat.com
  - registry.redhat.io
  - docker.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 1
  runRoot: /var/run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 2.0.0
  Built: 1601649039
  BuiltTime: Fri Oct  2 16:30:39 2020
  GitCommit: ""
  GoVersion: go1.13.15
  OsArch: linux/amd64
  Version: 2.1.1

Package info (e.g. output of rpm -q podman or apt list podman):

rpm -q podman
podman-2.1.1-4.el8.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

No

Additional environment details (AWS, VirtualBox, physical, etc.):

@openshift-ci-robot openshift-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Oct 16, 2020
@Luap99
Copy link
Member

Luap99 commented Oct 16, 2020

This is already tracked here: #5431

@Luap99 Luap99 closed this as completed Oct 16, 2020
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 22, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

No branches or pull requests

3 participants