Skip to content
This repository has been archived by the owner on Jun 21, 2023. It is now read-only.

Ability to have the entire docker network's traffic redirected to transparent proxy #85

Closed
lahabana opened this issue Aug 5, 2022 · 1 comment · Fixed by #101 or kumahq/kuma#5284
Assignees
Labels
kind/feature New feature triage/accepted The issue was reviewed and is complete enough to start working on it

Comments

@lahabana
Copy link
Contributor

lahabana commented Aug 5, 2022

Description

Sometimes someone want to run the proxy directly on the host machine and have workloads inside containers.

Have to add to the prerouting chain 2 rules per network:

  1. Redirect dns (udp port:53) traffic coming from the interface of the docker network to the sidecar (port 15053)
  2. Redirect tcp traffic not going to an ip of this docker network (there's a cidr for each docker network) to the sidecar (port 15001)
sudo iptables -t nat -A PREROUTING -i docker0 -p udp -m udp --dport 53 -j REDIRECT --to-ports 15053
sudo iptables -t nat -A PREROUTING -i br-+ -p udp -m udp --dport 53 -j REDIRECT --to-ports 15053
sudo iptables -t nat -A PREROUTING ! -d 172.17.0.0/16 -i docker0 -p tcp -j REDIRECT --to-ports 15001
sudo iptables -t nat -A PREROUTING ! -d 172.18.0.0/16 -i br-+ -p tcp -j REDIRECT --to-ports 15001

It would be nice to have this exposed as a parameter in transparent-proxy like:

install-transparent-proxy --vnet <interfaceName>:<cidr>

For example in the case above:

install-transparent-proxy --vnet docker0:172.17.0.0/16 br-68b3f774cd5c:172.18.0.0/16

Info: the docker interface for non default docker network br-<subsetOfTheId>. docker network inspect is helpful to find the id of a network.

Final result:

*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
:MESH_INBOUND - [0:0]
:MESH_IN_REDIRECT - [0:0]
:MESH_OUTPUT - [0:0]
:MESH_REDIRECT - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A PREROUTING -i docker0 -p udp -m udp --dport 53 -j REDIRECT --to-ports 15053
-A PREROUTING -i br-+ -p udp -m udp --dport 53 -j REDIRECT --to-ports 15053
-A PREROUTING ! -d 172.17.0.0/16 -i docker0 -p tcp -j REDIRECT --to-ports 15001
-A PREROUTING ! -d 172.18.0.0/16 -i br-+ -p tcp -j REDIRECT --to-ports 15001
-A PREROUTING -p tcp -j MESH_INBOUND
-A OUTPUT -p udp -m udp --dport 53 -m owner --uid-owner 1003 -j RETURN
-A OUTPUT -p udp -m udp --dport 53 -m owner --uid-owner 102 -j RETURN
-A OUTPUT -p udp -m udp --dport 53 -m owner --gid-owner 1004 -j RETURN
-A OUTPUT -p udp -m udp --dport 53 -m owner --gid-owner 103 -j RETURN
-A OUTPUT -p udp -m udp --dport 53 -j REDIRECT --to-ports 15053
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT -p tcp -j MESH_OUTPUT
-A POSTROUTING -s 172.18.0.0/16 ! -o br-68b3f774cd5c -j MASQUERADE
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.17.0.2/32 -d 172.17.0.2/32 -p tcp -m tcp --dport 8000 -j MASQUERADE
-A POSTROUTING -s 172.17.0.2/32 -j MASQUERADE
-A DOCKER -i br-68b3f774cd5c -j RETURN
-A DOCKER -i docker0 -j RETURN
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 8000 -j DNAT --to-destination 172.17.0.2:8000
-A MESH_INBOUND -p tcp -m tcp --dport 15008 -j RETURN
-A MESH_INBOUND -p tcp -m tcp --dport 22 -j RETURN
-A MESH_INBOUND -p tcp -j MESH_IN_REDIRECT
-A MESH_IN_REDIRECT -p tcp -j REDIRECT --to-ports 15006
-A MESH_OUTPUT -s 127.0.0.6/32 -o lo -j RETURN
-A MESH_OUTPUT ! -d 127.0.0.1/32 -o lo -p tcp -m tcp ! --dport 53 -m owner --uid-owner 1003 -j MESH_IN_REDIRECT
-A MESH_OUTPUT -o lo -p tcp -m tcp ! --dport 53 -m owner ! --uid-owner 1003 -j RETURN
-A MESH_OUTPUT -m owner --uid-owner 1003 -j RETURN
-A MESH_OUTPUT ! -d 127.0.0.1/32 -o lo -m owner --gid-owner 1004 -j MESH_IN_REDIRECT
-A MESH_OUTPUT -o lo -p tcp -m tcp ! --dport 53 -m owner ! --gid-owner 1004 -j RETURN
-A MESH_OUTPUT -m owner --gid-owner 1004 -j RETURN
-A MESH_OUTPUT -p tcp -m tcp --dport 53 -j REDIRECT --to-ports 15053
-A MESH_OUTPUT -d 127.0.0.1/32 -j RETURN
-A MESH_OUTPUT -j MESH_REDIRECT
-A MESH_REDIRECT -p tcp -j REDIRECT --to-ports 15001
COMMIT
@lahabana lahabana added kind/feature New feature triage/pending This issue will be looked at on the next triage meeting triage/accepted The issue was reviewed and is complete enough to start working on it and removed triage/pending This issue will be looked at on the next triage meeting labels Aug 5, 2022
@lahabana
Copy link
Contributor Author

lahabana commented Aug 5, 2022

Full iptables after:

sudo iptables -t nat -vnL
Chain PREROUTING (policy ACCEPT 116 packets, 6795 bytes)
 pkts bytes target     prot opt in     out     source               destination
 3640  215K LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0            LOG flags 0 level 4
 2689  159K DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL
    0     0 REDIRECT   udp  --  docker0 *       0.0.0.0/0            0.0.0.0/0            udp dpt:53 redir ports 15053
   12   670 REDIRECT   udp  --  br-+   *       0.0.0.0/0            0.0.0.0/0            udp dpt:53 redir ports 15053
    0     0 REDIRECT   tcp  --  docker0 *       0.0.0.0/0           !172.17.0.0/16        redir ports 15001
    6   360 REDIRECT   tcp  --  br-+   *       0.0.0.0/0           !172.18.0.0/16        redir ports 15001
   10   504 MESH_INBOUND  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 1321 packets, 77819 bytes)
 pkts bytes target     prot opt in     out     source               destination
  278 17542 RETURN     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:53 owner UID match 1003
    0     0 RETURN     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:53 owner UID match 102
    0     0 RETURN     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:53 owner GID match 1004
    0     0 RETURN     udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:53 owner GID match 103
  382 26940 REDIRECT   udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:53 redir ports 15053
  303 22794 REDIRECT   udp  --  *      *       0.0.0.0/0            0.0.0.0/0            udp dpt:53 redir ports 15053
   50  3000 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL
  556 33360 MESH_OUTPUT  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain POSTROUTING (policy ACCEPT 4086 packets, 247K bytes)
 pkts bytes target     prot opt in     out     source               destination
   15  1005 MASQUERADE  all  --  *      !br-68b3f774cd5c  172.18.0.0/16        0.0.0.0/0
 7716  591K MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0
    0     0 MASQUERADE  tcp  --  *      *       172.17.0.2           172.17.0.2           tcp dpt:8000
    0     0 MASQUERADE  all  --  *      *       172.17.0.2           0.0.0.0/0

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all  --  br-68b3f774cd5c *       0.0.0.0/0            0.0.0.0/0
    0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0
 2459  148K DNAT       tcp  --  !docker0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8000 to:172.17.0.2:8000

Chain MESH_INBOUND (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:15008
  166  9148 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22
 2726  163K MESH_IN_REDIRECT  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MESH_IN_REDIRECT (3 references)
 pkts bytes target     prot opt in     out     source               destination
 2728  163K REDIRECT   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            redir ports 15006

Chain MESH_OUTPUT (1 references)
 pkts bytes target     prot opt in     out     source               destination
    8   480 RETURN     all  --  *      lo      127.0.0.6            0.0.0.0/0
    2   120 MESH_IN_REDIRECT  tcp  --  *      lo      0.0.0.0/0           !127.0.0.1            tcp dpt:!53 owner UID match 1003
   13   780 RETURN     tcp  --  *      lo      0.0.0.0/0            0.0.0.0/0            tcp dpt:!53 ! owner UID match 1003
  329 19740 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            owner UID match 1003
    0     0 MESH_IN_REDIRECT  all  --  *      lo      0.0.0.0/0           !127.0.0.1            owner GID match 1004
    0     0 RETURN     tcp  --  *      lo      0.0.0.0/0            0.0.0.0/0            tcp dpt:!53 ! owner GID match 1004
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            owner GID match 1004
    0     0 REDIRECT   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:53 redir ports 15053
    0     0 RETURN     all  --  *      *       0.0.0.0/0            127.0.0.1
  204 12240 MESH_REDIRECT  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MESH_REDIRECT (1 references)
 pkts bytes target     prot opt in     out     source               destination
  302 18120 REDIRECT   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            redir ports 15001

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/feature New feature triage/accepted The issue was reviewed and is complete enough to start working on it
Projects
None yet
2 participants