Skip to content

Commit

Permalink
Combine outgoing and ICC iptables rules
Browse files Browse the repository at this point in the history 
Rather than:

  ACCEPT -i <bridge> -o <bridge>   // icc=true
  ACCEPT -i <bridge> ! -o <bridge> // outgoing

Do:

  ACCEPT -i <bridge>  // icc=true and outgoing

For ICC=false, continue to do:

  DROP -i <bridge> -o <bridge>
  ACCEPT -i <bridge> ! -o <bridge>

Signed-off-by: Rob Murray <rob.murray@docker.com>
  • Loading branch information
@robmry
robmry committed Dec 17, 2024
1 parent 0f259dd commit df3c78d
Show file tree
Hide file tree
Showing 14 changed files with 167 additions and 141 deletions.
35 changes: 11 additions & 24 deletions integration/network/bridge/iptablesdoc/generated/new-daemon.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,7 @@ Table `filter`:
2 0 0 ACCEPT 0 -- * * 0.0.0.0/0 0.0.0.0/0 match-set docker-ext-bridges-v4 dst ctstate RELATED,ESTABLISHED
3 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0
4 0 0 DOCKER 0 -- * * 0.0.0.0/0 0.0.0.0/0 match-set docker-ext-bridges-v4 dst
5 0 0 ACCEPT 0 -- docker0 docker0 0.0.0.0/0 0.0.0.0/0
6 0 0 ACCEPT 0 -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
5 0 0 ACCEPT 0 -- docker0 * 0.0.0.0/0 0.0.0.0/0

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
Expand Down Expand Up @@ -51,8 +50,7 @@ Table `filter`:
-A FORWARD -m set --match-set docker-ext-bridges-v4 dst -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -m set --match-set docker-ext-bridges-v4 dst -j DOCKER
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -j ACCEPT
-A DOCKER ! -i docker0 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
Expand Down Expand Up @@ -81,36 +79,25 @@ The FORWARD chain rules are numbered in the output above, they are:
2. Early ACCEPT for any RELATED,ESTABLISHED traffic to a docker bridge. This rule
matches against an `ipset` called `docker-ext-bridges-v4` (`v6` for IPv6). The
set contains the CIDR address of each docker network, and it is updated as networks
are created and deleted.
So, this rule could be set up during bridge driver initialisation. But, it is
currently set up when a network is created, in [setupIPTables][11].
are created and deleted. This rule is created during driver initialisation, in
`setupIPChains`.
3. Unconditional jump to DOCKER-ISOLATION-STAGE-1.
Set up during network creation by [setupIPTables][12], which ensures it appears
after the jump to DOCKER-USER (by deleting it and re-creating, while traffic
may be running for other networks).
Also created during driver initialisation, in `setupIPChains`.
4. Jump to DOCKER, for any packet destined for any bridge network, identified by
matching against the `docker-ext-bridge-v[46]` set. Added when the network is
created, in [setupIPTables][13].
matching against the `docker-ext-bridge-v[46]` set.
Also created during driver initialisation, in `setupIPChains`.
The DOCKER chain implements per-port/protocol filtering for each container.
5. ACCEPT packets flowing between containers within a network, because by default
container isolation is disabled. Also set up when the network is created, in
[setIcc][15].
6. ACCEPT any packet leaving a network, also set up when the network is created, in
[setupIPTablesInternal][14].

5. ACCEPT any packet leaving a network, set up when the network is created, in
`setupIPTablesInternal`. Note that this accepts any packet leaving the
network that's made it through the DOCKER and isolation chains, whether the
destination is external or another network.

[10]: https://github.com/moby/moby/blob/e05848c0025b67a16aaafa8cdff95d5e2c064105/libnetwork/firewall_linux.go#L50
[11]: https://github.com/robmry/moby/blob/52c89d467fc5326149e4bbb8903d23589b66ff0d/libnetwork/drivers/bridge/setup_ip_tables_linux.go#L230-L232
[12]: https://github.com/robmry/moby/blob/52c89d467fc5326149e4bbb8903d23589b66ff0d/libnetwork/drivers/bridge/setup_ip_tables_linux.go#L227-L229
[13]: https://github.com/robmry/moby/blob/52c89d467fc5326149e4bbb8903d23589b66ff0d/libnetwork/drivers/bridge/setup_ip_tables_linux.go#L223-L226
[14]: https://github.com/moby/moby/blob/333cfa640239153477bf635a8131734d0e9d099d/libnetwork/drivers/bridge/setup_ip_tables_linux.go#L264
[15]: https://github.com/moby/moby/blob/333cfa640239153477bf635a8131734d0e9d099d/libnetwork/drivers/bridge/setup_ip_tables_linux.go#L343

_With ICC enabled 5 and 6 could be combined, to ACCEPT anything from the bridge.
But, when ICC is disabled, rule 6 is DROP, so it would need to be placed before
rule 5. Because the rules are generated in different places, that's a slightly
bigger change than it should be._

The DOCKER chain has a single DROP rule for the bridge network, to drop any
packets routed to the network that have not originated in the network. Added by
[setDefaultForwardRule][21].
Expand Down
10 changes: 4 additions & 6 deletions integration/network/bridge/iptablesdoc/generated/swarm-portmap.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,9 @@ The filter table is:
3 0 0 ACCEPT 0 -- * * 0.0.0.0/0 0.0.0.0/0 match-set docker-ext-bridges-v4 dst ctstate RELATED,ESTABLISHED
4 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0
5 0 0 DOCKER 0 -- * * 0.0.0.0/0 0.0.0.0/0 match-set docker-ext-bridges-v4 dst
6 0 0 ACCEPT 0 -- docker0 docker0 0.0.0.0/0 0.0.0.0/0
7 0 0 ACCEPT 0 -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
8 0 0 DROP 0 -- docker_gwbridge docker_gwbridge 0.0.0.0/0 0.0.0.0/0
9 0 0 ACCEPT 0 -- docker_gwbridge !docker_gwbridge 0.0.0.0/0 0.0.0.0/0
6 0 0 ACCEPT 0 -- docker0 * 0.0.0.0/0 0.0.0.0/0
7 0 0 DROP 0 -- docker_gwbridge docker_gwbridge 0.0.0.0/0 0.0.0.0/0
8 0 0 ACCEPT 0 -- docker_gwbridge !docker_gwbridge 0.0.0.0/0 0.0.0.0/0

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
Expand Down Expand Up @@ -66,8 +65,7 @@ The filter table is:
-A FORWARD -m set --match-set docker-ext-bridges-v4 dst -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -m set --match-set docker-ext-bridges-v4 dst -j DOCKER
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -j ACCEPT
-A FORWARD -i docker_gwbridge -o docker_gwbridge -j DROP
-A FORWARD -i docker_gwbridge ! -o docker_gwbridge -j ACCEPT
-A DOCKER ! -i docker0 -o docker0 -j DROP
Expand Down
48 changes: 33 additions & 15 deletions integration/network/bridge/iptablesdoc/generated/usernet-internal.md
View file
Original file line number Diff line number Diff line change
@@ -1,12 +1,22 @@
## Container on a user-defined --internal network
## Containers on user-defined --internal networks

These are the rules for two containers on different `--internal` networks, with and
without inter-container communication.

Equivalent to:

docker network create \
-o com.docker.network.bridge.name=bridge1 \
-o com.docker.network.bridge.name=bridgeICC \
--internal \
--subnet 192.0.2.0/24 --gateway 192.0.2.1 bridge1
docker run --network bridge1 --name c1 busybox
docker run --network bridgeICC --name c1 busybox

docker network create \
-o com.docker.network.bridge.name=bridgeNoICC \
-o com.docker.network.bridge.enable_icc=true \
--internal \
--subnet 198.51.100.0/24 --gateway 198.51.100.1 bridge1
docker run --network bridgeNoICC --name c1 busybox

The filter table is updated as follows:

Expand All @@ -19,9 +29,9 @@ The filter table is updated as follows:
2 0 0 ACCEPT 0 -- * * 0.0.0.0/0 0.0.0.0/0 match-set docker-ext-bridges-v4 dst ctstate RELATED,ESTABLISHED
3 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0
4 0 0 DOCKER 0 -- * * 0.0.0.0/0 0.0.0.0/0 match-set docker-ext-bridges-v4 dst
5 0 0 ACCEPT 0 -- docker0 docker0 0.0.0.0/0 0.0.0.0/0
6 0 0 ACCEPT 0 -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
7 0 0 ACCEPT 0 -- bridge1 bridge1 0.0.0.0/0 0.0.0.0/0
5 0 0 ACCEPT 0 -- docker0 * 0.0.0.0/0 0.0.0.0/0
6 0 0 ACCEPT 0 -- bridgeICC bridgeICC 0.0.0.0/0 0.0.0.0/0
7 0 0 DROP 0 -- bridgeNoICC bridgeNoICC 0.0.0.0/0 0.0.0.0/0

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
Expand All @@ -32,9 +42,11 @@ The filter table is updated as follows:

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
num pkts bytes target prot opt in out source destination
1 0 0 DROP 0 -- * bridge1 !192.0.2.0/24 0.0.0.0/0
2 0 0 DROP 0 -- bridge1 * 0.0.0.0/0 !192.0.2.0/24
3 0 0 DOCKER-ISOLATION-STAGE-2 0 -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
1 0 0 DROP 0 -- * bridgeNoICC !198.51.100.0/24 0.0.0.0/0
2 0 0 DROP 0 -- bridgeNoICC * 0.0.0.0/0 !198.51.100.0/24
3 0 0 DROP 0 -- * bridgeICC !192.0.2.0/24 0.0.0.0/0
4 0 0 DROP 0 -- bridgeICC * 0.0.0.0/0 !192.0.2.0/24
5 0 0 DOCKER-ISOLATION-STAGE-2 0 -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
num pkts bytes target prot opt in out source destination
Expand All @@ -59,12 +71,14 @@ The filter table is updated as follows:
-A FORWARD -m set --match-set docker-ext-bridges-v4 dst -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -m set --match-set docker-ext-bridges-v4 dst -j DOCKER
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i bridge1 -o bridge1 -j ACCEPT
-A FORWARD -i docker0 -j ACCEPT
-A FORWARD -i bridgeICC -o bridgeICC -j ACCEPT
-A FORWARD -i bridgeNoICC -o bridgeNoICC -j DROP
-A DOCKER ! -i docker0 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -s 192.0.2.0/24 -o bridge1 -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d 192.0.2.0/24 -i bridge1 -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -s 198.51.100.0/24 -o bridgeNoICC -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d 198.51.100.0/24 -i bridgeNoICC -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -s 192.0.2.0/24 -o bridgeICC -j DROP
-A DOCKER-ISOLATION-STAGE-1 ! -d 192.0.2.0/24 -i bridgeICC -j DROP
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-USER -j RETURN
Expand All @@ -74,14 +88,18 @@ The filter table is updated as follows:

By comparison with the [network with external access][1]:

- In the FORWARD chain, there is no ACCEPT rule for outgoing packets `-i bridge1 -o ! bridge1`.
- In the FORWARD chain, there is no ACCEPT rule for outgoing packets (`-i bridgeINC`).
- There are no rules for this network in the DOCKER chain.
- In DOCKER-ISOLATION-STAGE-1:
- Rule 1 drops any packet routed to the network that does not have a source address in the network's subnet.
- Rule 2 drops any packet routed out of the network that does not have a dest address in the network's subnet.
- There is no jump to DOCKER-ISOLATION-STAGE-2.
- DOCKER-ISOLATION-STAGE-2 is unused.

The only difference between `bridgeICC` and `bridgeNoICC` is the rule in the FORWARD
chain. To enable ICC, the rule for packets looping through the bridge is ACCEPT. For
no-ICC it's DROP.

[1]: usernet-portmap.md

And the corresponding nat table:
Expand Down
12 changes: 4 additions & 8 deletions integration/network/bridge/iptablesdoc/generated/usernet-portmap-natunprot.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,8 @@ The filter table is:
2 0 0 ACCEPT 0 -- * * 0.0.0.0/0 0.0.0.0/0 match-set docker-ext-bridges-v4 dst ctstate RELATED,ESTABLISHED
3 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0
4 0 0 DOCKER 0 -- * * 0.0.0.0/0 0.0.0.0/0 match-set docker-ext-bridges-v4 dst
5 0 0 ACCEPT 0 -- docker0 docker0 0.0.0.0/0 0.0.0.0/0
6 0 0 ACCEPT 0 -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
7 0 0 ACCEPT 0 -- bridge1 bridge1 0.0.0.0/0 0.0.0.0/0
8 0 0 ACCEPT 0 -- bridge1 !bridge1 0.0.0.0/0 0.0.0.0/0
5 0 0 ACCEPT 0 -- docker0 * 0.0.0.0/0 0.0.0.0/0
6 0 0 ACCEPT 0 -- bridge1 * 0.0.0.0/0 0.0.0.0/0

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
Expand Down Expand Up @@ -61,10 +59,8 @@ The filter table is:
-A FORWARD -m set --match-set docker-ext-bridges-v4 dst -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -m set --match-set docker-ext-bridges-v4 dst -j DOCKER
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i bridge1 -o bridge1 -j ACCEPT
-A FORWARD -i bridge1 ! -o bridge1 -j ACCEPT
-A FORWARD -i docker0 -j ACCEPT
-A FORWARD -i bridge1 -j ACCEPT
-A DOCKER ! -i docker0 -o docker0 -j DROP
-A DOCKER ! -i bridge1 -o bridge1 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
Expand Down
16 changes: 7 additions & 9 deletions integration/network/bridge/iptablesdoc/generated/usernet-portmap-noicc.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,9 @@ The filter table is:
2 0 0 ACCEPT 0 -- * * 0.0.0.0/0 0.0.0.0/0 match-set docker-ext-bridges-v4 dst ctstate RELATED,ESTABLISHED
3 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0
4 0 0 DOCKER 0 -- * * 0.0.0.0/0 0.0.0.0/0 match-set docker-ext-bridges-v4 dst
5 0 0 ACCEPT 0 -- docker0 docker0 0.0.0.0/0 0.0.0.0/0
6 0 0 ACCEPT 0 -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
7 0 0 DROP 0 -- bridge1 bridge1 0.0.0.0/0 0.0.0.0/0
8 0 0 ACCEPT 0 -- bridge1 !bridge1 0.0.0.0/0 0.0.0.0/0
5 0 0 ACCEPT 0 -- docker0 * 0.0.0.0/0 0.0.0.0/0
6 0 0 DROP 0 -- bridge1 bridge1 0.0.0.0/0 0.0.0.0/0
7 0 0 ACCEPT 0 -- bridge1 !bridge1 0.0.0.0/0 0.0.0.0/0

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
Expand Down Expand Up @@ -62,8 +61,7 @@ The filter table is:
-A FORWARD -m set --match-set docker-ext-bridges-v4 dst -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -m set --match-set docker-ext-bridges-v4 dst -j DOCKER
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -j ACCEPT
-A FORWARD -i bridge1 -o bridge1 -j DROP
-A FORWARD -i bridge1 ! -o bridge1 -j ACCEPT
-A DOCKER -d 192.0.2.2/32 ! -i bridge1 -o bridge1 -p tcp -m tcp --dport 80 -j ACCEPT
Expand All @@ -80,11 +78,11 @@ The filter table is:

By comparison with [ICC=true][1]:

- Rule 7 in the FORWARD chain replaces an ACCEPT rule that would have followed rule 5, matching the same packets.
- Added in [setIcc][2]
- Rules 6 and 7 replace the accept rule for outgoing packets.
- Rule 6, added by `setIcc`, drops any packet sent from the internal network to itself.
- Rule 7, added by `setupIPTablesInternal` accepts any other outgoing packet.

[1]: usernet-portmap.md
[2]: https://github.com/moby/moby/blob/333cfa640239153477bf635a8131734d0e9d099d/libnetwork/drivers/bridge/setup_ip_tables_linux.go#L344

And the corresponding nat table:

Expand Down
12 changes: 4 additions & 8 deletions integration/network/bridge/iptablesdoc/generated/usernet-portmap-noproxy.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,8 @@ The filter table is the same as with the userland proxy enabled.
2 0 0 ACCEPT 0 -- * * 0.0.0.0/0 0.0.0.0/0 match-set docker-ext-bridges-v4 dst ctstate RELATED,ESTABLISHED
3 0 0 DOCKER-ISOLATION-STAGE-1 0 -- * * 0.0.0.0/0 0.0.0.0/0
4 0 0 DOCKER 0 -- * * 0.0.0.0/0 0.0.0.0/0 match-set docker-ext-bridges-v4 dst
5 0 0 ACCEPT 0 -- docker0 docker0 0.0.0.0/0 0.0.0.0/0
6 0 0 ACCEPT 0 -- docker0 !docker0 0.0.0.0/0 0.0.0.0/0
7 0 0 ACCEPT 0 -- bridge1 bridge1 0.0.0.0/0 0.0.0.0/0
8 0 0 ACCEPT 0 -- bridge1 !bridge1 0.0.0.0/0 0.0.0.0/0
5 0 0 ACCEPT 0 -- docker0 * 0.0.0.0/0 0.0.0.0/0
6 0 0 ACCEPT 0 -- bridge1 * 0.0.0.0/0 0.0.0.0/0

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
Expand Down Expand Up @@ -62,10 +60,8 @@ The filter table is the same as with the userland proxy enabled.
-A FORWARD -m set --match-set docker-ext-bridges-v4 dst -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -m set --match-set docker-ext-bridges-v4 dst -j DOCKER
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i bridge1 -o bridge1 -j ACCEPT
-A FORWARD -i bridge1 ! -o bridge1 -j ACCEPT
-A FORWARD -i docker0 -j ACCEPT
-A FORWARD -i bridge1 -j ACCEPT
-A DOCKER -d 192.0.2.2/32 ! -i bridge1 -o bridge1 -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER ! -i docker0 -o docker0 -j DROP
-A DOCKER ! -i bridge1 -o bridge1 -j DROP
Expand Down
Loading

0 comments on commit df3c78d

Please sign in to comment.