Docker WorkloadAttestor plugin does not work for RHEL

[vbotez]

• Version: 0.9.0
• Platform: 3.10.0-1062.12.1.el7.x86_64,
• Subsystem: WorkloadAttestor "docker" (spire-agent)

Docker WorkloadAttestor relies on cgroups to get details about the container selectors.

The code as-written cannot handle extracting the ID out of docker-<ID>.scope. It expects the ID to be an entire segment, i.e. /.../<ID>/...

Here is an example from docker4desktop (that I think is based on Ubuntu):

cat /proc/34089/cgroup
14:name=systemd:/docker/e465ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9/docker/e46
5ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9/system.slice/containerd.service
13:rdma:/
12:pids:/docker/e465ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9/docker/e465ef604af
b61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9/system.slice/containerd.service
11:hugetlb:/docker/e465ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9
10:net_prio:/docker/e465ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9
9:perf_event:/docker/e465ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9
8:net_cls:/docker/e465ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9
7:freezer:/docker/e465ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9
6:devices:/docker/e465ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9/docker/e465ef604
afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9/system.slice/containerd.service
5:memory:/docker/e465ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9/docker/e465ef604a
fb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9/system.slice/containerd.service
4:blkio:/docker/e465ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9/docker/e465ef604af
b61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9/system.slice/containerd.service
3:cpuacct:/docker/e465ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9
2:cpu:/docker/e465ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9/docker/e465ef604afb6
1d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9/system.slice/containerd.service
1:cpuset:/docker/e465ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9
0::/docker/e465ef604afb61d4c3c814f899f668368b0f2568c9b592dabcb4bf980b5597a9/system.slice/containerd
.service

vs. RHEL (and I guess CentOS also) system:

cat /proc/41223/cgroup
11:hugetlb:/system.slice/docker-41e4ab61d2860b0e1467de0da0a9c6068012761febec402dc04a5a94f32ea867.scope
10:memory:/system.slice/docker-41e4ab61d2860b0e1467de0da0a9c6068012761febec402dc04a5a94f32ea867.scope
9:blkio:/system.slice/docker-41e4ab61d2860b0e1467de0da0a9c6068012761febec402dc04a5a94f32ea867.scope
8:net_prio,net_cls:/system.slice/docker-41e4ab61d2860b0e1467de0da0a9c6068012761febec402dc04a5a94f32ea867.scope
7:perf_event:/system.slice/docker-41e4ab61d2860b0e1467de0da0a9c6068012761febec402dc04a5a94f32ea867.scope
6:cpuset:/system.slice/docker-41e4ab61d2860b0e1467de0da0a9c6068012761febec402dc04a5a94f32ea867.scope
5:pids:/system.slice/docker-41e4ab61d2860b0e1467de0da0a9c6068012761febec402dc04a5a94f32ea867.scope
4:devices:/system.slice/docker-41e4ab61d2860b0e1467de0da0a9c6068012761febec402dc04a5a94f32ea867.scope
3:freezer:/system.slice/docker-41e4ab61d2860b0e1467de0da0a9c6068012761febec402dc04a5a94f32ea867.scope
2:cpuacct,cpu:/system.slice/docker-41e4ab61d2860b0e1467de0da0a9c6068012761febec402dc04a5a94f32ea867.scope
1:name=systemd:/system.slice/docker-41e4ab61d2860b0e1467de0da0a9c6068012761febec402dc04a5a94f32ea867.scope

According to the docker workload attestor code here:

spire/pkg/agent/plugin/workloadattestor/docker/docker.go

Line 31 in 752d4d4

var defaultContainerIDMatchers = []string{

This will never match RHEL type cgroups.

[azdagron]

Thank you for filing this issue @vbotez!

To fix this we'll need to update the container ID finders that consume those matcher patterns to allow for a sub-segment match on <id> here:

spire/pkg/agent/plugin/workloadattestor/docker/cgroup/dockerfinder.go

Line 31 in a5b56f8

func newContainerIDFinder(pattern string) (ContainerIDFinder, error) {

Operators can then use the container_id_cgroup_matchers configurable to do something like:

WorkloadAttestor "docker" {
    plugin_data = {
        container_id_cgroup_matchers = ["/system.slice/docker-<id>.scope"]
    }
}

[vbotez]

Thank you for the pointers @azdagron !

[colek42]

Having the same issue on Ubuntu. Adding for context.

redacted@redacted:~$ sudo containerd -v
containerd containerd.io 1.2.13 7ad184331fa3e55e52b890ea95e65ba581ae3429

redacted@redacted:/sys/fs/cgroup/pids/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope$ cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.4 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.4 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

redacted@redacted:/sys/fs/cgroup/pids/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope$ docker version
Client: Docker Engine - Community
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        afacb8b7f0
 Built:             Wed Mar 11 01:25:46 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       afacb8b7f0
  Built:            Wed Mar 11 01:24:19 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
redacted@redacted:/sys/fs/cgroup/pids/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope$ uname -a
Linux node-ptt 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
boxboat@node-ptt:/sys/fs/cgroup/pids/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope$ sudo cat /proc/23076/cgroup
12:rdma:/
11:perf_event:/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope
10:blkio:/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope
9:cpuset:/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope
8:memory:/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope
7:devices:/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope
6:hugetlb:/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope
5:pids:/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope
4:freezer:/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope
3:net_cls,net_prio:/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope
2:cpu,cpuacct:/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope
1:name=systemd:/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope
0::/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope
redacted@redacted:/sys/fs/cgroup/pids/system.slice/docker-57020e25a99568afbcd013b457719a58affaad313d7917401c0573c292e4c886.scope$

[azdagron]

A fix for this was just merged. Mind giving it another go?