Launching container with shared volume on rootfs changes root filesystem to overlay and can't launch containers

[dnoland1]

When launching a container with a shared volume, the root filesystem changes from rootfs to overlay and from that point on, cannot launch containers successfully.

# df -h /
Filesystem      Size  Used Avail Use% Mounted on
rootfs           20G  6.9G   14G  35% /

# docker run -d  --name mounttest -v "/var/tmp:/var/tmp:shared" alpine sleep 1h
Unable to find image 'alpine:latest' locally
latest: Pulling from alpine
cd784148e348: Already exists
Digest: sha256:3d2e482b82608d153a374df3357c0291589a61cc194ec4a9ca2381073a17f58e
Status: Downloaded newer image for 10.66.0.150:5000/alpine:latest
a8f33cf01acd5ce7681081c6dc7abd3c2d28cd2b5607c107a4b1b8c24de4876c

# df -h /
Filesystem      Size  Used Avail Use% Mounted on
overlay          20G  6.9G   14G  35% /

# docker run --rm hello-world
docker: Error response from daemon: open /etc/resolv.conf: no such file or directory.

# docker info
Containers: 21
 Running: 9
 Paused: 0
 Stopped: 12
Images: 9
Server Version: 18.09.0
Storage Driver: overlay2
 Backing Filesystem: tmpfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: c4446665cb9c30056f4998ed953e6d4ff22c7c39
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: fec3683
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.15.0-24.26-genesis-7-generic
Operating System: Ubuntu 18.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 64
Total Memory: 376.7GiB
Name: s01
ID: N2B7:SLHD:4XN2:5OEP:RLHG:GUYX:N2LU:4DWU:55LQ:A52P:ZIDL:6I4L
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 72
 Goroutines: 78
 System Time: 2019-01-11T05:16:44.633405072Z
 EventsListeners: 0
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

WARNING: No swap limit support

For some use cases when launching containers with various volumes, also seeing the error:
Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused \"rootfs_linux.go:58: mounting \\\"cgroup\\\" to rootfs \\\"/var/lib/docker/overlay2/1dfdb3f2ff50fed4649de6e4fe563407e938e6c00115c3d5c6cb862abceb1fef/merged\\\" at \\\"/sys/fs/cgroup\\\" caused \\\"stat /sys/fs/cgroup/1b896c95cdea3a22cd7c41e6f7c175ab71e0a2fe9658697a738098f16a279517: no such file or directory\\\"\"": unknown

[cyphar]

Maybe you meant to submit this against Docker? I haven't looked into it, but this seems like a Docker misconfiguration to me (unless there's already an open Docker issue you forgot to link).

[dnoland1]

Ok, since I saw container_linux.go, process_linux.go, and rootfs_linux.go were in this project and I was getting errors referencing those files, I assume this was the correct home for this issue. If that's not the case, please let me know the URL I should use to report this problem.

[kfox1111]

I can confirm I've seen this with both docker and cri-o. Details in the linked issue above. I don't think they share much in common except runc.