Add --shm-size to docker container update

[turicas]

Description

I'd like to easily change the shm-size in a running container by using docker container update. I'm running PostgreSQL inside a container and receiving an error related to the small amount of shared memory available.

Steps to reproduce the issue:

1. Run any container you want to, without passing --shm-size
2. Use some software inside the container to fill the 64MB shared memory available.
3. Done, you're out of shared memory and you can't change it using the docker cli without destroying the container and creating a new one.

Describe the results you received:

I've found two ways of changing the shared memory of a running container:

1. Remount the shm file system of the container: mount -o remount,rw,nosuid,nodev,noexec,relatime,size=256M -t tmpfs /var/lib/docker/containers/<container-id>/mounts/shm. If shm is not found inside this path, try using nsenter as explained in Add --shm-size to docker container update #1278 (comment)

2. Change ShmSize inside /var/lib/docker/containers/<container-id>/hostconfig.json and restart container. EDIT: as noted by @dmitrygashnikov: you must stop docker service (service docker stop), then change the hostconfig.json file for this container and start docker again (service docker start).

The second option is the only one which makes the change persistent, but we need to stop all running containers to make the change.

Here is a little script to change /dev/shm (by default to 1GB), using the docker daemon restart method:

#!/bin/bash

set -e
CONTAINER_NAME="$1"
if [ -z "$CONTAINER_NAME" ]; then
        echo "ERROR - Usage: $0 <container-name> [shm-size]"
        exit 1
fi
if [ ! -z "$2" ]; then
        SHM_SIZE="$2"
else
        SHM_SIZE="1073741824"
fi

echotime() {
        echo [$(date "+%Y-%m-%d %H:%M:%S")] $@
}

CONTAINER_ID=$(docker inspect -f '{{ .ID }}' $CONTAINER_NAME)
HOST_CONFIG=/var/lib/docker/containers/$CONTAINER_ID/hostconfig.json

echotime "Stopping docker service"
service docker stop

echotime "Changing container's hostconfig.json"
sed -i 's/"ShmSize":[0-9]\+,/"ShmSize":'$SHM_SIZE',/' $HOST_CONFIG

echotime "Starting docker service"
service docker start

Describe the results you expected:

It would be awesome to change this setting from the command line, using docker container update --shm-size=256M.

Output of docker version:

Client:
 Version:      18.03.1-ce
 API version:  1.37
 Go version:   go1.9.5
 Git commit:   9ee9f40
 Built:        Thu Apr 26 07:17:14 2018
 OS/Arch:      linux/amd64
 Experimental: false
 Orchestrator: swarm

Server:
 Engine:
  Version:      18.03.1-ce
  API version:  1.37 (minimum version 1.12)
  Go version:   go1.9.5
  Git commit:   9ee9f40
  Built:        Thu Apr 26 07:15:24 2018
  OS/Arch:      linux/amd64
  Experimental: false

[cpuguy83]

It's not really easily done on a running container, so it could only take effect on container restart.
Why not create a new container?

[turicas]

@cpuguy83 because of some reasons:

• The filesystem of this running container is different from the image and as it's specific for this application, I do not consider commiting it, so the changes will be lost if I create a new one;
• I've started the container with a lot of options and it's not easy to retrieve the docker run command everytime I need to fix this problem.

If there are use cases for changing other container configurations inside docker container update, why not add shm?

[cpuguy83]

If you need persistent state, please consider using a volume for this. This would allow you to attach the same volume to a different container. It is also significantly more performant if that is at all needed.

All other options can be updated while a container is running. Mounts cannot.

[ony]

@cpuguy83 , I have similar issue. I have container that I created without use of volumes unfortunately. And each time I start it I have to perform mount -o remount,size=256m /dev/shm which is annoying.

Changing start configuration of container makes sense even if effect will be during next start. Ideally docker container inspect should allow you to see both running and next start configuration.

Right now work-around is:

1. Stop docker daemon (with your container)
2. Modify hostconfig.json as you want
3. Now you can start docker daemon and your container with updated ShmSize.

Note that though generic mounts changes might not be achievable with running container there still should be a way to change ShmSize since that's a managed option despite resulting in a mount. Changing that can result in something like:

# outside of container
mount -o remount,size=500m '/var/lib/docker/containers/.../mounts/shm'

[edib]

These suggestions do not work for version 19.03.
After updating hostconfig.json with new value, I start the container but config returns to old value. And there is not shm in mounts dir.

[dmitrygashnikov]

These suggestions do not work for version 19.03.
After updating hostconfig.json with new value, I start the container but config returns to old value. And there is not shm in mounts dir.

Make sure, that docker service is stoped, before modify hostconfig.json.
When service reloading or something it`s rewriting all changes using running config.

[Monstrofil]

The fast and ditry solution for all who want to increase shm in runtime without recreating daemon.

1. Find the PID of process that is running inside container, in my case it was postgresql with pid=1239.

root@Ubuntu-1804-bionic-64-minimal ~ # ps auxf | grep 882142e6eb97 -A 10
root      1213  0.0  0.0 107700  5120 ?        Sl   15:30   0:00  \_ containerd-shim -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/882142e6eb979835e58ac62a78d0951add9c556ca723e6037138f0c7bd42a582 -address /run/containerd/containerd.sock -containerd-binary /usr/bin/containerd -runtime-root /var/run/docker/runtime-runc
influxdb  1239  0.5  0.3 8760360 214212 ?      Ss   15:30   0:01      \_ postgres
influxdb  1319  0.0  0.0 8760360 3728 ?        Ss   15:30   0:00          \_ postgres: checkpointer

2. Use nsenter to enter the process namespace and remount shm:

root@Ubuntu-1804-bionic-64-minimal ~ # nsenter --mount=/proc/1327/ns/mnt
mesg: ttyname failed: Success
root@Ubuntu-1804-bionic-64-minimal:/# mount -o remount,size=2048m /dev/shm

Unless your application spawns processes in different namespaces, you shoud be fine.

DO NOT USE THIS IN PRODUCTION