Skip to content

Commit

Permalink
[scripts/fast-reboot] Shutdown remaining containers through systemd (s…
Browse files Browse the repository at this point in the history
…onic-net#2133)

The current implementation has two issues:

1. In case containers from "docker ps" output are ordered in a way that database is first in the list, the "systemctl stop database" followed by "docker kill database" will stop all other containers through systemd
and ruin this optimization
2. After "docker kill database" there are lots of errors from daemons like hostcfgd, system-healthd, caclmgrd, etc. Also it causes those daemons to hang when received SIGTERM making a delay on following "systemctl stop database".

In the new implementation, services are implicitly stopped by systemd in the order that is correct. If a certain container needs an optimization that will kill the container instead of stopping it the container may implement this optimization in its /usr/local/bin/*.sh script.

It is also more optimal since independent services might be stopped in parallel.

- What I did
Stop services using systemd

- How I did it
Stop services using systemd

- How to verify it
Run warm-reboot.

Signed-off-by: Stepan Blyschak <stepanb@nvidia.com>
  • Loading branch information
stepanblyschak authored and judyjoseph committed Apr 18, 2022
1 parent 82cb0ea commit a5f55aa
Showing 1 changed file with 0 additions and 20 deletions.
20 changes: 0 additions & 20 deletions scripts/fast-reboot
Original file line number Diff line number Diff line change
Expand Up @@ -731,26 +731,6 @@ for service in ${SERVICES_TO_STOP}; do
fi
done
# Kill other containers to make the reboot faster
# We call `docker kill ...` to ensure the container stops as quickly as possible,
# then immediately call `systemctl stop ...` to prevent the service from
# restarting the container automatically.
debug "Stopping all remaining containers ..."
if test -f /usr/local/bin/ctrmgr_tools.py
then
/usr/local/bin/ctrmgr_tools.py kill-all
else
for CONTAINER_NAME in $(docker ps --format '{{.Names}}'); do
CONTAINER_STOP_RC=0
docker kill $CONTAINER_NAME &> /dev/null || CONTAINER_STOP_RC=$?
systemctl stop $CONTAINER_NAME || debug "Ignore stopping $CONTAINER_NAME error $?"
if [[ CONTAINER_STOP_RC -ne 0 ]]; then
debug "Failed killing container $CONTAINER_NAME RC $CONTAINER_STOP_RC ."
fi
done
fi
debug "Stopped all remaining containers ..."
# Stop the docker container engine. Otherwise we will have a broken docker storage
systemctl stop docker.service || debug "Ignore stopping docker service error $?"
Expand Down

0 comments on commit a5f55aa

Please sign in to comment.