forked from grafana/mimir
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add the ability to wait for a period of time after SIGTERM (grafana#3298
) * Add the ability to wait for a period of time after SIGTERM Adds the ability to supply a "shutdown delay" to Mimir components such that they will disable HTTP keep-alives and mark themselves as not ready when receiving SIGTERM or SIGINT (via HTTP /ready or gRPC) but wait a configurable amount of time before actually stopping. Fixes an issue during rollouts on Kubernetes where the Grafana Cloud Gateway holds on to connections to query-frontends even when they shutdown resulting in user-facing read errors. By closing connections during shutdown, marking the component as "not ready", and still continuing to serve requests we ensure that: * Users don't see any disruption * Connections to the stopping component are not pooled * Kubernetes service endpoints are removed before the pod Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Code review changes. Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Handle shutdown inline in `Run` goroutine. Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> * Phrasing. Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com> Signed-off-by: Nick Pillitteri <nick.pillitteri@grafana.com>
- Loading branch information
1 parent
06775ea
commit 6bae854
Showing
9 changed files
with
92 additions
and
31 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
62 changes: 41 additions & 21 deletions
62
vendor/github.com/grafana/dskit/grpcutil/health_check.go
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.