Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement graceful termination for the server, canceling outstanding operations #607

Closed
EronWright opened this issue Jul 24, 2024 · 2 comments · Fixed by #659
Closed

Implement graceful termination for the server, canceling outstanding operations #607

EronWright opened this issue Jul 24, 2024 · 2 comments · Fixed by #659
Assignees
@blampe
Labels
kind/task Work that's part of an ongoing epic resolution/fixed This issue was fixed

Comments

@EronWright
Copy link
Contributor

EronWright commented Jul 24, 2024 •
edited
Loading

When the grpc server receives a shutdown signal (SIGINT), it should gracefully shutdown. It should not accept new RPC requests, and should wait for outstanding requests to complete.

Shutdown is usually due to a planned disruption, e.g. workspace pod termination due to a change to the project source or for node maintenance.

Ideally, any outstanding Pulumi deployment operation would be sent a CTRL-C signal to give it a chance to wind down gracefully. That's an unimplemented aspect of the core auto API: pulumi/pulumi#13160
@EronWright EronWright mentioned this issue Jul 24, 2024
Closed
@pulumi-bot pulumi-bot added the needs-triage Needs attention from the triage team label Jul 24, 2024
@cleverguy25
Copy link

Added to epic #586

@cleverguy25 cleverguy25 mentioned this issue Jul 24, 2024
Open
5 tasks
@EronWright EronWright changed the title Implement graceful termination for the server, canceling outstanding operations. Implement graceful termination for the server, canceling outstanding operations Jul 24, 2024
@cleverguy25
Copy link

Added to epic #586

@EronWright EronWright removed the needs-triage Needs attention from the triage team label Jul 24, 2024
@mjeffryes mjeffryes added the kind/task Work that's part of an ongoing epic label Aug 23, 2024
@blampe blampe mentioned this issue Sep 11, 2024
Merged
blampe added a commit that referenced this issue Sep 24, 2024
@blampe
[v2] Graceful shutdown (#659)
181286c 
This implements graceful shutdown by propagating interrupts to our child
processes.

Killing a workspace pod in the middle of an update will mark the Update
as failed ("update canceled" error), and subsequent Updates will resume
where it left off as you would expect.

Fixes #607.
@EronWright EronWright linked a pull request Sep 25, 2024 that will close this issue
[v2] Graceful shutdown #659
Merged
@EronWright EronWright added the resolution/fixed This issue was fixed label Sep 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@blampe blampe

Labels
kind/task Work that's part of an ongoing epic resolution/fixed This issue was fixed
Projects
Pulumi Kubernetes Operator v2
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[v2] Graceful shutdown pulumi/pulumi-kubernetes-operator
5 participants
@blampe @mjeffryes @EronWright @cleverguy25 @pulumi-bot