[pipes] subprocess termination forwarding #18685
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Current dependencies on/for this PR:
This stack of pull requests is managed by Graphite. |
alangenfeld
force-pushed
the
al/12-12-_pipes_subprocess_termination_forwarding
branch
2 times, most recently
from
f8fd3ea
to
e2d5a47
Compare
alangenfeld
commented
Dec 13, 2023
Comment on lines
123
to
124
| process.send_signal(signal.SIGINT) | ||
| process.wait(timeout=INTERRUPT_FWD_WAIT_SECONDS) |
alangenfeld
commented
Dec 13, 2023
Comment on lines
+705
to
+718
| import os | ||
| import time | ||
|
|
||
| from dagster_pipes import open_dagster_pipes | ||
|
|
||
| with open_dagster_pipes() as pipes: | ||
| timeout = pipes.get_extra("timeout") | ||
| log_path = pipes.get_extra("log_path") | ||
| with open(log_path, "w") as f: | ||
| f.write(f"{os.getpid()}") | ||
| f.flush() | ||
| start = time.time() | ||
| while time.time() - start < timeout: | ||
| ... |
alangenfeld
commented
Dec 13, 2023
| except DagsterExecutionInterruptedError: | ||
| context.log.info("[pipes] execution interrupted, sending SIGINT to subprocess.") | ||
| # send sigint to give external process chance to exit gracefully | ||
| process.send_signal(signal.SIGINT) |
There was a problem hiding this comment.
the alternative choice of using process.terminate() which uses SIGTERM would be having open_dagster_pipes install a SIGTERM handler since the default no-handler behavior is immediate process halt
geoHeil
reviewed
Dec 13, 2023
alangenfeld
force-pushed
the
al/12-12-_pipes_subprocess_termination_forwarding
branch
from
e2d5a47
to
0230e7a
Compare
|
Deploy preview for dagster-docs ready! Preview available at https://dagster-docs-fv5c6ruhu-elementl.vercel.app Direct link to changed pages: |
alangenfeld
force-pushed
the
al/12-12-_pipes_subprocess_termination_forwarding
branch
from
0230e7a
to
2c8c568
Compare
alangenfeld
deleted the
al/12-12-_pipes_subprocess_termination_forwarding
branch
rexledesma
added a commit
that referenced
this pull request
Jan 29, 2024
## Summary & Motivation The third attempt at interrupting the dbt subprocess on exit. Instead of relying on `atexit` (#15897) or overriding signal handlers, just catch the execution interrupt exception like `dagster-pipes` (#18685) and terminate the subprocess when that's handled. ## How I Tested These Changes pytest, local https://github.com/dagster-io/dagster/assets/16431325/05d1e002-430b-49b3-a81e-c08743e3ed63
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When a pipes subprocess is interrupted, forward the termination to the external process to avoid leaving it orphaned.
Addresses #18570 for subprocess client
How I Tested These Changes
added test