Skip to content

Commit

Permalink
Change subproc sigint handler to exit more cleanly
Browse files Browse the repository at this point in the history
The very fast exit of os._exit can cause the stdlib multiproc glue code in the parent process, specifically the queues readers that shuttle results between processes, to deadlock.

It appears that using `sys.exit` in subprocesses instead allows those queues to close in a way that does not cause the parent process to deadlock, while still ensuring that subprocesses exit.

As a refresher: Even if the work function passed to subprocs catches KeyboardInterrupt, the multiprocessing queue code still throws, so a signal handler of some kind is needed to prevent $numproc stacktraces being printed on interrupt.

Testing Done:
https://travis-ci.org/pantsbuild/pants/builds/56935938

Reviewed at https://rbcommons.com/s/twitter/r/2024/
  • Loading branch information
dt committed Apr 7, 2015
1 parent e4fb930 commit 14886c5
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/python/pants/base/worker_pool.py
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ class SubprocPool(object):
@staticmethod
def worker_init():
# Exit quietly on sigint, otherwise we get {num_procs} keyboardinterrupt stacktraces spewn
signal.signal(signal.SIGINT, lambda *args: os._exit(0))
signal.signal(signal.SIGINT, lambda *args: sys.exit())

@classmethod
def foreground(cls):
Expand Down

0 comments on commit 14886c5

Please sign in to comment.