Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix: possible race condition in
AutoRestartTrick
(#1002)
* fixes a possible race condition in AutoRestartTrick Just a long shot for a failure observed on #998. My hypothesis is that when we stop ProcessWatcher before we restart the process manually, we don't yield to it and immediately kill the process. Next, when the ProcessWatcher thread is woken up, we have to conditions ready - the popen_obj and stopped_event, see the corresponding code, ``` while True: if self.popen_obj.poll() is not None: break if self.stopped_event.wait(timeout=0.1): return ``` And desipte that `stopped_event` is set, we first check for `popen_obj` and trigger the process restart. We can also make the ProcessWatcher logic more robust, by checking if we are stopped before calling the termination callback, e.g., ``` try: if not self.stopped_event.is_set(): self.process_termination_callback() except Exception: logger.exception("Error calling process termination callback") ``` I am not 100% sure about that, as I don't really know what semantics is expected from ProcessWatcher by other users. But at least the AutoRestarter expects this semantics - i.e., a watcher shall not call any events after it was stopped. * tries an alternative solution i.e., don't send events if stopped
- Loading branch information