Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-120012: clarify the behaviour of multiprocessing.Queue.empty on closed queues. #120102

Merged
merged 10 commits into from
Jun 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Doc/library/multiprocessing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -837,6 +837,8 @@ For an example of the usage of queues for interprocess communication see
Return ``True`` if the queue is empty, ``False`` otherwise. Because of
multithreading/multiprocessing semantics, this is not reliable.

May raise an :exc:`OSError` on closed queues. (not guaranteed)

.. method:: full()

Return ``True`` if the queue is full, ``False`` otherwise. Because of
Expand Down Expand Up @@ -940,6 +942,8 @@ For an example of the usage of queues for interprocess communication see

Return ``True`` if the queue is empty, ``False`` otherwise.

Always raises an :exc:`OSError` if the SimpleQueue is closed.

.. method:: get()

Remove and return an item from the queue.
Expand Down
26 changes: 26 additions & 0 deletions Lib/test/_test_multiprocessing.py
Original file line number Diff line number Diff line change
Expand Up @@ -1332,6 +1332,23 @@ def _on_queue_feeder_error(e, obj):
self.assertTrue(not_serializable_obj.reduce_was_called)
self.assertTrue(not_serializable_obj.on_queue_feeder_error_was_called)

def test_closed_queue_empty_exceptions(self):
picnixz marked this conversation as resolved.
Show resolved Hide resolved
# Assert that checking the emptiness of an unused closed queue
# does not raise an OSError. The rationale is that q.close() is
# a no-op upon construction and becomes effective once the queue
# has been used (e.g., by calling q.put()).
for q in multiprocessing.Queue(), multiprocessing.JoinableQueue():
q.close() # this is a no-op since the feeder thread is None
q.join_thread() # this is also a no-op
self.assertTrue(q.empty())

for q in multiprocessing.Queue(), multiprocessing.JoinableQueue():
q.put('foo') # make sure that the queue is 'used'
q.close() # close the feeder thread
q.join_thread() # make sure to join the feeder thread
with self.assertRaisesRegex(OSError, 'is closed'):
q.empty()

def test_closed_queue_put_get_exceptions(self):
for q in multiprocessing.Queue(), multiprocessing.JoinableQueue():
q.close()
Expand Down Expand Up @@ -5815,6 +5832,15 @@ def _test_empty(cls, queue, child_can_start, parent_can_continue):
finally:
parent_can_continue.set()

def test_empty_exceptions(self):
# Assert that checking emptiness of a closed queue raises
# an OSError, independently of whether the queue was used
# or not. This differs from Queue and JoinableQueue.
q = multiprocessing.SimpleQueue()
q.close() # close the pipe
with self.assertRaisesRegex(OSError, 'is closed'):
q.empty()

def test_empty(self):
queue = multiprocessing.SimpleQueue()
child_can_start = multiprocessing.Event()
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Clarify the behaviours of :meth:`multiprocessing.Queue.empty` and
:meth:`multiprocessing.SimpleQueue.empty` on closed queues.
Patch by Bénédikt Tran.
Loading