Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gh-120012: clarify the behaviour of multiprocessing.Queue.empty on closed queues. #120102

Merged
merged 10 commits into from
Jun 13, 2024
Merged

gh-120012: clarify the behaviour of multiprocessing.Queue.empty on closed queues. #120102

merged 10 commits into from
Jun 13, 2024

Conversation

picnixz
Copy link
Contributor

@picnixz picnixz commented Jun 5, 2024 •
edited by github-actions bot
Loading

I took the liberty of clarifying the behaviour of SimpleQueue and JoinableQueue as well since SimpleQueue is a bit different.

I tested the behaviour on main, so I'm not entirely sure that it's the same on the previous versions, and that's why I added some tests for that.

📚 Documentation preview 📚: https://cpython-previews--120102.org.readthedocs.build/

@picnixz picnixz requested a review from gpshead as a code owner June 5, 2024 10:41
@bedevere-app bedevere-app bot added tests Tests in the Lib/test dir awaiting review labels Jun 5, 2024
@bedevere-app bedevere-app bot mentioned this pull request Jun 5, 2024
Closed
@picnixz picnixz marked this pull request as draft June 5, 2024 11:26
@picnixz
Copy link
Contributor Author

picnixz commented Jun 5, 2024 •
edited
Loading

Ok, there are strange things happening there. I'm no longer able to reproduce the OSError locally so I'll try to investigate what happened.

EDIT: I managed to fix them locally. It appears that the interpreter did not require the explicit join_thread() call sometimes (it might be because of my PC?) but with join_thread explicitly called, then we can be sure that an OSError is raised.

Maybe someone can explain to me why I managed to raise the OSError without that call and if I should actually update the docs accordingly (my feeling is that the feeder thread might be closed but not terminated, in which case self._reader.poll() would still return and thus you would not have an OSError).

@picnixz picnixz marked this pull request as ready for review June 5, 2024 11:47
@picnixz
Copy link
Contributor Author

picnixz commented Jun 12, 2024

@ericvsmith since you commented on the issue, I would like to know whether you would be available for reviewing it. Thank you in advance.

ericvsmith
ericvsmith requested changes Jun 12, 2024
View reviewed changes
Copy link
Member

@ericvsmith ericvsmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small suggestions.

Lib/test/_test_multiprocessing.py Show resolved Hide resolved
Misc/NEWS.d/next/Documentation/2024-06-05-12-36-18.gh-issue-120012.f14DbQ.rst Outdated Show resolved Hide resolved
@bedevere-app
Copy link

bedevere-app bot commented Jun 12, 2024

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@picnixz
Copy link
Contributor Author

picnixz commented Jun 12, 2024

Thank you Eric for your suggestions. I have made the requested changes; please review again.

@bedevere-app
Copy link

bedevere-app bot commented Jun 12, 2024

Thanks for making the requested changes!

@ericvsmith: please review the changes made to this pull request.

@bedevere-app bedevere-app bot requested a review from ericvsmith June 12, 2024 15:53
ericvsmith
ericvsmith approved these changes Jun 12, 2024
View reviewed changes
Copy link
Member

@ericvsmith ericvsmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

picnixz and others added 2 commits June 13, 2024 09:57
@picnixz
simplify documentation
5bd3f12
@gpshead
Simplify and clarify the doc wording.
5fefd9c 
further doc simplification to alleviate confusion.
@gpshead gpshead enabled auto-merge (squash) June 13, 2024 18:41
@gpshead gpshead added docs Documentation in the Doc dir needs backport to 3.12 bug and security fixes needs backport to 3.13 bugs and security fixes labels Jun 13, 2024
@gpshead gpshead merged commit a3711af into python:main Jun 13, 2024
36 checks passed
@miss-islington-app
Copy link

Thanks @picnixz for the PR, and @gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 13, 2024
@picnixz @gpshead @miss-islington
pythongh-120012: clarify the behaviour of `multiprocessing.Queue.empt…
92fcf3e 
…y` on closed queues. (pythonGH-120102)

* improve doc for `multiprocessing.Queue.empty`
* add tests for checking emptiness of queues

(cherry picked from commit a3711af)

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
@bedevere-app
Copy link

bedevere-app bot commented Jun 13, 2024

GH-120469 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Jun 13, 2024
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 13, 2024
@picnixz @gpshead @miss-islington
pythongh-120012: clarify the behaviour of `multiprocessing.Queue.empt…
c0a2a50 
…y` on closed queues. (pythonGH-120102)

* improve doc for `multiprocessing.Queue.empty`
* add tests for checking emptiness of queues

(cherry picked from commit a3711af)

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
@bedevere-app
Copy link

bedevere-app bot commented Jun 13, 2024

GH-120470 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.12 bug and security fixes label Jun 13, 2024
@picnixz picnixz deleted the doc-queue-empty branch June 14, 2024 08:49
mrahtz pushed a commit to mrahtz/cpython that referenced this pull request Jun 30, 2024
@picnixz @gpshead @mrahtz
pythongh-120012: clarify the behaviour of `multiprocessing.Queue.empt…
dbafe5c 
…y` on closed queues. (pythonGH-120102)

* improve doc for `multiprocessing.Queue.empty`
* add tests for checking emptiness of queues

Co-authored-by: Gregory P. Smith <greg@krypto.org>
noahbkim pushed a commit to hudson-trading/cpython that referenced this pull request Jul 11, 2024
@picnixz @gpshead @noahbkim
pythongh-120012: clarify the behaviour of `multiprocessing.Queue.empt…
4f70520 
…y` on closed queues. (pythonGH-120102)

* improve doc for `multiprocessing.Queue.empty`
* add tests for checking emptiness of queues

Co-authored-by: Gregory P. Smith <greg@krypto.org>
hugovk pushed a commit that referenced this pull request Jul 14, 2024
@miss-islington @picnixz @gpshead
[3.13] gh-120012: clarify the behaviour of `multiprocessing.Queue.emp…
dfedbdb 
…ty` on closed queues. (GH-120102) (#120469)

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
hugovk pushed a commit that referenced this pull request Jul 14, 2024
@miss-islington @picnixz @gpshead
[3.12] gh-120012: clarify the behaviour of `multiprocessing.Queue.emp…
863c0a3 
…ty` on closed queues. (GH-120102) (#120470)

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
estyxx pushed a commit to estyxx/cpython that referenced this pull request Jul 17, 2024
@picnixz @gpshead @estyxx
pythongh-120012: clarify the behaviour of `multiprocessing.Queue.empt…
3045f6c 
…y` on closed queues. (pythonGH-120102)

* improve doc for `multiprocessing.Queue.empty`
* add tests for checking emptiness of queues

Co-authored-by: Gregory P. Smith <greg@krypto.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gpshead gpshead gpshead approved these changes

@ericvsmith ericvsmith ericvsmith approved these changes

Assignees
No one assigned
Labels
docs Documentation in the Doc dir tests Tests in the Lib/test dir
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@picnixz @gpshead @ericvsmith