Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Python 3.12 cannot pickle PickleBufferIOPicklable instances #4299

Closed
IAlibay opened this issue Sep 22, 2023 · 3 comments · Fixed by #4300
Closed

Python 3.12 cannot pickle PickleBufferIOPicklable instances #4299

IAlibay opened this issue Sep 22, 2023 · 3 comments · Fixed by #4300

Comments

@IAlibay
Copy link
Member

IAlibay commented Sep 22, 2023

This is one of the biggest issues we're facing with Py3.12:

https://github.com/MDAnalysis/mdanalysis/actions/runs/6277097391/job/17048178820?pr=4265#step:8:375

As far as I can tell there's isn't anything super obvious in the changelog that would cause this.
@yuxuanzhuang
Copy link
Contributor

yuxuanzhuang commented Sep 22, 2023 •
edited
Loading

This should be due to python/cpython#104370

But allegedly, only protocol 0 and 1 should be disallowed so I am not sure what's happening here. I am also not able to test on Python 3.12 at the moment.

PEP ref: https://peps.python.org/pep-0687/

tylerjereddy added a commit to tylerjereddy/mdanalysis that referenced this issue Sep 24, 2023
@tylerjereddy
MAINT: Python 3.12 support improve
fd7df3c 
* Fixes MDAnalysisgh-4299, and allows the testsuite to have 12 failures
instead of 56 with Python `3.12.0rc3` for me locally on x86_64 Linux;
full suite still passing for Python `3.11.x`

* the main idea is to circumvent more aggressive CPython
blocks on serialization by overriding the highest-priority
pickling method available, `__reduce_ex__`, which CPython
had locked down, making it such that our old `__getstate__`
pickling shims were ignored as indicators of pickling safety

* there is probably a slightly simpler diff that allows
this to work, but this took a few hours to draft so feel
free to push in simplifications if you're sure you've tested
on 3.11.x and 3.12 RC

[skip cirrus]
@tylerjereddy tylerjereddy mentioned this issue Sep 24, 2023
Merged
@harryhritik12
Copy link

Hey, I would like to solve this issue.

@IAlibay
Copy link
Member Author

IAlibay commented Sep 28, 2023

@harryhritik12 - Apologies, I believe this is already in progress of being fixed by #4300

IAlibay pushed a commit that referenced this issue Oct 12, 2023
@tylerjereddy
MAINT: Python 3.12 support improve (#4300)
8c41dc8 
* Fixes gh-4299, and allows the testsuite to have 12 failures
instead of 56 with Python `3.12.0rc3` for me locally on x86_64 Linux;
full suite still passing for Python `3.11.x`

* the main idea is to circumvent more aggressive CPython
blocks on serialization by overriding the highest-priority
pickling method available, `__reduce_ex__`, which CPython
had locked down, making it such that our old `__getstate__`
pickling shims were ignored as indicators of pickling safety

* there is probably a slightly simpler diff that allows
this to work, but this took a few hours to draft so feel
free to push in simplifications if you're sure you've tested
on 3.11.x and 3.12 RC

[skip cirrus]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

MAINT: Python 3.12 support improve tylerjereddy/mdanalysis
3 participants
@IAlibay @yuxuanzhuang @harryhritik12