-
Notifications
You must be signed in to change notification settings - Fork 158
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Steps towards Python 3.11 support #3231
Conversation
* Replace "Py_TYPE(obj) = type" with: "Py_SET_TYPE(obj, type)" * Replace "Py_REFCNT(Py_None) += n" with: "Py_SET_REFCNT(Py_None, Py_REFCNT(Py_None) + n)" * Add pythoncapi_compat.h to get Py_SET_TYPE() and Py_SET_REFCNT() on Python 3.9 and older. File copied from: https://github.com/pythoncapi/pythoncapi_compat On Python 3.10, Py_REFCNT() can no longer be used to set a reference count: * https://docs.python.org/dev/c-api/structures.html#c.Py_REFCNT * https://docs.python.org/dev/whatsnew/3.10.html#id2 On Python 3.11, Py_TYPE() can no longer be used to set an object type: * https://docs.python.org/dev/c-api/structures.html#c.Py_TYPE * https://docs.python.org/dev/whatsnew/3.11.html#id2
Thanks @vstinner. I guess python As for the python |
On the other side, |
I built manually Python 3.11 from source on Linux, and I installed it in /opt/py311 (pick your favorite directory):
Currently, building datatable with Python 3.11 fails with 2 compiler errors on Py_TYPE() used as an l-value:
With this PR, it builds successfully:
|
I agree that it's tedious to use Python 3.11 right now. I'm using Fedora 35 which does provide Python 3.11 (Python version 3.11.0a3)! ... But I got issues while trying to build database using My concern is getting as many Python dependencies as possible compatible with Python 3.11 before Python 3.11 final will be released next October. Note: I'm also the author of https://www.python.org/dev/peps/pep-0674/ which introduced the incompatible change in Python :-) The PEP explains the rationale for these changes. |
Great, thank you for merging my fix :-) Ping me if you get more Python 3.11 compatibility issues ;-) |
We will add python 3.11 to our pipeline as long as it is released. If by that time you introduce some more incompatible changes, your help is very much appreciated :) |
@vstinner Do you think you can fix these warnings on a separate PR? |
Oh. I can reproduce some warnings on Linux using the command:
I see multiple warnings. NULL:
size_t vs Py_ssize_t:
"use of old-style cast":
|
I see different options:
|
Thanks, @vstinner. So |
I believe As for the static casting, it doesn't work
Could you please provide some details as to what kind of casting you're doing in |
I'm working on a pythoncapi_compat.h update to make it compatible with C++. I created #3237 to check if there are still warnings on pythoncapi_compat.h when built by datatable, before merging it in my upstream project. |
I don't understand why the C++ compiler only complains about pythoncapi_compat.h whereas Python.h is far from being C++ compatible. For example, Python.h defines Py_REFCNT() which also uses old-style cast:
datatable uses |
@vstinner which python is it? For Python 3.9 |
Python.h includes object.h which defines the Py_REFCNT() macro and static inline function. In recent Python functions, a static inline fnuction is used. Previously, it was only a macro. |
@vstinner Actually, I'm now trying to build datatable with the officially released python 3.11 and get quite a lot of warnings/errors. Wasn't it the case for you when you built datatable with the python 3.11 dev version?
|
Hi, I suggest you to open a separated issue. Your errors are different than the ones that I tried to address. For exaple:
You should not redefine Py_buffer type in datatable. |
It is actually python 3.11 that redefined it, as we had this type for a long type in datatable :) Anyways, this issue could be easily fixed. What I wonder is why you had no issues compiling with py311 at the time of this PR and, if something important changed in py311 meanwhile, do you have any suggestions or updates for |
I tested alpha/beta versions of Python 3.11. It was still modified until the Python 3.11 final release. For example, _PySys_GetObjectId() got removed, datatable should not use this private function ;-) |
Sorry, I don't have the bandwidth to investigate these build issues. |
I see, what I meant was if you did any improvements to |
"Py_SET_TYPE(obj, type)"
"Py_SET_REFCNT(Py_None, Py_REFCNT(Py_None) + n)"
Python 3.9 and older. File copied from:
https://github.com/pythoncapi/pythoncapi_compat
On Python 3.10, Py_REFCNT() can no longer be used to set a reference
count:
On Python 3.11, Py_TYPE() can no longer be used to set an object
type: