Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: support Python 3.13.0b1 (PEP 667 fix) #5127

Merged
merged 5 commits into from
May 24, 2024

Conversation

henryiii
Copy link
Collaborator

@henryiii henryiii commented May 10, 2024

Description

Since beta 1 is out, time to start testing all the time on Python 3.13, not just PR’s with a dev label.

Suggested changelog entry:

* Run CI on Python 3.13

henryiii added 4 commits May 10, 2024 17:00
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
@henryiii
Copy link
Collaborator Author

henryiii commented May 23, 2024

Some destructors are not being called when they used to be. The following three tests break:

FAILED ../../tests/test_factory_constructors.py::test_init_factory_alias - assert [3, 3] == [0, 0]
FAILED ../../tests/test_factory_constructors.py::test_init_factory_dual - assert [9, 6] == [7, 4]
FAILED ../../tests/test_virtual_functions.py::test_override - assert 2 == 0

In these, .alive() is called, which runs the gc, then the number of times .destroyed() is called is less on 3.13.0b1 than 3.13.0a6.

Running git bisect between v3.13.0a6 and v3.13.0b1 with the follow script:

#/usr/bin/env bash
set -euo pipefail

./configure --prefix $PWD/../python
make -j8
make install -j8

cd ..
uv pip install --python=./Python/bin/python3 pytest
cmake -S. -Bbuild -DPython_ROOT_DIR=Python -DPYBIND11_TEST_OVERRIDE='test_factory_constructors.py;test_virtual_functions.py'
cmake --build build --target pytest

Showed this broke with python/cpython#115153, PEP 667.

These are now replaced with new functions:

$ git grep -E 'PyEval_GetLocals|PyEval_GetGlobals|PyEval_GetBuiltins|\.f_locals'
include/pybind11/detail/internals.h:446:    state_dict = reinterpret_borrow<object>(PyEval_GetBuiltins());
include/pybind11/pybind11.h:1347:    PyObject *p = PyEval_GetGlobals();
include/pybind11/pybind11.h:2773:            PyObject *locals = PyEval_GetLocals();
include/pybind11/pybind11.h:2820:                       "        self_caller = frame.f_locals[frame.f_code.co_varnames[0]]\n"
tests/constructor_stats.h:116:        PyObject *globals = PyEval_GetGlobals();

These are now no-ops:

$ git grep -E 'PyFrame_FastToLocalsWithError|PyFrame_FastToLocals|git grep PyFrame_LocalsToFast'
include/pybind11/pybind11.h:2797:        PyFrame_FastToLocals(frame);

If I update to use the new functions (only a couple of instances, the rest are for PyPy), then nothing changes (Edit: never mind, messed up the version number, forgot a 0).

@henryiii henryiii force-pushed the henryiii/ci/py313 branch 2 times, most recently from 0a457ea to b5a125a Compare May 23, 2024 21:59
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
@henryiii henryiii force-pushed the henryiii/ci/py313 branch from b5a125a to 540bef2 Compare May 23, 2024 22:00
@henryiii henryiii changed the title ci: add Python 3.13 fix: support Python 3.13.0b1 (PEP 667 fix) May 23, 2024
@henryiii
Copy link
Collaborator Author

FYI, four tests fail (related to refcounts) when using free-threaded 3.13 but holding the GIL.

@henryiii henryiii merged commit 4d0fced into pybind:master May 24, 2024
82 checks passed
@henryiii henryiii deleted the henryiii/ci/py313 branch May 24, 2024 15:00
@github-actions github-actions bot added the needs changelog Possibly needs a changelog entry label May 24, 2024
PyObject *locals = PyEval_GetLocals();
Py_INCREF(locals);
Copy link
Collaborator

@rwgk rwgk May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@henryiii This Py_INCREF(locals) appears before the locals != nullptr test, therefore I think it could segfault in object.h: op->ob_refcnt++;. Did you consider that already? Did you mean to use Py_XINCREF(locals) here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I did, thanks! I think it’s very unlikely that this will be null, but best to be correct!

rwgk pushed a commit to rwgk/pybind11 that referenced this pull request Jun 12, 2024
* Fix merge accident in pybind11/detail/descr.h (pybind#5086)

This was noticed only when manually reviewing the diffs with the Google review tools.

* Fix typo in changelog date (pybind#5096)

This was actually released in 2024!

* ci: macos-latest is changing to macos-14 ARM runners (pybind#5109)

Committed via https://github.com/asottile/all-repos

* chore: docs and nox bump (pybind#5071)

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* docs(numpy): drop duplicated ndim (pybind#5119)

* chore(deps): bump idna from 3.6 to 3.7 in /docs (pybind#5121)

Bumps [idna](https://github.com/kjd/idna) from 3.6 to 3.7.
- [Release notes](https://github.com/kjd/idna/releases)
- [Changelog](https://github.com/kjd/idna/blob/master/HISTORY.rst)
- [Commits](kjd/idna@v3.6...v3.7)

---
updated-dependencies:
- dependency-name: idna
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump jinja2 from 3.1.3 to 3.1.4 in /docs (pybind#5122)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.3 to 3.1.4.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.3...3.1.4)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump the actions group with 1 update (pybind#5082)

* chore(deps): bump the actions group with 1 update

Bumps the actions group with 1 update: [actions/labeler](https://github.com/actions/labeler).


Updates `actions/labeler` from 4 to 5
- [Release notes](https://github.com/actions/labeler/releases)
- [Commits](actions/labeler@v4...v5)

---
updated-dependencies:
- dependency-name: actions/labeler
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>

* ci: fix labeler

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* ci: move eigen to 64-bit only

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>

* chore(deps): update pre-commit hooks (pybind#5123)

* chore(deps): update pre-commit hooks

updates:
- [github.com/pre-commit/mirrors-clang-format: v18.1.2 → v18.1.4](pre-commit/mirrors-clang-format@v18.1.2...v18.1.4)
- [github.com/astral-sh/ruff-pre-commit: v0.3.5 → v0.4.3](astral-sh/ruff-pre-commit@v0.3.5...v0.4.3)
- [github.com/pre-commit/mirrors-mypy: v1.9.0 → v1.10.0](pre-commit/mirrors-mypy@v1.9.0...v1.10.0)
- [github.com/pre-commit/pre-commit-hooks: v4.5.0 → v4.6.0](pre-commit/pre-commit-hooks@v4.5.0...v4.6.0)
- [github.com/python-jsonschema/check-jsonschema: 0.28.1 → 0.28.2](python-jsonschema/check-jsonschema@0.28.1...0.28.2)

* style: pre-commit fixes

* Update .pre-commit-config.yaml

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>

* --- (pybind#5130)

updated-dependencies:
- dependency-name: requests
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix: support Python 3.13.0b1 (PEP 667 fix) (pybind#5127)

* ci: add Python 3.13

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* tests: run the gc for 3.13+

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* Revert "tests: run the gc for 3.13+"

This reverts commit fe8a3ce.

* ci: drop macos ARM for now, need pin updates

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* fix: use Python 3.13 API if on 3.13

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

* chore: some cleanup (pybind#5137)

* Tracking ci.yml changes from master.

---------

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Tim Stumbaugh <stum@hudson-trading.com>
Co-authored-by: Henry Schreiner <HenrySchreinerIII@gmail.com>
Co-authored-by: nobkd <44443899+nobkd@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@henryiii henryiii removed the needs changelog Possibly needs a changelog entry label Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants