UNSAFE_PYO3_BUILD_FREE_THREADED not checked in 0.22.5

[minrk]

Bug Description

My understanding of #4327 is that build is supposed to fail on free-threaded Python without UNSAFE_PYO3_BUILD_FREE_THREADED=1 since 0.22.2, but leaving this unset, 0.22.5 completes a broken free-threaded install instead of failing with the intended error.

If I change Cargo.toml from:

[dependencies.pyo3]
version = "0.22.5"

to

[dependencies.pyo3]
git = "https://github.com/pyo3/pyo3"
branch = "main"

I do get the expected error:

       error: the Python interpreter was built with the GIL disabled, which is not yet supported by PyO3
        = help: see https://github.com/PyO3/pyo3/issues/4265 for more information
        = help: please check if an updated version of PyO3 is available. Current version: 0.23.0-dev
        = help: set UNSAFE_PYO3_BUILD_FREE_THREADED=1 to suppress this check and build anyway for free-threaded Python

Maybe a missing backport or something?

Steps to Reproduce

1. get free-threaded Python 3.13, e.g. mamba create -n 313t python-freethreading python=3.13
2. python3 -m pip install -v --no-cache rpds-py
3. python3 -c 'import rpds'

It will complete installation and segfault on import. Expected build to fail.

Backtrace

Not a backtrace, but the build output:

> python3 -m pip install --no-cache -v rpds-py
Using pip 24.3.1 from /Users/minrk/conda/envs/313t/lib/python3.13/site-packages/pip (python 3.13)
Collecting rpds-py
  Downloading rpds_py-0.20.1.tar.gz (25 kB)
  Running command pip subprocess to install build dependencies
  Using pip 24.3.1 from /Users/minrk/conda/envs/313t/lib/python3.13/site-packages/pip (python 3.13)
  Collecting maturin<2.0,>=1.2
    Obtaining dependency information for maturin<2.0,>=1.2 from https://files.pythonhosted.org/packages/4f/0a/1b2a2fda3f338dedd219de358d77b01f1138bc4bc6b4c73b6ea01d8ddd9e/maturin-1.7.4-py3-none-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl.metadata
    Using cached maturin-1.7.4-py3-none-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl.metadata (18 kB)
  Using cached maturin-1.7.4-py3-none-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl (15.9 MB)
  Installing collected packages: maturin
  Successfully installed maturin-1.7.4
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
  Running command Preparing metadata (pyproject.toml)
  📦 Including license file "/private/var/folders/qr/3vxfnp1x2t1fw55dr288mphc0000gn/T/pip-install-ey5o35qi/rpds-py_11c0324721ba45958dbf1f9ecf0d9e89/LICENSE"
  🔗 Found pyo3 bindings
  🐍 Found CPython 3.13t at /Users/minrk/conda/envs/313t/bin/python3
  📡 Using build options features from pyproject.toml
  rpds_py-0.20.1.dist-info
  Checking for Rust toolchain....
  Running `maturin pep517 write-dist-info --metadata-directory /private/var/folders/qr/3vxfnp1x2t1fw55dr288mphc0000gn/T/pip-modern-metadata-0e4pg0h2 --interpreter /Users/minrk/conda/envs/313t/bin/python3`
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: rpds-py
  Running command Building wheel for rpds-py (pyproject.toml)
  Running `maturin pep517 build-wheel -i /Users/minrk/conda/envs/313t/bin/python3 --compatibility off`
  📦 Including license file "/private/var/folders/qr/3vxfnp1x2t1fw55dr288mphc0000gn/T/pip-install-ey5o35qi/rpds-py_11c0324721ba45958dbf1f9ecf0d9e89/LICENSE"
  🔗 Found pyo3 bindings
  🐍 Found CPython 3.13t at /Users/minrk/conda/envs/313t/bin/python3
  📡 Using build options features from pyproject.toml
  💻 Using `MACOSX_DEPLOYMENT_TARGET=11.0` for aarch64-apple-darwin by default
     Compiling target-lexicon v0.12.14
     Compiling python3-dll-a v0.2.10
     Compiling once_cell v1.19.0
     Compiling proc-macro2 v1.0.86
     Compiling unicode-ident v1.0.12
     Compiling autocfg v1.3.0
     Compiling libc v0.2.155
     Compiling heck v0.5.0
     Compiling triomphe v0.1.13
     Compiling unindent v0.2.3
     Compiling indoc v2.0.5
     Compiling archery v1.2.1
     Compiling cfg-if v1.0.0
     Compiling rpds v1.1.0
     Compiling memoffset v0.9.1
     Compiling pyo3-build-config v0.22.5
     Compiling quote v1.0.36
     Compiling syn v2.0.69
     Compiling pyo3-macros-backend v0.22.5
     Compiling pyo3-ffi v0.22.5
     Compiling pyo3 v0.22.5
     Compiling pyo3-macros v0.22.5
     Compiling rpds-py v0.20.1 (/private/var/folders/qr/3vxfnp1x2t1fw55dr288mphc0000gn/T/pip-install-ey5o35qi/rpds-py_11c0324721ba45958dbf1f9ecf0d9e89)
      Finished `release` profile [optimized] target(s) in 8.51s
  📖 Found type stub file at rpds.pyi
  📦 Built wheel for CPython 3.13t to /private/var/folders/qr/3vxfnp1x2t1fw55dr288mphc0000gn/T/pip-install-ey5o35qi/rpds-py_11c0324721ba45958dbf1f9ecf0d9e89/target/wheels/rpds_py-0.20.1-cp313-cp313t-macosx_11_0_arm64.whl
  /private/var/folders/qr/3vxfnp1x2t1fw55dr288mphc0000gn/T/pip-install-ey5o35qi/rpds-py_11c0324721ba45958dbf1f9ecf0d9e89/target/wheels/rpds_py-0.20.1-cp313-cp313t-macosx_11_0_arm64.whl
  Building wheel for rpds-py (pyproject.toml) ... done
  Created wheel for rpds-py: filename=rpds_py-0.20.1-cp313-cp313t-macosx_11_0_arm64.whl size=334849 sha256=97af520be70b284576fe8c21fe7cf71e02568431ff77a6da68b22d88680b92df
  Stored in directory: /private/var/folders/qr/3vxfnp1x2t1fw55dr288mphc0000gn/T/pip-ephem-wheel-cache-vngxfqkf/wheels/0c/e4/ec/6118a889ea0298bd8e1f0eb4cfc097da268089d3fc99638399
Successfully built rpds-py
Installing collected packages: rpds-py
Successfully installed rpds-py-0.20.1

Your operating system and version

macOS 15.0.1 (ARM M1)

Your Python version (python --version)

Python 3.13.0

Your Rust version (rustc --version)

rustc 1.82.0 (f6e511eec 2024-10-15) (Homebrew)

Your PyO3 version

0.22.5

How did you install python? Did you use a virtualenv?

mamba create -n 313t -c conda-forge python-freethreading python=3.13

Additional Info

No response

[ngoldbaum]

Hmm, I guess the lack of testing for this functionality might have been a problem.

It looks like the logic inside of pyo3_build_config to get the sysconfig variables is broken on 3.13.0t:

[pyo3-build-config/src/impl_.rs:1087:62] &script = "import sysconfig\nconfig = sysconfig.get_config_vars()\nprint(config.get('Py_DEBUG', '0'))\nprint(config.get('Py_REF_DEBUG', '0'))\nprint(config.get('Py_TRACE_REFS', '0'))\nprint(config.get('COUNT_ALLOCS', '0'))\n"
[pyo3-build-config/src/impl_.rs:1088:39] stdout.trim_end().lines().collect() = [
    "0",
    "0",
    "0",
    "0",
]

[ngoldbaum]

Ah I see, I guess it just never worked at all on the 0.22 branch? And I never put two and two together when I first started working on this that I should have been seeing build errors.

The following patch to the 0.22 branch fixes the issue:

± git diff
diff --git a/pyo3-build-config/src/impl_.rs b/pyo3-build-config/src/impl_.rs
index 674691b2..016d79fa 100644
--- a/pyo3-build-config/src/impl_.rs
+++ b/pyo3-build-config/src/impl_.rs
@@ -996,6 +996,7 @@ pub enum BuildFlag {
     Py_DEBUG,
     Py_REF_DEBUG,
     Py_TRACE_REFS,
+    Py_GIL_DISABLED,
     COUNT_ALLOCS,
     Other(String),
 }
@@ -1040,10 +1041,11 @@ impl FromStr for BuildFlag {
 pub struct BuildFlags(pub HashSet<BuildFlag>);

 impl BuildFlags {
-    const ALL: [BuildFlag; 4] = [
+    const ALL: [BuildFlag; 5] = [
         BuildFlag::Py_DEBUG,
         BuildFlag::Py_REF_DEBUG,
         BuildFlag::Py_TRACE_REFS,
+        BuildFlag::Py_GIL_DISABLED,
         BuildFlag::COUNT_ALLOCS,
     ];

Given the impending release of 0.23, I'm not sure what we want to do. I guess another 0.22 release that fixes this is probably needed to fix the runtime segfaults while the ecosystem transitions to 0.23?

[davidhewitt]

🤦 I think I must have broken this in the backport. @ngoldbaum the problem is that I didn't want to change the BuildFlag enum in a patch release (as that's theoretically breaking), which is why I didn't include that bit. I missed that it would (of course) break the extraction.

[davidhewitt]

Fixed in #4684