poetry add 'gym[box2d]' fails while pip install 'gym[box2d]' within virtual environment

[manucalop]

Poetry
Version: 1.2.2
Python: 3.10.7

Virtualenv
Python: 3.10.7
Implementation: CPython
Path: /Users/manucalop/main_ws/projects/ucjc/cailab_rl_course/.venv
Executable: /Users/manucalop/main_ws/projects/ucjc/cailab_rl_course/.venv/bin/python
Valid: True

System
Platform: darwin
OS: posix
Python: 3.10.7
Path: /Users/manucalop/.asdf/installs/python/3.10.7
Executable: /Users/manucalop/.asdf/installs/python/3.10.7/bin/python3.10

• pyproject.toml:
• <script src="https://gist.github.com/manucalop/453304bc7adf1d8ec72df527aab6e145.js"></script>

Issue

When I do

poetry add 'gym[box2d]' 

The following error arise:

Using version ^0.26.2 for gym

Updating dependencies
Resolving dependencies... (0.3s)

Package operations: 2 installs, 0 updates, 0 removals

  • Installing box2d-py (2.3.5): Failed

  CalledProcessError

  Command '['/Users/manucalop/main_ws/projects/ucjc/cailab_rl_course/.venv/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--prefix', '/Users/manucalop/main_ws/projects/ucjc/cailab_rl_course/.venv', '--no-deps', '/Users/manucalop/Library/Caches/pypoetry/artifacts/60/cc/2f/724811aa156554a55c090fe48fde3202d997b991b26f569cf0d834da60/box2d-py-2.3.5.tar.gz']' returned non-zero exit status 1.

  at ~/.asdf/installs/python/3.10.7/lib/python3.10/subprocess.py:524 in run
       520│             # We don't call process.wait() as .__exit__ does that for us.
       521│             raise
       522│         retcode = process.poll()
       523│         if check and retcode:
    →  524│             raise CalledProcessError(retcode, process.args,
       525│                                      output=stdout, stderr=stderr)
       526│     return CompletedProcess(process.args, retcode, stdout, stderr)
       527│
       528│

The following error occurred when trying to handle this error:


  EnvCommandError

  Command ['/Users/manucalop/main_ws/projects/ucjc/cailab_rl_course/.venv/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--prefix', '/Users/manucalop/main_ws/projects/ucjc/cailab_rl_course/.venv', '--no-deps', '/Users/manucalop/Library/Caches/pypoetry/artifacts/60/cc/2f/724811aa156554a55c090fe48fde3202d997b991b26f569cf0d834da60/box2d-py-2.3.5.tar.gz'] errored with the following return code 1, and output:
  Processing /Users/manucalop/Library/Caches/pypoetry/artifacts/60/cc/2f/724811aa156554a55c090fe48fde3202d997b991b26f569cf0d834da60/box2d-py-2.3.5.tar.gz
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Building wheels for collected packages: box2d-py
    Building wheel for box2d-py (pyproject.toml): started
    Building wheel for box2d-py (pyproject.toml): finished with status 'error'
    error: subprocess-exited-with-error

    × Building wheel for box2d-py (pyproject.toml) did not run successfully.
    │ exit code: 1
    ╰─> [20 lines of output]
        Using setuptools (version 65.5.0).
        running bdist_wheel
        running build
        running build_py
        creating build
        creating build/lib.macosx-12.6-arm64-cpython-310
        creating build/lib.macosx-12.6-arm64-cpython-310/Box2D
        copying library/Box2D/Box2D.py -> build/lib.macosx-12.6-arm64-cpython-310/Box2D
        copying library/Box2D/__init__.py -> build/lib.macosx-12.6-arm64-cpython-310/Box2D
        creating build/lib.macosx-12.6-arm64-cpython-310/Box2D/b2
        copying library/Box2D/b2/__init__.py -> build/lib.macosx-12.6-arm64-cpython-310/Box2D/b2
        running build_ext
        building 'Box2D._Box2D' extension
        swigging Box2D/Box2D.i to Box2D/Box2D_wrap.cpp
        swig -python -c++ -IBox2D -small -O -includeall -ignoremissing -w201 -globals b2Globals -outdir library/Box2D -keyword -w511 -D_SWIG_KWARGS -o Box2D/Box2D_wrap.cpp Box2D/Box2D.i
        Traceback (most recent call last):
          File "/Users/manucalop/main_ws/projects/ucjc/cailab_rl_course/.venv/bin/swig", line 5, in <module>
            from swig import swig
        ModuleNotFoundError: No module named 'swig'
        error: command '/Users/manucalop/main_ws/projects/ucjc/cailab_rl_course/.venv/bin/swig' failed with exit code 1
        [end of output]

    note: This error originates from a subprocess, and is likely not a problem with pip.
    ERROR: Failed building wheel for box2d-py
  Failed to build box2d-py
  ERROR: Could not build wheels for box2d-py, which is required to install pyproject.toml-based projects


  at ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env.py:1476 in _run
      1472│                 output = subprocess.check_output(
      1473│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1474│                 )
      1475│         except CalledProcessError as e:
    → 1476│             raise EnvCommandError(e, input=input_)
      1477│
      1478│         return decode(output)
      1479│
      1480│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  PoetryException

  Failed to install /Users/manucalop/Library/Caches/pypoetry/artifacts/60/cc/2f/724811aa156554a55c090fe48fde3202d997b991b26f569cf0d834da60/box2d-py-2.3.5.tar.gz

  at ~/Library/Application Support/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/pip.py:51 in pip_install
       47│
       48│     try:
       49│         return environment.run_pip(*args)
       50│     except EnvCommandError as e:
    →  51│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       52│

However, when I do

poetry shell
pip install 'gym[box2d]'

it gets installed without problems.

Swig was installed using

brew install swig

[neersighted]

Looks like an issue with undeclared build-time dependencies in box2d-py -- pip install --use-pep517 gym[box2d] uses the same build-time isolation Poetry does during install, and reproduces the failure.

See #6658 for more details/concrete examples of how to resolve this in the upstream (box2d-py) project.

[manucalop]

I wasn't able to reproduce it with @neersighted. How did you do it?

(cailab-rl-course-py3.10) ➜  cailab_rl_course git:(main) ✗ pip install --use-pep517 --no-cache-dir 'gym[box2d]'
Collecting gym[box2d]
  Downloading gym-0.26.2.tar.gz (721 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 721.7/721.7 kB 7.5 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting numpy>=1.18.0
  Downloading numpy-1.23.4-cp310-cp310-macosx_11_0_arm64.whl (13.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.3/13.3 MB 10.8 MB/s eta 0:00:00
Collecting gym-notices>=0.0.4
  Downloading gym_notices-0.0.8-py3-none-any.whl (3.0 kB)
Collecting cloudpickle>=1.2.0
  Downloading cloudpickle-2.2.0-py3-none-any.whl (25 kB)
Collecting swig==4.*
  Downloading swig-4.0.2-py2.py3-none-macosx_10_9_universal2.whl (1.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 10.8 MB/s eta 0:00:00
Collecting pygame==2.1.0
  Downloading pygame-2.1.0-cp310-cp310-macosx_11_0_arm64.whl (4.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.8/4.8 MB 10.7 MB/s eta 0:00:00
Collecting box2d-py==2.3.5
  Downloading box2d-py-2.3.5.tar.gz (374 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 374.4/374.4 kB 11.2 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: box2d-py, gym
  Building wheel for box2d-py (pyproject.toml) ... done
  Created wheel for box2d-py: filename=box2d_py-2.3.5-cp310-cp310-macosx_12_0_arm64.whl size=477705 sha256=efa5137afa3c14f287a6183f06fd8824979174bbb21e0d6abb3585a598b50bc9
  Stored in directory: /private/var/folders/2h/phlf1r514bn_y23wmn06r9wr0000gn/T/pip-ephem-wheel-cache-zk3btq5h/wheels/db/8f/6a/eaaadf056fba10a98d986f6dce954e6201ba3126926fc5ad9e
  Building wheel for gym (pyproject.toml) ... done
  Created wheel for gym: filename=gym-0.26.2-py3-none-any.whl size=827635 sha256=d6ddb036ba89bbbee541cec067786df567a4cdd8f0eced354d740ab584066c18
  Stored in directory: /private/var/folders/2h/phlf1r514bn_y23wmn06r9wr0000gn/T/pip-ephem-wheel-cache-zk3btq5h/wheels/b9/22/6d/3e7b32d98451b4cd9d12417052affbeeeea012955d437da1da
Successfully built box2d-py gym
Installing collected packages: swig, gym-notices, box2d-py, pygame, numpy, cloudpickle, gym
Successfully installed box2d-py-2.3.5 cloudpickle-2.2.0 gym-0.26.2 gym-notices-0.0.8 numpy-1.23.4 pygame-2.1.0 swig-4.0.2

[neersighted]

$ docker run --rm -it --entrypoint /bin/bash python:3.10                                                                                                                                                                                     
root@3d97ab8e5de8:/# pip install --use-pep517 gym[box2d]
Collecting gym[box2d]
  Downloading gym-0.26.2.tar.gz (721 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 721.7/721.7 kB 7.7 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting numpy>=1.18.0
  Downloading numpy-1.23.4-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (13.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.9/13.9 MB 61.2 MB/s eta 0:00:00
Collecting cloudpickle>=1.2.0
  Downloading cloudpickle-2.2.0-py3-none-any.whl (25 kB)
Collecting gym-notices>=0.0.4
  Downloading gym_notices-0.0.8-py3-none-any.whl (3.0 kB)
Collecting box2d-py==2.3.5
  Downloading box2d-py-2.3.5.tar.gz (374 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 374.4/374.4 kB 46.6 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting swig==4.*
  Downloading swig-4.0.2-py2.py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (4.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.7/4.7 MB 51.7 MB/s eta 0:00:00
Collecting pygame==2.1.0
  Downloading pygame-2.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (17.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.2/17.2 MB 41.4 MB/s eta 0:00:00
Building wheels for collected packages: box2d-py, gym
  Building wheel for box2d-py (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for box2d-py (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]
      Using setuptools (version 65.5.0).
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-aarch64-cpython-310
      creating build/lib.linux-aarch64-cpython-310/Box2D
      copying library/Box2D/Box2D.py -> build/lib.linux-aarch64-cpython-310/Box2D
      copying library/Box2D/__init__.py -> build/lib.linux-aarch64-cpython-310/Box2D
      creating build/lib.linux-aarch64-cpython-310/Box2D/b2
      copying library/Box2D/b2/__init__.py -> build/lib.linux-aarch64-cpython-310/Box2D/b2
      running build_ext
      building 'Box2D._Box2D' extension
      swigging Box2D/Box2D.i to Box2D/Box2D_wrap.cpp
      swig -python -c++ -IBox2D -small -O -includeall -ignoremissing -w201 -globals b2Globals -outdir library/Box2D -keyword -w511 -D_SWIG_KWARGS -o Box2D/Box2D_wrap.cpp Box2D/Box2D.i
      error: command 'swig' failed: No such file or directory
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for box2d-py
  Building wheel for gym (pyproject.toml) ... done
  Created wheel for gym: filename=gym-0.26.2-py3-none-any.whl size=827633 sha256=0190991e62afbdcb73b202ba0050485bd5f496092394921a65a679f8b1fc6237
  Stored in directory: /root/.cache/pip/wheels/b9/22/6d/3e7b32d98451b4cd9d12417052affbeeeea012955d437da1da
Successfully built gym
Failed to build box2d-py
ERROR: Could not build wheels for box2d-py, which is required to install pyproject.toml-based projects

I suspect your externally installed swig was picked up -- this might work for Poetry as well, but given that swig is distributed as a wheel and is meant to be used as a build-system requirement (https://pypi.org/project/swig/), I would still assert that this should be fixed in the upstream package.

[manucalop]

Amazing. Thanks!

[craig-sony]

This still seems to be an issue and I don't see a solution listed here, so I don't know why it's closed. Adding swig as a build-system requirement does not seem to work.

[dimbleby]

it's closed because it's not a poetry problem. The open issue is jonasschneider/box2d-py#1.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.