`pip-compile` resolves against internal python version #82

CharlesChen0823 · 2023-05-01T12:47:15Z

execute cmd

rye init testrye
rye pin cpython@3.7
rye sync
rye add flask
rye sync

then failed.

I found that the requirements using pip-compile found for flask is not match the python version 3.7.
According search the result in google, I found that the tools pip-compile cannot specify python-version, so I thought the method of using pip-compile with self env is not proper.

The text was updated successfully, but these errors were encountered:

mitsuhiko · 2023-05-01T13:35:21Z

Please share the output. Rye is using pip-tools behind the scenes in a way that Python versions should be accurately reflected.

CharlesChen0823 · 2023-05-01T13:47:54Z

ubuntu@VM-16-15-ubuntu:~/program/python$ rye init testrye
success: Initialized project in /home/ubuntu/program/python/testrye
ubuntu@VM-16-15-ubuntu:~/program/python$ cd testrye/
ubuntu@VM-16-15-ubuntu:~/program/python/testrye$ rye pin cpython@3.7
pinned 3.7.9 in /home/ubuntu/program/python/testrye/.python-version
ubuntu@VM-16-15-ubuntu:~/program/python/testrye$ rye sync
Initializing new virtualenv in /home/ubuntu/program/python/testrye/.venv
Python version: cpython@3.7.9
Generating production lockfile: /home/ubuntu/program/python/testrye/requirements.lock
Generating dev lockfile: /home/ubuntu/program/python/testrye/requirements-dev.lock
Installing dependencies
Looking in indexes: http://mirrors.tencentyun.com/pypi/simple
Obtaining file:///. (from -r /tmp/tmps7dtdf2_ (line 1))
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: testrye
  Building editable for testrye (pyproject.toml) ... done
  Created wheel for testrye: filename=testrye-0.1.0-py3-none-any.whl size=974 sha256=6618d278b5131d04b2122dd1168fd47fbb33aa75387a61b1f5d9aa23ea931623
  Stored in directory: /tmp/pip-ephem-wheel-cache-rx8o29g3/wheels/3d/fd/25/9e94f5ebf536886bdc86d1d95c38af8d2fcdcd085944f114d4
Successfully built testrye
Installing collected packages: testrye
Successfully installed testrye-0.1.0
Done!
ubuntu@VM-16-15-ubuntu:~/program/python/testrye$ rye add flask
Added flask~=2.2.4
ubuntu@VM-16-15-ubuntu:~/program/python/testrye$ rye sync
Reusing already existing virtualenv
Generating production lockfile: /home/ubuntu/program/python/testrye/requirements.lock
Generating dev lockfile: /home/ubuntu/program/python/testrye/requirements-dev.lock
Installing dependencies
Found existing installation: testrye 0.1.0
Uninstalling testrye-0.1.0:
  Successfully uninstalled testrye-0.1.0
Looking in indexes: http://mirrors.tencentyun.com/pypi/simple
Obtaining file:///. (from -r /tmp/tmpvvmw0p9c (line 2))
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting click==8.1.3 (from -r /tmp/tmpvvmw0p9c (line 1))
  Using cached http://mirrors.tencentyun.com/pypi/packages/c2/f1/df59e28c642d583f7dacffb1e0965d0e00b218e0186d7858ac5233dce840/click-8.1.3-py3-none-any.whl (96 kB)
Collecting flask==2.2.4 (from -r /tmp/tmpvvmw0p9c (line 3))
  Using cached http://mirrors.tencentyun.com/pypi/packages/cf/e6/cfd7227e18fc44a56594a7b7df21b7ac63954ea652987e6da7707aba6064/Flask-2.2.4-py3-none-any.whl (101 kB)
Collecting itsdangerous==2.1.2 (from -r /tmp/tmpvvmw0p9c (line 4))
  Using cached http://mirrors.tencentyun.com/pypi/packages/68/5f/447e04e828f47465eeab35b5d408b7ebaaaee207f48b7136c5a7267a30ae/itsdangerous-2.1.2-py3-none-any.whl (15 kB)
Collecting jinja2==3.1.2 (from -r /tmp/tmpvvmw0p9c (line 5))
  Using cached http://mirrors.tencentyun.com/pypi/packages/bc/c3/f068337a370801f372f2f8f6bad74a5c140f6fda3d9de154052708dd3c65/Jinja2-3.1.2-py3-none-any.whl (133 kB)
Collecting markupsafe==2.1.2 (from -r /tmp/tmpvvmw0p9c (line 6))
  Using cached http://mirrors.tencentyun.com/pypi/packages/95/88/8c8cce021ac1b1eedde349c6a41f6c256da60babf95e572071361ff3f66b/MarkupSafe-2.1.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
ERROR: Ignored the following versions that require a different python version: 2.3.0 Requires-Python >=3.8; 2.3.1 Requires-Python >=3.8; 2.3.2 Requires-Python >=3.8
ERROR: Could not find a version that satisfies the requirement werkzeug==2.3.2 (from versions: 0.1, 0.2, 0.3, 0.3.1, 0.4, 0.4.1, 0.5, 0.5.1, 0.6, 0.6.1, 0.6.2, 0.7, 0.7.1, 0.7.2, 0.8, 0.8.1, 0.8.2, 0.8.3, 0.9, 0.9.1, 0.9.2, 0.9.3, 0.9.4, 0.9.5, 0.9.6, 0.10, 0.10.1, 0.10.2, 0.10.4, 0.11, 0.11.1, 0.11.2, 0.11.3, 0.11.4, 0.11.5, 0.11.6, 0.11.7, 0.11.8, 0.11.9, 0.11.10, 0.11.11, 0.11.12, 0.11.13, 0.11.14, 0.11.15, 0.12, 0.12.1, 0.12.2, 0.13, 0.14, 0.14.1, 0.15.0, 0.15.1, 0.15.2, 0.15.3, 0.15.4, 0.15.5, 0.15.6, 0.16.0, 0.16.1, 1.0.0rc1, 1.0.0, 1.0.1, 2.0.0rc1, 2.0.0rc2, 2.0.0rc3, 2.0.0rc4, 2.0.0rc5, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0, 2.1.1, 2.1.2, 2.2.0a1, 2.2.0, 2.2.1, 2.2.2, 2.2.3)
ERROR: No matching distribution found for werkzeug==2.3.2
Traceback (most recent call last):
  File "/home/ubuntu/.rye/self/bin/pip-sync", line 8, in <module>
    sys.exit(cli())
  File "/home/ubuntu/.rye/self/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/ubuntu/.rye/self/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/ubuntu/.rye/self/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ubuntu/.rye/self/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/ubuntu/.rye/self/lib/python3.10/site-packages/piptools/scripts/sync.py", line 174, in cli
    sync.sync(
  File "/home/ubuntu/.rye/self/lib/python3.10/site-packages/piptools/sync.py", line 244, in sync
    run(  # nosec
  File "/home/ubuntu/.rye/py/cpython@3.10.9/install/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/ubuntu/program/python/testrye/.venv/bin/python', '-m', 'pip', 'install', '-r', '/tmp/tmpvvmw0p9c', '--index-url', 'http://mirrors.tencentyun.com/pypi/simple', '--trusted-host', 'mirrors.tencentyun.com', '--python=/home/ubuntu/program/python/testrye/.venv/bin/python']' returned non-zero exit status 1.
Error: Installation of dependencies failed

CharlesChen0823 · 2023-05-01T13:55:40Z

I found that the pip-tools using the self venv python version, not the pin python version.

mitsuhiko · 2023-05-01T15:07:21Z

Can you please paste the output of rye --version?

CharlesChen0823 · 2023-05-01T16:50:50Z

I have reproduced on my mac with the latest version.

[0:45:04] @.local ~/program/python/testrye % rye sync
Reusing already existing virtualenv
Generating production lockfile: /Users/charles/program/python/testrye/requirements.lock
Generating dev lockfile: /Users/charles/program/python/testrye/requirements-dev.lock
Installing dependencies
Looking in indexes: http://mirrors.aliyun.com/pypi/simple
Obtaining file:///. (from -r /var/folders/z4/4q0sj3vn5fvbmp8v961pnkl40000gn/T/tmpqlhzkfuz (line 2))
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting click==8.1.3 (from -r /var/folders/z4/4q0sj3vn5fvbmp8v961pnkl40000gn/T/tmpqlhzkfuz (line 1))
  Using cached http://mirrors.aliyun.com/pypi/packages/c2/f1/df59e28c642d583f7dacffb1e0965d0e00b218e0186d7858ac5233dce840/click-8.1.3-py3-none-any.whl (96 kB)
Collecting flask==2.2.4 (from -r /var/folders/z4/4q0sj3vn5fvbmp8v961pnkl40000gn/T/tmpqlhzkfuz (line 3))
  Downloading http://mirrors.aliyun.com/pypi/packages/cf/e6/cfd7227e18fc44a56594a7b7df21b7ac63954ea652987e6da7707aba6064/Flask-2.2.4-py3-none-any.whl (101 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 101.7/101.7 kB 105.0 kB/s eta 0:00:00
Collecting itsdangerous==2.1.2 (from -r /var/folders/z4/4q0sj3vn5fvbmp8v961pnkl40000gn/T/tmpqlhzkfuz (line 4))
  Downloading http://mirrors.aliyun.com/pypi/packages/68/5f/447e04e828f47465eeab35b5d408b7ebaaaee207f48b7136c5a7267a30ae/itsdangerous-2.1.2-py3-none-any.whl (15 kB)
Collecting jinja2==3.1.2 (from -r /var/folders/z4/4q0sj3vn5fvbmp8v961pnkl40000gn/T/tmpqlhzkfuz (line 5))
  Downloading http://mirrors.aliyun.com/pypi/packages/bc/c3/f068337a370801f372f2f8f6bad74a5c140f6fda3d9de154052708dd3c65/Jinja2-3.1.2-py3-none-any.whl (133 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 kB 57.6 kB/s eta 0:00:00
Collecting markupsafe==2.1.2 (from -r /var/folders/z4/4q0sj3vn5fvbmp8v961pnkl40000gn/T/tmpqlhzkfuz (line 6))
  Downloading http://mirrors.aliyun.com/pypi/packages/7b/0f/0e99c2f342933c43af69849a6ba63f2eef54e14c6d0e10a26470fb6b40a9/MarkupSafe-2.1.2-cp37-cp37m-macosx_10_9_x86_64.whl (13 kB)
ERROR: Ignored the following versions that require a different python version: 2.3.0 Requires-Python >=3.8; 2.3.1 Requires-Python >=3.8; 2.3.2 Requires-Python >=3.8; 2.3.3 Requires-Python >=3.8
ERROR: Could not find a version that satisfies the requirement werkzeug==2.3.3 (from versions: 0.1, 0.2, 0.3, 0.3.1, 0.4, 0.4.1, 0.5, 0.5.1, 0.6, 0.6.1, 0.6.2, 0.7, 0.7.1, 0.7.2, 0.8, 0.8.1, 0.8.2, 0.8.3, 0.9, 0.9.1, 0.9.2, 0.9.3, 0.9.4, 0.9.5, 0.9.6, 0.10, 0.10.1, 0.10.2, 0.10.4, 0.11, 0.11.1, 0.11.2, 0.11.3, 0.11.4, 0.11.5, 0.11.6, 0.11.7, 0.11.8, 0.11.9, 0.11.10, 0.11.11, 0.11.12, 0.11.13, 0.11.14, 0.11.15, 0.12, 0.12.1, 0.12.2, 0.13, 0.14, 0.14.1, 0.15.0, 0.15.1, 0.15.2, 0.15.3, 0.15.4, 0.15.5, 0.15.6, 0.16.0, 0.16.1, 1.0.0rc1, 1.0.0, 1.0.1, 2.0.0rc1, 2.0.0rc2, 2.0.0rc3, 2.0.0rc4, 2.0.0rc5, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0, 2.1.1, 2.1.2, 2.2.0a1, 2.2.0, 2.2.1, 2.2.2, 2.2.3)
ERROR: No matching distribution found for werkzeug==2.3.3
Traceback (most recent call last):
  File "/Users/charles/.rye/self/bin/pip-sync", line 8, in <module>
    sys.exit(cli())
  File "/Users/charles/.rye/self/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/charles/.rye/self/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/charles/.rye/self/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/charles/.rye/self/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/charles/.rye/self/lib/python3.10/site-packages/piptools/scripts/sync.py", line 174, in cli
    sync.sync(
  File "/Users/charles/.rye/self/lib/python3.10/site-packages/piptools/sync.py", line 244, in sync
    run(  # nosec
  File "/Users/charles/.rye/py/cpython@3.10.9/install/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/Users/charles/program/python/testrye/.venv/bin/python', '-m', 'pip', 'install', '-r', '/var/folders/z4/4q0sj3vn5fvbmp8v961pnkl40000gn/T/tmpqlhzkfuz', '--index-url', 'http://mirrors.aliyun.com/pypi/simple', '--trusted-host', 'mirrors.aliyun.com', '--python=/Users/charles/program/python/testrye/.venv/bin/python', '--no-deps']' returned non-zero exit status 1.
Error: Installation of dependencies failed
[0:48:16] @.local ~/program/python/testrye % rye --version
rye 0.1.0
commit: unknown (e51b54bdf 2023-05-01)
platform: macos (x86_64)
self-python: cpython@3.10

commit: unknown (e51b54b 2023-05-01)
platform: macos (x86_64)
self-python: cpython@3.10

mitsuhiko · 2023-05-01T19:55:29Z

Ah damn. I thought i managed to get piptools to honor the right python version. Looks like more hackery is needed.

mitsuhiko · 2023-05-01T21:33:00Z

It does indeed sound like pip-compile cannot be coerced into behaving like I want. Probably even for a more functional prototype rye would have to move away from pip-tools for something else.

cnpryer · 2023-05-01T21:37:44Z

Probably even for a more functional prototype rye would have to move away from pip-tools for something else

This is raw but I thought it was worth sharing https://github.com/konstin/monotrail-resolve.

davegaeddert · 2023-05-01T21:40:47Z

@mitsuhiko you've probably considered this too (feel free to ignore me if so!), but I've been wondering if the newish pip install --dry-run --report could be used? https://pip.pypa.io/en/stable/reference/installation-report/

mitsuhiko · 2023-05-02T06:39:25Z

@davegaeddert I did play around with it, but it does not seem like it's particularly cooperative:

$ ~/.rye/self/bin/pip install \
  --ignore-installed --dry-run --quiet \
  --report - --python-version=3.7 --only-binary=:all: --target=/tmp flask
ERROR: Cannot install flask because these package versions have conflicting dependencies.

I also tried patching around in pip's internal resolver to pass py_version_info explicitly but it's not doing anything positive.

CharlesChen0823 · 2023-05-02T08:24:03Z

I tried run this command

~/.rye/self/bin/pip install --ignore-installed --dry-run --quiet --report - flask --python-version=3.7 --only-binary=:all: -target=linux

and return

Output

{
  "version": "1",
  "pip_version": "23.1.2",
  "install": [
    {
      "download_info": {
        "url": "http://mirrors.aliyun.com/pypi/packages/cf/e6/cfd7227e18fc44a56594a7b7df21b7ac63954ea652987e6da7707aba6064/Flask-2.2.4-py3-none-any.whl",
        "archive_info": {
          "hash": "sha256=13f6329ddbfff11340939cd11919daf150a01358ded4b7e81c03c055dfecb559",
          "hashes": {
            "sha256": "13f6329ddbfff11340939cd11919daf150a01358ded4b7e81c03c055dfecb559"
          }
        }
      },
      "is_direct": false,
      "requested": true,
      "metadata": {
        "metadata_version": "2.1",
        "name": "Flask",
        "version": "2.2.4",
        "summary": "A simple framework for building complex web applications.",
        "description_content_type": "text/x-rst",
        "home_page": "https://palletsprojects.com/p/flask",
        "author": "Armin Ronacher",
        "author_email": "armin.ronacher@active-4.com",
        "maintainer": "Pallets",
        "maintainer_email": "contact@palletsprojects.com",
        "license": "BSD-3-Clause",
        "classifier": [
          "Development Status :: 5 - Production/Stable",
          "Environment :: Web Environment",
          "Framework :: Flask",
          "Intended Audience :: Developers",
          "License :: OSI Approved :: BSD License",
          "Operating System :: OS Independent",
          "Programming Language :: Python",
          "Topic :: Internet :: WWW/HTTP :: Dynamic Content",
          "Topic :: Internet :: WWW/HTTP :: WSGI",
          "Topic :: Internet :: WWW/HTTP :: WSGI :: Application",
          "Topic :: Software Development :: Libraries :: Application Frameworks"
        ],
        "requires_dist": [
          "Werkzeug (>=2.2.2)",
          "Jinja2 (>=3.0)",
          "itsdangerous (>=2.0)",
          "click (>=8.0)",
          "importlib-metadata (>=3.6.0) ; python_version < \"3.10\"",
          "asgiref (>=3.2) ; extra == 'async'",
          "python-dotenv ; extra == 'dotenv'"
        ],
        "requires_python": ">=3.7",
        "project_url": [
          "Donate, https://palletsprojects.com/donate",
          "Documentation, https://flask.palletsprojects.com/",
          "Changes, https://flask.palletsprojects.com/changes/",
          "Source Code, https://github.com/pallets/flask/",
          "Issue Tracker, https://github.com/pallets/flask/issues/",
          "Twitter, https://twitter.com/PalletsTeam",
          "Chat, https://discord.gg/pallets"
        ],
        "provides_extra": [
          "async",
          "dotenv"
        ],
        "description": ""
      }
    },
    {
      "download_info": {
        "url": "http://mirrors.aliyun.com/pypi/packages/c2/f1/df59e28c642d583f7dacffb1e0965d0e00b218e0186d7858ac5233dce840/click-8.1.3-py3-none-any.whl",
        "archive_info": {
          "hash": "sha256=bb4d8133cb15a609f44e8213d9b391b0809795062913b383c62be0ee95b1db48",
          "hashes": {
            "sha256": "bb4d8133cb15a609f44e8213d9b391b0809795062913b383c62be0ee95b1db48"
          }
        }
      },
      "is_direct": false,
      "requested": false,
      "metadata": {
        "metadata_version": "2.1",
        "name": "click",
        "version": "8.1.3",
        "platform": [
          "UNKNOWN"
        ],
        "summary": "Composable command line interface toolkit",
        "description_content_type": "text/x-rst",
        "home_page": "https://palletsprojects.com/p/click/",
        "author": "Armin Ronacher",
        "author_email": "armin.ronacher@active-4.com",
        "maintainer": "Pallets",
        "maintainer_email": "contact@palletsprojects.com",
        "license": "BSD-3-Clause",
        "classifier": [
          "Development Status :: 5 - Production/Stable",
          "Intended Audience :: Developers",
          "License :: OSI Approved :: BSD License",
          "Operating System :: OS Independent",
          "Programming Language :: Python"
        ],
        "requires_dist": [
          "colorama ; platform_system == \"Windows\"",
          "importlib-metadata ; python_version < \"3.8\""
        ],
        "requires_python": ">=3.7",
        "project_url": [
          "Donate, https://palletsprojects.com/donate",
          "Documentation, https://click.palletsprojects.com/",
          "Changes, https://click.palletsprojects.com/changes/",
          "Source Code, https://github.com/pallets/click/",
          "Issue Tracker, https://github.com/pallets/click/issues/",
          "Twitter, https://twitter.com/PalletsTeam",
          "Chat, https://discord.gg/pallets"
        ],
        "description": ""
      }
    },
    {
      "download_info": {
        "url": "http://mirrors.aliyun.com/pypi/packages/68/5f/447e04e828f47465eeab35b5d408b7ebaaaee207f48b7136c5a7267a30ae/itsdangerous-2.1.2-py3-none-any.whl",
        "archive_info": {
          "hash": "sha256=2c2349112351b88699d8d4b6b075022c0808887cb7ad10069318a8b0bc88db44",
          "hashes": {
            "sha256": "2c2349112351b88699d8d4b6b075022c0808887cb7ad10069318a8b0bc88db44"
          }
        }
      },
      "is_direct": false,
      "requested": false,
      "metadata": {
        "metadata_version": "2.1",
        "name": "itsdangerous",
        "version": "2.1.2",
        "platform": [
          "UNKNOWN"
        ],
        "summary": "Safely pass data to untrusted environments and back.",
        "description_content_type": "text/x-rst",
        "home_page": "https://palletsprojects.com/p/itsdangerous/",
        "author": "Armin Ronacher",
        "author_email": "armin.ronacher@active-4.com",
        "maintainer": "Pallets",
        "maintainer_email": "contact@palletsprojects.com",
        "license": "BSD-3-Clause",
        "classifier": [
          "Development Status :: 5 - Production/Stable",
          "Intended Audience :: Developers",
          "License :: OSI Approved :: BSD License",
          "Operating System :: OS Independent",
          "Programming Language :: Python"
        ],
        "requires_python": ">=3.7",
        "project_url": [
          "Donate, https://palletsprojects.com/donate",
          "Documentation, https://itsdangerous.palletsprojects.com/",
          "Changes, https://itsdangerous.palletsprojects.com/changes/",
          "Source Code, https://github.com/pallets/itsdangerous/",
          "Issue Tracker, https://github.com/pallets/itsdangerous/issues/",
          "Twitter, https://twitter.com/PalletsTeam",
          "Chat, https://discord.gg/pallets"
        ],
        "description": ""
      }
    },
    {
      "download_info": {
        "url": "http://mirrors.aliyun.com/pypi/packages/bc/c3/f068337a370801f372f2f8f6bad74a5c140f6fda3d9de154052708dd3c65/Jinja2-3.1.2-py3-none-any.whl",
        "archive_info": {
          "hash": "sha256=6088930bfe239f0e6710546ab9c19c9ef35e29792895fed6e6e31a023a182a61",
          "hashes": {
            "sha256": "6088930bfe239f0e6710546ab9c19c9ef35e29792895fed6e6e31a023a182a61"
          }
        }
      },
      "is_direct": false,
      "requested": false,
      "metadata": {
        "metadata_version": "2.1",
        "name": "Jinja2",
        "version": "3.1.2",
        "platform": [
          "UNKNOWN"
        ],
        "summary": "A very fast and expressive template engine.",
        "description_content_type": "text/x-rst",
        "home_page": "https://palletsprojects.com/p/jinja/",
        "author": "Armin Ronacher",
        "author_email": "armin.ronacher@active-4.com",
        "maintainer": "Pallets",
        "maintainer_email": "contact@palletsprojects.com",
        "license": "BSD-3-Clause",
        "classifier": [
          "Development Status :: 5 - Production/Stable",
          "Environment :: Web Environment",
          "Intended Audience :: Developers",
          "License :: OSI Approved :: BSD License",
          "Operating System :: OS Independent",
          "Programming Language :: Python",
          "Topic :: Internet :: WWW/HTTP :: Dynamic Content",
          "Topic :: Text Processing :: Markup :: HTML"
        ],
        "requires_dist": [
          "MarkupSafe (>=2.0)",
          "Babel (>=2.7) ; extra == 'i18n'"
        ],
        "requires_python": ">=3.7",
        "project_url": [
          "Donate, https://palletsprojects.com/donate",
          "Documentation, https://jinja.palletsprojects.com/",
          "Changes, https://jinja.palletsprojects.com/changes/",
          "Source Code, https://github.com/pallets/jinja/",
          "Issue Tracker, https://github.com/pallets/jinja/issues/",
          "Twitter, https://twitter.com/PalletsTeam",
          "Chat, https://discord.gg/pallets"
        ],
        "provides_extra": [
          "i18n"
        ],
        "description": ""
      }
    },
    {
      "download_info": {
        "url": "http://mirrors.aliyun.com/pypi/packages/f6/f8/9da63c1617ae2a1dec2fbf6412f3a0cfe9d4ce029eccbda6e1e4258ca45f/Werkzeug-2.2.3-py3-none-any.whl",
        "archive_info": {
          "hash": "sha256=56433961bc1f12533306c624f3be5e744389ac61d722175d543e1751285da612",
          "hashes": {
            "sha256": "56433961bc1f12533306c624f3be5e744389ac61d722175d543e1751285da612"
          }
        }
      },
      "is_direct": false,
      "requested": false,
      "metadata": {
        "metadata_version": "2.1",
        "name": "Werkzeug",
        "version": "2.2.3",
        "summary": "The comprehensive WSGI web application library.",
        "description_content_type": "text/x-rst",
        "home_page": "https://palletsprojects.com/p/werkzeug/",
        "author": "Armin Ronacher",
        "author_email": "armin.ronacher@active-4.com",
        "maintainer": "Pallets",
        "maintainer_email": "contact@palletsprojects.com",
        "license": "BSD-3-Clause",
        "classifier": [
          "Development Status :: 5 - Production/Stable",
          "Environment :: Web Environment",
          "Intended Audience :: Developers",
          "License :: OSI Approved :: BSD License",
          "Operating System :: OS Independent",
          "Programming Language :: Python",
          "Topic :: Internet :: WWW/HTTP :: Dynamic Content",
          "Topic :: Internet :: WWW/HTTP :: WSGI",
          "Topic :: Internet :: WWW/HTTP :: WSGI :: Application",
          "Topic :: Internet :: WWW/HTTP :: WSGI :: Middleware",
          "Topic :: Software Development :: Libraries :: Application Frameworks"
        ],
        "requires_dist": [
          "MarkupSafe (>=2.1.1)",
          "watchdog ; extra == 'watchdog'"
        ],
        "requires_python": ">=3.7",
        "project_url": [
          "Donate, https://palletsprojects.com/donate",
          "Documentation, https://werkzeug.palletsprojects.com/",
          "Changes, https://werkzeug.palletsprojects.com/changes/",
          "Source Code, https://github.com/pallets/werkzeug/",
          "Issue Tracker, https://github.com/pallets/werkzeug/issues/",
          "Twitter, https://twitter.com/PalletsTeam",
          "Chat, https://discord.gg/pallets"
        ],
        "provides_extra": [
          "watchdog"
        ],
        "description": ""
      }
    },
    {
      "download_info": {
        "url": "http://mirrors.aliyun.com/pypi/packages/7b/0f/0e99c2f342933c43af69849a6ba63f2eef54e14c6d0e10a26470fb6b40a9/MarkupSafe-2.1.2-cp37-cp37m-macosx_10_9_x86_64.whl",
        "archive_info": {
          "hash": "sha256=a6e40afa7f45939ca356f348c8e23048e02cb109ced1eb8420961b2f40fb373a",
          "hashes": {
            "sha256": "a6e40afa7f45939ca356f348c8e23048e02cb109ced1eb8420961b2f40fb373a"
          }
        }
      },
      "is_direct": false,
      "requested": false,
      "metadata": {
        "metadata_version": "2.1",
        "name": "MarkupSafe",
        "version": "2.1.2",
        "summary": "Safely add untrusted strings to HTML/XML markup.",
        "description_content_type": "text/x-rst",
        "home_page": "https://palletsprojects.com/p/markupsafe/",
        "author": "Armin Ronacher",
        "author_email": "armin.ronacher@active-4.com",
        "maintainer": "Pallets",
        "maintainer_email": "contact@palletsprojects.com",
        "license": "BSD-3-Clause",
        "classifier": [
          "Development Status :: 5 - Production/Stable",
          "Environment :: Web Environment",
          "Intended Audience :: Developers",
          "License :: OSI Approved :: BSD License",
          "Operating System :: OS Independent",
          "Programming Language :: Python",
          "Topic :: Internet :: WWW/HTTP :: Dynamic Content",
          "Topic :: Text Processing :: Markup :: HTML"
        ],
        "requires_python": ">=3.7",
        "project_url": [
          "Donate, https://palletsprojects.com/donate",
          "Documentation, https://markupsafe.palletsprojects.com/",
          "Changes, https://markupsafe.palletsprojects.com/changes/",
          "Source Code, https://github.com/pallets/markupsafe/",
          "Issue Tracker, https://github.com/pallets/markupsafe/issues/",
          "Twitter, https://twitter.com/PalletsTeam",
          "Chat, https://discord.gg/pallets"
        ],
        "description": ""
      }
    }
  ],
  "environment": {
    "implementation_name": "cpython",
    "implementation_version": "3.10.9",
    "os_name": "posix",
    "platform_machine": "x86_64",
    "platform_release": "22.4.0",
    "platform_system": "Darwin",
    "platform_version": "Darwin Kernel Version 22.4.0: Mon Mar  6 21:00:17 PST 2023; root:xnu-8796.101.5~3/RELEASE_X86_64",
    "python_full_version": "3.10.9",
    "platform_python_implementation": "CPython",
    "python_version": "3.10",
    "sys_platform": "darwin"
  }
}

I thought the output of the requirements has the right version.

mitsuhiko · 2023-05-02T08:30:48Z

I think the issue here might be in parts that --only-binary=:all: only picks up wheels and no longer looks at source distributions. On my machine for sure this does not resolve.

mitsuhiko · 2023-05-02T08:34:12Z

I think the fundamental issue this all runs into are these pip issues:

Various package-index filtering flags do not affect the environment markers pypa/pip#11664
Allow changing the entire dependency resolution “context” (Markers, dependency resolution and cross compiling) pypa/pip#10050 (closed, but not resolved)

This is a good minimal example that shows the issue:

$ rye init -p 3.11 issue-82
$ cd issue-82
$ rye pin cpython@3.11
$ rye add flask
$ rye sync

It will fail with

pip._internal.exceptions.UnsupportedPythonVersion: Package 'issue-82' requires a different Python: 3.10.9 not in '>=3.11'
Error: could not write production lockfile for project

Note that adding --python to --pip-args does not actually resolve this.

It looks like the only temporary workaround before switching of pip-tools might be to install them (temporarily) into the virtualenv :-/

CharlesChen0823 · 2023-05-02T09:36:43Z

only using pip directly, remove the dependency pip-tools is the same results?

mitsuhiko · 2023-05-02T09:42:44Z

The only version I found is to temporarily make pip available on the PYTHONPATH for the target virtualenv, something which likely comes with a lot of other challenges. Maybe that's a good enough workaround though I need to actually verify that the report generated by pip is a good replacement for what pip-compile does.

mitsuhiko · 2023-05-03T13:09:37Z

I think this is at least somewhat resolved for now, but longer term pip-tools should be replaced with something else.

CharlesChen0823 · 2023-05-03T13:27:19Z

I think that a tool without python version dependencies should exists, which is using for finding packages and resolve dependencies.

bb010g · 2023-05-11T06:37:27Z

It's probably worth considering what https://github.com/ddelange/pipgrip does. See also ddelange/pipgrip#109, ddelange/pipgrip#40, and the PEP 665 discussion (and its 2nd take, and the main successor workshop thread) for more nuance.

My personal desire is that until a successor to PEP 665 is accepted, a locking requirements.txt is portable both across platforms and Python versions, encoding all known resolutions.

T-256 · 2023-05-14T12:57:47Z

I think rye pin should prompt user if new pinning toolchain has version conflict with requires-python.

> rye init testrye
success: Initialized project in E:\testrye
  Run `rye sync` to get started

> rye pin cpython@3.7
pinned cpython@3.7.9 in E:\testrye\.python-version
new toolchain has conflict with project's required python ">=3.8" - auto fix it? (y/n)

jsirois · 2023-05-15T01:41:22Z

@mitsuhiko you found my Pip issue above in #82 (comment)

I did just see in the rye README that it currently only supports Linux & Mac. If that's still true you may find Pex useful, it handles the issues detailed in that bug and uses Pip under the covers. It currently also only supports Linux & Mac, although I have been working on Windows support. The locking and creation of venvs from locks support resides under the pex3 lock create and pex3 venv create commands, but pex3 lock export can also be used to get Pex out of the loop asap and get a hashed requirements file for use with Pip directly. The primary user of this tooling is Pants, but it's probably easiest to see use of this tooling here. I'm happy to provide more info if this seems of interest.

mitsuhiko · 2023-05-15T08:06:53Z

@jsirois Rye supports windows at this point, I corrected the Readme. I will have a look at the tool though.

mvaled · 2024-01-28T14:36:10Z

I can reproduce this issue with Python 3.8, in a newly created project:

$ rye self update
Updating to latest
Checking checksum
Updated!

rye 0.19.0
commit: 0.19.0 (676b1bffa 2024-01-21)
platform: linux (x86_64)
self-python: cpython@3.11
symlink support: true

$ rye init -p 3.8 rye-demo-82
success: Initialized project in /home/manu/src/self/rye-demo-82/.
  Run `rye sync` to get started

$ cd rye-demo-82
$ rye pin --relaxed 3.8
pinned 3.8 in /home/manu/src/self/rye-demo-82/.python-version

$ rye lock
Initializing new virtualenv in /home/manu/src/self/rye-demo-82/.venv
Python version: cpython@3.8.18
Generating production lockfile: /home/manu/src/self/rye-demo-82/requirements.lock
error: could not write production lockfile for project

Caused by:
    0: unable to run pip-compile
    1: No such file or directory (os error 2)

mitsuhiko · 2024-01-30T17:22:13Z

There is a bug where rye lock does not properly bootstrap. Run rye sync once instead. I will fix this, but that's an unrelated issue.

mvaled · 2024-02-02T13:44:23Z

Hi @mitsuhiko, After updating to latest rye, rye sync does work.

CharlesChen0823 · 2024-08-26T11:00:57Z

I notice, currenty rye had use uv, and pip-tools had removed, I think this issue can close when new version released.

mitsuhiko added the need-more-information Further information is requested label May 1, 2023

mitsuhiko added enhancement New feature or request bug Something isn't working and removed need-more-information Further information is requested enhancement New feature or request labels May 2, 2023

mitsuhiko mentioned this issue May 2, 2023

Sync fails with requires-python = ">= 3.11" #91

Closed

mitsuhiko changed the title ~~the method using pip-compile has some problem~~ pip-compile resolves against internal python version May 2, 2023

This was referenced May 2, 2023

Use pip install --report over pip-compile #92

Closed

Ugly hack to link pip-tools on demand into the venv #95

Merged

mitsuhiko mentioned this issue May 4, 2023

Sync with dependency lit=16.0.2 failed #104

Closed

T-256 mentioned this issue May 14, 2023

init command should warn about conflicted python versions #160

Closed

This was referenced May 18, 2023

Replace pip-tools with Monotrail or Similar #192

Closed

Emit Markers when Locking ddelange/pipgrip#112

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`pip-compile` resolves against internal python version #82

`pip-compile` resolves against internal python version #82

CharlesChen0823 commented May 1, 2023

mitsuhiko commented May 1, 2023

CharlesChen0823 commented May 1, 2023

CharlesChen0823 commented May 1, 2023

mitsuhiko commented May 1, 2023

CharlesChen0823 commented May 1, 2023 •

edited

Loading

mitsuhiko commented May 1, 2023

mitsuhiko commented May 1, 2023

cnpryer commented May 1, 2023

davegaeddert commented May 1, 2023

mitsuhiko commented May 2, 2023

CharlesChen0823 commented May 2, 2023 •

edited

Loading

mitsuhiko commented May 2, 2023

mitsuhiko commented May 2, 2023 •

edited

Loading

CharlesChen0823 commented May 2, 2023

mitsuhiko commented May 2, 2023

mitsuhiko commented May 3, 2023

CharlesChen0823 commented May 3, 2023

bb010g commented May 11, 2023 •

edited

Loading

T-256 commented May 14, 2023 •

edited

Loading

jsirois commented May 15, 2023 •

edited

Loading

mitsuhiko commented May 15, 2023

mvaled commented Jan 28, 2024 •

edited

Loading

mitsuhiko commented Jan 30, 2024

mvaled commented Feb 2, 2024

CharlesChen0823 commented Aug 26, 2024

pip-compile resolves against internal python version #82

pip-compile resolves against internal python version #82

Comments

CharlesChen0823 commented May 1, 2023

mitsuhiko commented May 1, 2023

CharlesChen0823 commented May 1, 2023

CharlesChen0823 commented May 1, 2023

mitsuhiko commented May 1, 2023

CharlesChen0823 commented May 1, 2023 • edited Loading

mitsuhiko commented May 1, 2023

mitsuhiko commented May 1, 2023

cnpryer commented May 1, 2023

davegaeddert commented May 1, 2023

mitsuhiko commented May 2, 2023

CharlesChen0823 commented May 2, 2023 • edited Loading

mitsuhiko commented May 2, 2023

mitsuhiko commented May 2, 2023 • edited Loading

CharlesChen0823 commented May 2, 2023

mitsuhiko commented May 2, 2023

mitsuhiko commented May 3, 2023

CharlesChen0823 commented May 3, 2023

bb010g commented May 11, 2023 • edited Loading

T-256 commented May 14, 2023 • edited Loading

jsirois commented May 15, 2023 • edited Loading

mitsuhiko commented May 15, 2023

mvaled commented Jan 28, 2024 • edited Loading

mitsuhiko commented Jan 30, 2024

mvaled commented Feb 2, 2024

CharlesChen0823 commented Aug 26, 2024

`pip-compile` resolves against internal python version #82

`pip-compile` resolves against internal python version #82

CharlesChen0823 commented May 1, 2023 •

edited

Loading

CharlesChen0823 commented May 2, 2023 •

edited

Loading

mitsuhiko commented May 2, 2023 •

edited

Loading

bb010g commented May 11, 2023 •

edited

Loading

T-256 commented May 14, 2023 •

edited

Loading

jsirois commented May 15, 2023 •

edited

Loading

mvaled commented Jan 28, 2024 •

edited

Loading