Replace pip-tools with Monotrail or Similar

[mitsuhiko]

The use of pip-tools in rye is because this is what we use at @getsentry. It's however by no means perfect, among other things it does not have proper locking support and the resolver is very hacky. I would like to investigate alternatives. Options that might exist:

• Use/reuse pdm or it's internals
• monotrail-resolve: can resolve, does not lock.

Related issues that this would close:

• pip-compile resolves against internal python version #82
• Adding the same package with different environment markers only keeps last #200

[ddelange]

Hi 👋

Maybe you find this Makefile useful :) it uses pipgrip to to create (and later commit) a lockfile, and then uses that lockfile with pip's -c/--constraint flag in make install and in our Dockerfile. So the python package just has open version ranges, and the corresponding lockfile lives next to it.

.PHONY: lint
## Run linting
lint:
	pre-commit run --all-files

.PHONY: test
## Run tests
test:
	python -m pytest

.PHONY: install
## Install this repo, plus dev requirements, in editable mode
install:
	test -e requirements/pipgrip.lock || make lockfile
	pip install -r requirements/ci.txt -e . --constraint=requirements/pipgrip.lock
	pre-commit install

.PHONY: lockfile
## Create a lockfile for all python requirements.
lockfile:
	pip install -U pipgrip
	pipgrip -vv --sort --lock --tree .
	mv pipgrip.lock requirements/pipgrip.lock

.PHONY: help
## Print Makefile documentation
help:
	@perl -0 -nle 'printf("%-25s - %s\n", "$$2", "$$1") while m/^##\s*([^\r\n]+)\n^([\w-]+):[^=]/gm' $(MAKEFILE_LIST) | sort
.DEFAULT_GOAL := help

in Github Actions, we block PRs if they updated prod.txt (which is read into the package's install_requires), but they forgot to run make lockfile:

    - name: Check whether requirements/prod.txt changed
      id: requirements-changed
      uses: tj-actions/changed-files@v34
      with:
        files: |
          requirements/prod.txt

    - name: Check whether make lockfile was forgotten
      if: steps.requirements-changed.outputs.any_changed == 'true'
      env:
        PIP_INDEX_URL: ${{ secrets.PIP_INDEX_URL }}
      run: |
        make lockfile
        test ! "$(git diff)" && echo "no changes" || ( git diff && exit 1 )

[mitsuhiko]

That might be an interesting alternative to try.

[charbeljc]

Hi,

You also might look at rustine and pubgrub-pyo3 but this is a rather quick hack, nothing production ready :-)

Regards,
Charbel

[tekumara]

See also the prefix.dev resolver

[axel-kah]

See also the prefix.dev resolver

There's also a noteworthy blog post from the developers of Rattler Resolver which is a port of libsolv to Rust. They started with conda package support and they have set their eyes on PyPI packages support next.

[axel-kah]

There's also a noteworthy blog post from the developers of Rattler Resolver which is a port of libsolv to Rust. They started with conda package support and they have set their eyes on PyPI packages support next.

Seems like they already delivered: RIP (pip in Rust) 🦀

[GeorchW]

I think this can be closed now

[astrojuanlu]

xref #1185

[charliermarsh]

Yes sorry -- we now use uv here.