Unable to install constraint artifactory dependencies with nox: Conflicting dependencies, but equal versions

[LordFckHelmchen]

Dear @cjolowicz,

Thanks for the great work you put into the hypermodern Python part - this really inspired me! So the problem I have is hopefully something to improve the framework and not just a user layer error ;-)

Problem Description

• What did I do: Dot-install dependencies in a nox session of a project that uses a package from an company-internal artifactory.
• Expected behavior: Successful installation
• Observed behavior: A pip error about conflicting dependencies - where the versions are actually exactly matching:

  nox > Creating virtual environment (virtualenv) using python3.7 in .nox\test
  nox > poetry build --format=wheel
  nox > pip uninstall --yes file:///C:/Users/320009833/Git/nox_err_test/dist/nox_err_test-0.1.0-py3-none-any.whl
  nox > poetry export --format=requirements.txt --dev --without-hashes
  nox > python -m pip install '--constraint=.nox\test\tmp\requirements.txt' file:///PATH_TO/nox_err_test/dist/nox_err_test-0.1.0-py3-none-any.whl
  nox > Command python -m pip install '--constraint=.nox\test\tmp\requirements.txt' file:///PATH_TO/nox_err_test/dist/nox_err_test-0.1.0-py3-none-any.whl failed with exit code 1:
  Processing c:\PATH_TO\nox_err_test\dist\nox_err_test-0.1.0-py3-none-any.whl
  ERROR: Cannot install nox-err-test==0.1.0 because these package versions have conflicting dependencies.
  
  The conflict is caused by:
      nox-err-test 0.1.0 depends on ARTIFACTORY-PACKAGE==1.4.0
      The user requested (constraint) ARTIFACTORY-PACKAGE==1.4.0

Additional steps & observations

• Checked that the package has been successfully installed via poetry install and that it is present in the project-internal virtual environment (.venv/Lib/site-packages/)
• Manually re-executed the failed pip install in the virtual environment of
  • nox (.nox/test/Scripts/python.exe) -> Observed the same error
  • the project (.venv/Scripts/python.exe) -> Success and confirmed that the package is already installed:

    Requirement already satisfied: ARTIFACTORY-PACKAGE==1.4.0 in PATH_TO\nox_err_test\.venv\lib\site-packages (from nox-err-test==0.1.0) (1.4.0)

Any help would be greatly appreciated!

Details

Here is minimal working example to reproduce the error (still requiring an artifactory to download from).
Note: All internals have been replaced with Uppercase words for confidentially reasons.

System setup

OS	Windows 10, 64bit
Python (system)	3.10.2 used for poetry
Python (local)	3.7.1, installed with pyenw-win and used for project
pyenw-win	2.64.11
Poetry	1.1.13

Project layout

nox_err_test/
|-- .nox/
|-- .venv/               # The locally installed venv
|-- src/
|   |-- nox_err_test/    # The package to build - empty
|   |   |-- __init__.py
|-- noxfile.py
|-- poetry.lock
|-- pyproject.toml

Installation script

Follow these steps to setup the project

import subprocess

subprocess.run(["pyenv", "install", "3.7.1"], shell=True, check=True)
which_python = subprocess.run(["pyenv", "which", "python"], shell=True, check=True, stdout=subprocess.PIPE)

# Poetry config
subprocess.run(["poetry", "config", "virtualenvs.in-project", "true"], check=True)
subprocess.run(["poetry", "config", "repositories.ARTIFACTORY-PACKAGE", "https://URL_TO_PYPI_AT_COMPANY_ARTIFACTORY/simple/"])
subprocess.run(["poetry", "config", "http-basic.artifactory", "USER"])

# Setup & install
subprocess.run(["poetry", "env", "use", which_python.stdout.decode().strip()], check=True)
subprocess.run(["poetry", "install"], check=True)

noxfile

Simple executes an dot-install.

import nox
from nox_poetry import session, Session

package = "nox_err_test_test"

# Nox configuration
nox.options.error_on_external_run = True
nox.options.sessions = ("test",)


@session(python="3.7.1")
def test(nox_session: Session) -> None:
    nox_session.install(".")

pyproject.toml

[[tool.poetry.source]]
name = "artifactory"  # This name will be used in the configuration to retrieve the proper credentials
url = "https://URL_TO_PYPI_AT_COMPANY_ARTIFACTORY/simple/"  # URL used to download your packages from

[tool.poetry.dependencies]
python = "3.7.1"
ARTIFACTORY-PACKAGE = "1.4.0"

[tool.poetry.dev-dependencies]
nox = "2022.1.7"
nox-poetry = "0.9.0"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[cjolowicz]

Thanks for the detailed report!

Could this be related to #398 ? The repository config is currently not forwarded to pip. So you would need to explicitly configure pip, for example by creating a pip.ini file in the %APPDATA%\pip folder.

If that doesn't help, could you post the output of the poetry export command.

[LordFckHelmchen]

Hey @cjolowicz. It surely got us in the right direction. Since I'm using in-project venvs and don't want to fiddle around with the users pip config, I decided to try to manually add the argument to the session.install call. This works! E.g. replacing

@session(python="3.7.1")
def test(nox_session: Session) -> None:
    nox_session.install(".")

with

@session(python="3.7.1")
def test(nox_session: Session) -> None:
    nox_session.install(".", "--extra-index-url", "https://URL_TO_PYPI_AT_COMPANY_ARTIFACTORY/simple/")

I added a function my noxfile to automatically extract the artifactory URL from the according section in the pyproject.toml to reduce duplication. E.g.:

def get_extra_index_url(repository_name: str = "artifactory"):
    with open('pyproject.toml', 'r') as file:
        pyproject_toml = toml.load(file)
    return next((source.get("url") for source in pyproject_toml["tool"]["poetry"].get("source", []) if source["name"] == repository_name), "")

and then adding the url as above. Thanks a lot for the help. Would it make sense to make this a feature in nox-poetry? Or do you think users should add if required (then maybe a mention in the docs would make sense)?