Flag to ignore constraint for installing .

[thenewguy]

What's the problem this feature will solve?
Global constraint file that is shared among many related apps where the constraints point to git repos this will allow installing pip install . to work for the tox testing environment.

Currently you get an error like: ERROR: Could not satisfy constraints for 'app-foo': installation from path or url cannot be constrained to a version

Describe the solution you'd like
pip install --ignore-constraint=app-foo .

Or any other way to suppress this error for installing "."

An example where this is handy is requirements.txt like this:

--index-url https://pypi.python.org/simple/
--constraint https://example.com/raw/master/constraints.txt
.

Where tox installs the project like:

deps =
    -r{toxinidir}/requirements.txt

[thenewguy]

Right now, I've written a script that templates my constraint file as a workaround. I use a tox wrapper to run the script before dependencies are installed. Pretty clunky.

This issue is important for being able to develop sub-apps independently. My specific problem arose from django migrations working when testing one app but the integration failing because a related app was constrained to an older version of django-taggit and the expected migrations did not exist.

import re
from os import getenv
from os.path import join, abspath, dirname
from urllib.request import urlopen

PIP_CERT = getenv('PIP_CERT')
REQUESTS_CA_BUNDLE = getenv('REQUESTS_CA_BUNDLE')

url = 'https://path/to/raw/master/constraints.txt'
cafile = PIP_CERT or REQUESTS_CA_BUNDLE

setup_path = abspath(join(dirname(__file__), 'setup.py'))
print('Looking for setup.py at', setup_path)

with open(setup_path, 'r') as fp:
    data = fp.read()
    pattern = r'''name\s*=\s*['"]([a-zA-Z0-9_-]+)['"]'''
    match = re.search(pattern, data)
    if not match:
        raise ValueError('name not found in setup.py')
    name = match.group(1)

print("Building constraints for: '%s'" % name)

with urlopen(url, cafile=cafile) as response:
   constraints = response.read().decode('utf-8')

print("Using constraints template:\n%s" % constraints)

with open('constraints-readonly.txt', 'w') as fp:
    for constraint in constraints.splitlines():
        if name not in constraint:
            fp.write(constraint)
            fp.write('\n')
        else:
            print("Skipped constraint containing '%s': %s" % (name, constraint))

with open('constraints-readonly.txt') as fp:
    print("Output constraints:\n%s" % fp.read())

Here is an example of the tox wrapper incase this is helpful for anyone:

wget -O /vagrant/constraints.py -q https://path/to/raw/master/constraints.py
python3 /vagrant/constraints.py

tox -c /vagrant/tox.ini -vv "$@"

rm /vagrant/constraints.py /vagrant/constraints-readonly.txt

[froli-code]

i think it is not a supported use-case to use unversioned constraints in the constraint-file. this has been discussed multiple times (#8210 or #6310) and usually the suggested workaround is to place the dependency in the requirements-file

this is also what the current error message in pip 22.3 suggests:

DEPRECATION: Constraints are only allowed to take the form of a package name and a version specifier. Other forms were originally permitted as an accident of the implementation, but were undocumented. The new implementation of the resolver no longer supports these forms. A possible replacement is replacing the constraint with a requirement. Discussion can be found at https://github.com/pypa/pip/issues/8210
ERROR: Unnamed requirements are not allowed as constraints

from your description i do not quite understand your exact use case, maybe you could provide some more information why this constraint is required? :)

[thenewguy]

This is from a long time ago and is not fresh on my mind.

Consider your constraint file is the following:

app-a
app-b
app-c
... lots of other apps ...

This constraint file is shared among app-a, app-b, app-c, and lots of other apps so writing one for each app presents the challenge of keeping all of the constraint files in sync (i.e. not possible to maintain).

Now consider you are developing a new feature in app-c. app-c requires app-a and app-b and lots of other apps of specific versions so you want to install these via the constraint file.

Your test runner installs "." (the local working dir). Since app-c is listed in the constraints file it fails on the local version.

[froli-code]

thank you for clarifying, it makes sense to me now.

As I understand this issue does only apply for the packages which you are developing by yourself. Would it be a possibility to allow newer versions of those packages in the constraints file?

e.g.

#constraint file
example_package>=1.0.0

this should allow the package to be installed.

[pfmoore]

I still don't see the logic here. A constraint of app-a says "if you ask for app-a to be installed, it must satisfy the constraint app-a". Or to put it another way, app-a must be app-a. That seems nonsensical to me, as it's a constraint that doesn't actually constrain anything...

Just omitting app-a from the constraints file will have an identical effect.

[froli-code]

@pfmoore , true, but as I understand the author is using a constraint file on organization-level (or similar) in order to enforce certain versions organization-wide. if app-a is omitted from the constraints file, he would need to create a specific constraint-file for each app, as app-b would need a constraints-file with app-a, but without app-b and so on

[pfmoore]

I fail to see why. Having app-a in the constraints file won't result in app-a being installed - constraints are solely to restrict the list of versions visible to the resolver, and a constraint with no version specifier doesn't limit anything (and therefore is useless, and can be omitted).

What do you think putting app-a in a constraints file does?

[thenewguy]

@pfmoore , true, but as I understand the author is using a constraint file on organization-level (or similar) in order to enforce certain versions organization-wide. if app-a is omitted from the constraints file, he would need to create a specific constraint-file for each app, as app-b would need a constraints-file with app-a, but without app-b and so on

This, exactly.

[thenewguy]

The constraint lists a particular constraint. Not just the app name. Sorry if the way I presented made that detail confusing.

And then it prevents installing "." for the test runner

[notatallshaw]

Rather than adding yet another pip option with a new file standard or syntax pattern I think a far more viable solution here in the face of any specific standards it to change this from an error to a warning.

I think it makes sense when a requirement is found that can not have it's version checked but a constraint exists against it emit a warning to the user that this requirement can not be constrained and proceed as though that constraint doesn't exist.

This will inform users who for some reason think it could work, but allow users who are using constraints in complex scenarios like this (such as managing multiple application development environments) at least the ability to keep using the constraints file.

That said I think user built templating solutions are a reasonable solution to this problem. Yes it requires a small script then to build a dev environment, rather than one pip install, but pip can't solve all problems related to creating dev environments anyway so some users will always need to make themselves a script to reduce it to one command.

[stephenfin]

I still don't see the logic here. A constraint of app-a says "if you ask for app-a to be installed, it must satisfy the constraint app-a". Or to put it another way, app-a must be app-a. That seems nonsensical to me, as it's a constraint that doesn't actually constrain anything...

Just omitting app-a from the constraints file will have an identical effect.

@pfmoore I documented this further in a duplicate issue, #11718, but to give a concrete example, OpenStack uses constraints files to provide upper constraints across all libraries including the ones we develop ourselves. When working on e.g. https://github.com/openstack/oslo.db, you would want to constrain all the dependencies of oslo.db but not oslo.db itself as constraining oslo.db itself means you can't install it. A flag to say either "ignore the constraint if installing from a local repo/in editable mode" or "ignore the constraint for the package with this name" would be ideal. The alternative is to have a separate constraints file for every single library you want to develop, or to install all dependencies of a package separately with constraints before installing the package itself without constraints.

[sbidoul]

@stephenfin imagine you have a mechanism that says to pip: whenever you meet a dependency on oslo.db, replace all version specifiers for that package (whether they are provided by dependencies or constraints) by --editable ./oslo.db.

Would that solve your problem?

I think such a mechanism has a deeper reach than just ignoring constraints because:

• it helps if you have conflicting constraints that you don't control
• it also helps if the conflict is due to a dependency that you don't control
• it helps with Relaxing / Ignoring constraints during dependency resolution #8076

[Hu1buerger]

@sbidoul i agree it might have a deeper result. But the current situation of pips refusal an overwrite of a dependency constraint dosnt imo fit the lax nature of python.

I would suggest a warning like use at your own risks which are ...
But i would like to use this feature asap.

[nouiz]

We also need this. Hopefully it can advance soon.
Thanks for working on this!

[stephenfin]

@stephenfin imagine you have a mechanism that says to pip: whenever you meet a dependency on oslo.db, replace all version specifiers for that package (whether they are provided by dependencies or constraints) by --editable ./oslo.db.

Would that solve your problem?

I think such a mechanism has a deeper reach than just ignoring constraints because:

* it helps if you have conflicting constraints that you don't control

* it also helps if the conflict is due to a dependency that you don't control

* it helps with [Relaxing / Ignoring constraints during dependency resolution #8076](https://github.com/pypa/pip/issues/8076)

This sounds very similar to go's replace directive. That isn't a bad idea, but it is a far bigger change that I suspect would require a PEP seeing as requirements.txt is a defacto standard. Not sure I'm willing to put in that work if so.

[gforcada]

The @plone community would benefit a lot from it as well, exactly like the Openstack community, we are also trying to use a constraints.txt with our +200 packages+dependencies. When working on any of them we want all but the development package to be constrained.

I see that the @stephenfin commit on his pip fork is small, yet a major enabler of our use case, is there any reason a PR was not opened so far? 🤞🏾

[stephenfin]

I actually do have a PR open, #11723. It just hasn't received much attention, unfortunately.

[notatallshaw]

Rather than adding yet another pip option with a new file standard or syntax pattern I think a far more viable solution here in the face of any specific standards it to change this from an error to a warning.

Would Pip maintainers be interested in a PR that does this based on my reasoning in the comment: #7839 (comment)

I would be happy to raise a PR if #11723 isn't accepted and no one else wants to.

[jaklan]

Are there any updates related to that issue? AWS enforces usage of constraints in MWAA:

Beginning with Apache Airflow v2.7.2, your requirements file must include a --constraint statement. If you do not provide a constraint, Amazon MWAA will specify one for you to ensure the packages listed in your requirements are compatible with the version of Apache Airflow you are using.

Due to that, we have a huge problem to install our custom libraries, which need different versions of some dependencies (although we know they won't break anything).