Support newer pip versions.

[benjyw]

Plumbs --pip-version through to pex commands that support it.

Newer pips have perf improvements that may benefit users, in particular for very long-running resolves.

Does not change the default. We can do that via deprecation in a followup.

[benjyw]

This is potentially controversial, as we're also upgrading everyone to newer pip, rather than making it opt-in.

I think that's fine, since pip is very heavily tested, and ubiquitous, and we want to bias towards doing the right thing for new users who will not be expecting to be going back to some ancient pip.

We have an escape hatch for those who encounter an unexpected and unlikely problem in an upgrade.

[jsirois]

LGTM. I don't feel good about the world shift on existing users but it is probably mitigated by lockfile use allowing some degree of choice about when to deal with any "breaks" new resolve outputs may cause. We definitely have an ongoing tension where we fight our own deprecation rules with some regularity it seems to me.

[jsirois]

There probably need not be verbiage about this, but the newer Pips plumbed here will cause Pex to issue a warning and automatically fall back to vendored Pip if the Python being used / specified with --interpreter-constraint falls before 3.7 (These newer Pips all require >=3.7). As such, you can imagine a user that gets different resolve results in different contexts and perhaps becomes confused by that. I'm not sure this is very likely or even much outside of what someone might expect, but FYI in case that was not a known property of the --pip-version support.

[Eric-Arellano]

Thanks! Thoughts on backporting to 2.15, but not changing the default to reduce the amount of API change? That gives an escape hatch for people with long resolves.

[benjyw]

Hmm, I don't think this justifies backporting to 2.15, it's a feature not a bugfix.

I'm OK with being more conservative, on second thoughts that makes sense. We can deprecate use of the older pip, now or later, the main thing is to give people with bad resolve perf something to try.

[benjyw]

And re the warning you mentioned @jsirois : I had originally thought to check the ICs on the Pants side and only set this flag if >=3.7, for that reason, but then figured that the user won't see a warning, and the fallback is in line with what they can reasonably expect. Or even better than what they can reasonably expect, since the actual expectation might be to error on the incompatibility.

[jsirois]

Aha, great. Yeah, having them see the warning is probably what's wanted. That does make me question what happens, though, when Pants executes a Process that succeeds but spews stderr. Does that stderr automatically just go to the terminal? I think it does, since we used to get black emoji anyhow, but I'm not positive at all on this.

[benjyw]

We return Black's stdout/stderr in a LintResult and then emit them manually in src/python/pants/core/goals/lint.py . Users won't see pex's warning unless we take pains to show them.

[jsirois]

Aha. That's unfortunate.

[martimlobao]

Hmm, I don't think this justifies backporting to 2.15, it's a feature not a bugfix.

I'm OK with being more conservative, on second thoughts that makes sense. We can deprecate use of the older pip, now or later, the main thing is to give people with bad resolve perf something to try.

fwiw, we're considering leapfrogging from 2.12 straight to 2.16 just so we can address some issues we've been seeing with performance 😄

[benjyw]

fwiw, we're considering leapfrogging from 2.12 straight to 2.16 just so we can address some issues we've been seeing with performance 😄

I'd still recommend upgrading one version at a time, in a branch, so you get all the deprecation warnings. You can just merge the final 2.16 result into main of course.