Release 24.3

[pradyunsg]

Filing this eagerly, because I figure it can't hurt. :)

[pradyunsg]

I won't have bandwidth to take on this release cycle and also don't wanna be hoarding on the release manager responsibilities. 😉

[pradyunsg]

And, FYI: pypa/get-pip#218 (comment)

[notatallshaw]

For anyone who is also a maintainer of resolvelib I'm waiting on a release (sarugaku/resolvelib#159) to fix several known incorrect ResolutionImpossible errors.

And while I'm asking for things that affect pip but require other projects with overlapping maintainers: pypa/packaging#794

[sbidoul]

I'm happy to take the RM hat for 24.3, unless anyone else wants to. I have very limited availability at the moment, so the release would be towards end October, mostly taking what is merged on main at that time.

[pfmoore]

If you'd prefer I can take it. My availability is also limited, but for me it's more unreliable, so it'll be a case of "you'll get the release when I have a free weekend". I'd also stick with just releasing what's on main - I'm happy to remind people of stuff on the 24.3 milestone, but I won't complete them myself.

Ping me if you'd rather I did it.

[kotborealis]

When will be 24.3 released? pip 24.2 still contains urllib3 1.26.18 which is affected by CVE-2024-37891 (GHSA-34jh-p97f-mpxf).

[sbidoul]

I'll have a look at what is in the 24.3 milestone soon.

Then I plan to release what is in main around October 20 or 27.

[pradyunsg]

@sbidoul I took the liberty of assigning this issue to you. 😅

[notatallshaw]

FYI, I've moved the issues which depended on a new resolvelib release and vendor to from milestone 24.3 to 25.0, as I reason here: sarugaku/resolvelib#159 (comment)

[notatallshaw]

If 24.3 is the last version to support Python 3.8 I think it would be a big help, for a small number of users, if truststore released with sethmlarson/truststore#157 and it was vendored by pip. Just on the intuition that old versions of MacOS might be using old versions of Python.

If at least pip 25.0 supports Python 3.8 I think it's less important.

[potiuk]

Comment from a side/user: agree with @notatallshaw. The worst that could happen is that users who are sticking with 3.8 will not be able to run 25.*. Which might incentivise them to upgrade Python.

I think it might be worthwile to add a warning - if someone uses < 25.* and they use Python 3.8, the "upgrade available" message could include the warning about using EOL Python and telling the user to migrate to higher version of Python as well. That would be a win-win for the ecosystem - encouraging people who are not even aware that Python 3.8 is already EOL.

I think it would also be great to agree general policy on which Python versions pip supports in general - and make it a "standard" approach for future migrations as well. This helped a lot in Airflow - we have now policies agreed on that and rather than discussing it, we jus follow the rule we agreed before (or if we want to change the rule, it's clear we need to start a discussion on it including justification for change).

[potiuk]

BTW. My proposal above means adding support / code to produce such message in the latest version of 24.*. of course.

[hugovk]

I think it would also be great to agree general policy on which Python versions pip supports in general - and make it a "standard" approach for future migrations as well. This helped a lot in Airflow - we have now policies agreed on that and rather than discussing it, we jus follow the rule we agreed before (or if we want to change the rule, it's clear we need to start a discussion on it including justification for change).

The policy is at https://pip.pypa.io/en/stable/development/release-process/#python-support-policy:

Python Support Policy

pip supports CPython versions that are not end-of-life. Older versions of CPython may be supported at the discretion of pip maintainers (based on criteria such as download statistics on PyPI, Python versions supported by the vendored dependencies and maintenance burden).

[sbidoul]

If 24.3 is the last version to support Python 3.8 I think it would be a big help, for a small number of users, if truststore released with sethmlarson/truststore#157 and it was vendored by pip. Just on the intuition that old versions of MacOS might be using old versions of Python.

If at least pip 25.0 supports Python 3.8 I think it's less important.

@notatallshaw I'm not comfortable with including an unmerged trusttore PR (assuming our vendoring policy would allow it). Can you post that comment on #12989 so we keep track of that there?

[notatallshaw]

I'm not comfortable with including an unmerged trusttore PR (assuming our vendoring policy would allow it). Can you post that comment on #12989 so we keep track of that there?

100% agreed, I didn't mean to imply that, more that if truststore is able to release with that PR merged it would be a great help and would make it easier to consider dropping support for Python 3.8 in 25.0. I did mention it in #12989 (comment) but I'll add something that's more explicit.

[notatallshaw]

Actually, I withdraw my comments about truststore, I got confused and thought it was doing openssl 1.1.1+ detection, but I see it's actually doing Python version detection: https://github.com/sethmlarson/truststore/blob/v0.9.2/src/truststore/__init__.py#L5

So any vendor of truststore is irrelevant to Python 3.8 and 3.9. Sorry for the noise.

[pradyunsg]

assuming our vendoring policy would allow it

It does not.

[WilliamRoyNelson]

Is the plan still to release what is in main around October 27th? I know CVE-2024-37891 (which will be fixed by the updated urllib3 package in main) doesn't seem like an incredibly serious problem, but because pip is so ubiquitous, it's probably a good idea to cross that off the list.

[notatallshaw]

Is the plan still to release what is in main around October 27th? I know CVE-2024-37891 (which will be fixed by the updated urllib3 package in main) doesn't seem like an incredibly serious problem, but because pip is so ubiquitous, it's probably a good idea to cross that off the list.

Yes, @sbidoul will decide exactly when to release, but pip 24.3 will be released soon: https://pip.pypa.io/en/stable/development/release-process/#release-cadence

It will be a cut of main which is currently on urllib3 1.26.20 (https://github.com/pypa/pip/blob/main/src/pip/_vendor/vendor.txt), which includes a fix for this CVE: https://github.com/urllib3/urllib3/blob/1.26.x/CHANGES.rst

[hugovk]

It would be nice to include the vendored truststore update (issue: #12901, PR: #13041), so it can be included in an ensurepip update for the next CPython releases:

• 2024-11-19: 3.14.0a2
• 2024-12-03: 3.12.8
• 2024-12-??: 3.13.1

Edit: I now see it's already been added to the 24.3 milestone 👍

Thanks!

[sbidoul]

Yep, will do. It is already in the 24.3 milestone.

[sbidoul]

• prepare Release/24.3 #13044
• publish to PyPI
• push tag
• update get-pip Update to 24.3 get-pip#224
• release announcement https://discuss.python.org/t/announcement-pip-24-3-release/69350
• CPython PR

[potiuk]

Would it be possible to post here when a candidate/beta is ready for testing ?

I will be happy to run our extensive checks on 24.3 in Airflow - while most of our CI runs with uv for speed, we can switch it to pip and run complete set of tests including eager upgrade and a number of other cases (and with Airflow's complexity it stretches pip a bit) - just to help with testing.

It does not have to be released in pypi - if there is a branch or tag that is "ready for testing", we can modify our CI to install from GitHub directly rather than from released package.

[sbidoul]

@potiuk we won't make a beta for 24.3, but you can test with the main branch, as it will become 24.3 later this week end.

[potiuk]

@potiuk we won't make a beta for 24.3, but you can test with the main branch, as it will become 24.3 later this week end.

Will do and report it.

[potiuk]

Running here: apache/airflow#43395 -> I will likely have to do small modifications, but eventually I will turn it into one-switch-testing for the future.

[potiuk]

Update: looks good so far :)

[sbidoul]

Heads-up: there was a little issue with the get-pip CI in pypa/get-pip#224 which I resolved with pypa/get-pip@90f1b11 in order to not block the release process.

[sbidoul]

All done. I'll do the CPython PR in a few days.

[xmatthias]

@sbidoul I'll take the liberty to point out #13046 (not sure if you're monitoring for all issues in this repo) - which is an issue that popped up about an hour ago - and is caused by the new pip release.

[sbidoul]

And 24.3.1 is out.

[notatallshaw]

Might be even faster than uv fixes and releases, very impressive @sbidoul !!

[potiuk]

Very nice and impressive, indeed !

[notatallshaw]

Are we considering 24.3 release done now? Seems that other than recursive requirements we've not heard much noise about this one.

[sbidoul]

I'm waiting for a packaging release (@pradyunsg gentle nudge) and may do a 24.3.2 with that, and then I'll do the CPython PR.

So if main could be kept quiet for a little more time, that would be nice.

[notatallshaw]

I would suggest against that, a packaging release is going to introduce behavior changes, e.g. pypa/packaging#794

I understand there is this issue with how packaging was patched and users who devendored pip, but these are by definition very informed users, I am worried that less informed users will be caught out by behavior changes between 24.3.1 and 24.3.2.

As I read #13053 (comment) it's not that a new packaging release needed to be vendored into pip, just that a new packaging needed to be released and the user could vendor it into their pip build.

[pradyunsg]

https://github.com/pypa/packaging/releases/tag/24.2 is done.

[sbidoul]

Cool. Thanks a lot @pradyunsg!

Now that is indeed a lot of changes, and way too much to vendor in a patch release.

So I see 3 options here:

1/ Do nothing, and consider 24.3 done with 24.3.1, despite the fact that it vendors a patched packaging. Debundlers can install packaging 24.2.
2/ Revert PEP 730 iOS wheels support in 24.3.x only
3/ Add a protection against devendored packaging<24.2 (#13057) in 24.3.x only

The lazy me leans towards (1) but I can do (2) or (3) if there are arguments in favor of those.

cc/ @hugovk @mgorny

[hugovk]

I'm fine with (1) as well and would prefer to avoid (2).

[pradyunsg]

Hmm... lemme see if I can cut a release of packaging as 24.2.1, with just the iOS support added in, so that we can vendor that in.

[notatallshaw]

lemme see if I can cut a release of packaging as 24.2.1, with just the iOS support added in,

Do you mean 24.1.1? It would be weird for 24.2.1 to have less features than 24.2

[pradyunsg]

Yea, that. Sorry. 😅