Plan for 2.0

[henryiii]

Now that the main style merge is in, and manylinux is about to drop Python 3.5, I think we should look at the 2.0 work. This is what I'm thinking (maintainers please edit the list directly):

Version 1.x:

• Create branch 1.x
• Pin auditwheel < 4 for automatic updates (ref)
• Enable CI on 1.x (I have done GHA only for now, Azure, Circle and Gitlab picked it up anyway).
• Add v1 branch to the docs and allow tests to run when merging to v1. Maybe automatic updates on this branch too, but probably better to just do it manually if needed.

Version 2.0.0a2:

• Drop Python 3.5
• Drop Python 2.7 (unpin manylinux 2010)
• Add links to the v1 docs for "how to build with Python 2" or something like that.
• Perhaps --pre support for pre-releases and Python 3.10 Add CPython 3.10 support #675
• Drop warnings for old specifiers (deprecated in 1.x) feat: update deprecated option check #700
• Produce a warning if "?p27" is detected in SKIP, and an error if "?p27" is detected in BUILD? Likewise with 3.5. feat: update deprecated option check #700

Version 2.0.0 final:

• Add "build" as an optional build method feat: support pypa build #521
• Support configuration from pyproject.toml feat: configuration support #684
• Discuss CIBW_MANYLINUX_PYPY_* options as mentioned in Update PyPy to 7.3.5 and use PyPA's manylinux images with PyPy #671 (comment) and Update PyPy to 7.3.5 and use PyPA's manylinux images with PyPy #671 (comment) - manylinux-pypy settings? #711 - no change

Some future release:

• Add nicer Limited API support Error on creation of wheel with a name that already exists, and add Py_LIMITED_API to the docs' FAQ #569
• Perhaps musllinux musllinux support (PEP 656) #627 support... (currently needed: auditwheel support)

Although we developers love features like "drop X", those are not very exciting new features for users, so pyproject.toml support could be a nice shiny new feature to get users to upgrade. Better Limited API support is nice, but affects very few users. Musllinux would be nice, but don't think that auditwheel / images will be ready in time - probably will be 2.x down the line. Python 3.10 beta is nice, but not really supposed to be used by anyone besides testing yet, so not so exciting. build is also interesting from a back-end, but not likely to be that exciting as a reason to upgrade. And PyPy 64-bit Windows & any manylinux PyPy is also nice, honestly that might be the second most exciting thing in 2.0 for users.

[joerick]

Nice one. Regarding the 1.x branch, I think let's not bother with the autoupdate stuff there. It feels like a YAGNI situation. We can add it back if we really need it later.

Other than that, looks good to me!

[joerick]

Depending on how things go upstream, musllinux #627 would be a cool 2.0 feature also :)

[henryiii]

I'd love that, I'm a fan of Apline. :)

I think we are ready to make the v1 branch and set it up in RtD?

For the 2.0 branch, should we produce a warning if "p27" is detected in SKIP, saying it's no longer needed, and an error if "p27" is detected in BUILD? Likewise with 35, and future dropped versions. Just a thought, haven't looked into it.

[henryiii]

Also we should add Python 3.10 beta 1, pypa/manylinux#1077 :)

PS: Is it ABI stable in Beta or just RC? Maybe we could add a --dev flag that would enable pre-release Pythons?

[joerick]

For the 2.0 branch, should we produce a warning if "p27" is detected in SKIP, saying it's no longer needed, and an error if "p27" is detected in BUILD? Likewise with 35, and future dropped versions. Just a thought, haven't looked into it.

Good idea 👍

[joerick]

Is it ABI stable in Beta or just RC?

No info in PEP 619 on that, but last time we waited for rc1 before considering it ABI stable. I can't find a reference on that, unfortunately.

Maybe we could add a --dev flag that would enable pre-release Pythons?

This would be pretty cool.

[joerick]

I ticked a few jobs off that list above @henryiii. I wasn't sure of the reason/need for the auditwheel version specifier, so I left that one. Feel free to hack on 1.x if you like!

[mayeut]

Is it ABI stable in Beta or just RC?

No info in PEP 619 on that, but last time we waited for rc1 before considering it ABI stable. I can't find a reference on that, unfortunately.

I knew it was somewhere in beta for python 3.9 but had a hard time finding where this piece of information was.
Finally I found this:
https://www.python.org/downloads/release/python-3100b1/

Our goal is have no ABI changes after beta 4 and as few code changes as possible after 3.10.0rc1, the first release candidate. To achieve that, it will be extremely important to get as much exposure for 3.10 as possible during the beta phase.

[henryiii]

I think before 2.0 might be a good time to update our vendored getpip copy? Actually just before 3.10 would likely be best, once we are sure pip will work with 3.10.

[joerick]

I think before 2.0 might be a good time to update our vendored getpip copy?

I'm kinda of the opinion 'if it ain't broke don't fix it', but one thing we could try (again) is using ensurepip rather than getpip, since the former is built-in to Python, and we probably have a better chance at making that work now that we've dropped a few Python versions...

EDIT: trying this in #665...

[mayeut]

but one thing we could try (again) is using ensurepip rather than getpip

It's certainly worth a try.

[mayeut]

Pin auditwheel < 4 for automatic updates

This would in fact translate as "Pin all manylinux images":

• manylinux1, before or equal to 2021-05-05-b64d921
• PyPy, I guess the current one should be pinned.
• other images, before or equal to 2021-05-05-1ac6ef3 (manylinux2010 is already pinned before that)

[joerick]

Pin auditwheel < 4 for automatic updates

This would in fact translate as "Pin all manylinux images":

I'm thinking that rather than pinning the auto-update, we just don't enable auto-updates on 1.x, or run the scripts. We can still update the pins there manually, if it's ever needed.

[mayeut]

Maybe we could add a --dev flag that would enable pre-release Pythons?

This would be pretty cool.

Could we use --pre instead to get some consistency with pip ?

Install Options:
  ...
  --pre Include pre-release and development versions. By default, pip only finds stable versions.

[henryiii]

Sure, not glued to any name and consistency is good. :)

[henryiii]

While we add the checks for mentioning p27 and such, what about dropping the check for the old deprecated specifiers? I think no one is on a version that supported that now?

[joerick]

what about dropping the check for the old deprecated specifiers

sure, sounds good! A 2.0 is a good time for that kind of clean up.

[joerick]

I think we're closing in on this release! I don't think we need the limited_api support in v2.0.0, we can punt that down the road I reckon. I don't see anything in the PR queue that would block the release either.

I'm thinking we aim for a v2.0.0 release next week - Monday 19 July? Obviously that can move if we discover something core that needs fixing, but I think we're more-or-less there.

[henryiii]

Sounds good! Agree on the limited_api; it's a rather nice feature to have eventually, but it's mostly a convenience / testing feature, and has a very small number of users interested in it. It's fine to add later. It also might interact with the wheel renaming I'd like to work on soon(ish) - some packages (like cmake & ninja, or ctypes projects) produce non-Python dependent binaries and it would potentially be nice to include support for those while we are at it - that's a much larger number of users.

Just in case it affects your release plans, on Friday I'll be giving a talk at SciPy on pybind11, and I'm planning on including a slide each on cibuildwheel and scikit-build. Don't know if it's better to say "2.0 coming soon" or "2.0 just released today" ;).

[joerick]

Great :)

Don't know if it's better to say "2.0 coming soon" or "2.0 just released today" ;).

Oh, nice! I think we could pull the release forward to Friday :)

[joerick]

Now that this is released, I think we can close :)