Tracking issue for manylinux2010 rollout

[ncoghlan]

I've accepted the manylinux2010 PEP, so that's now an active interoperability standard: https://www.python.org/dev/peps/pep-0571/

However, there are a number of further steps needed before folks can actually make use of that new baseline, and the order of operations matters (since building manylinux2010 wheels isn't very useful if installers won't install them):

Essential client support:

• Update pip 19.0 to install manylinux2010 wheels (PR: Manylinux2010 pip#5410)
• Update pip 9.x to install manylinux2010 wheels (needed to avoid the manylinux2010 rollout being delayed by the Python API break between pip 9.x and pip 10.x) (@ncoghlan: skip for now, wait and see if we get complaints about needing this)
• Update pipenv to install manylinux2010 wheels (needed due to pipenv vendoring pip, so the support depends on the version of pipenv being used, rather than the version of pip in the host or target environments)

Enable publication of manylinux2010 wheel archives:

• Update auditwheel to emit manylinux2010 wheel archives (PR: Manylinux2010 auditwheel#92)
• Confirm that twine will upload manylinux2010 wheel archives
• Update warehouse to accept manylinux2010 wheel uploads (Allow manylinux2010 wheel upload pypi/warehouse#3951)
• Provide a manylinux2010 build environment: https://quay.io/repository/pypa/manylinux2010_x86_64

Management of transition from manylinux1:

• Update https://packaging.python.org/specifications/platform-compatibility-tags/ to cover manylinux2010, with the caveat that it doesn't actually have broad tooling support yet
• Reword the "not yet widely supported" caveat from the platform compatibility tagging page to instead specify the minimum required versions of auditwheel to emit manylinux2010 packages, and pip and pipenv to install them (Update manylinux tags section packaging.python.org#656)
• Nominate a timeline and/or "percentage of PyPI downloads by manylinux2010 compatible installers" for switching the default build environment and auditwheel output to manylinux2010

Additional projects to consider once core capability support has rolled out:

• Option to target manylinux2010 instead of manylinux1 in https://github.com/joerick/cibuildwheel#readme
• Option to target manylinux2010 instead of manylinux1 in https://github.com/matthew-brett/multibuild#readme
• Distinguish manylinux2010 and manylinux1 images in https://github.com/dockcross/dockcross#readme
• ... any other suggestions?

Tagging @takluyver @markrwilliams as the PEP 571 authors, and @di and @brainwane from the PyPI/PSF side of things.

[markrwilliams]

See:

pypa/auditwheel#88
pypa/pip#5008

[takluyver]

Updating the spec page on PyPUG:

pypa/packaging.python.org#470

[mayeut]

Confirm that twine will upload manylinux2010 wheel archives

Confirmed

Update warehouse to accept manylinux2010 wheel uploads

PR: pypi/warehouse#3951

[di]

pypi/warehouse#3951 has been merged.

[wtolson]

pip merge request has been updated here: pypa/pip#5410

[wtolson]

auditwheel merge request has been updated here: pypa/auditwheel#92

[armudgal]

Hi,
First of all great job with this, I just wanted to know is there an estimate when the manylinux2010 will roll-out. Thanks :)

[takluyver]

There's no one roll-out date, because support needs to be added to various different tools in the ecosystem (see the list at the top of the issue). I imagine that it's probably a matter of a few weeks to a few months before you can use it and expect it to work.

[armudgal]

Oh, thanks for the information

[techalchemy]

@ncoghlan Pipenv will have pip 10 support after we merge pypa/pipenv#2255 which un-vendors pip9 completely, and bundles a patched version of pip10 which can be re-vendored and patched when updates land (this was possible because pip-tools accepted my PR which makes them pip10 compatible as well)

so whenever the pr lands in pip we can make the call about whether to actually go ahead and pull that into our patched version before it's released (technically we could do this now, even) and we will be set

[westurner]

"SEC: Spectre variant 2: GCC: -mindirect-branch=thunk -mindirect-branch-register"
https://mail.python.org/mm3/archives/list/distutils-sig@python.org/thread/4BGE226DB5EWIAT5VCJ75QD5ASOVJZCM/

[safijari]

Other than the open pull requests needing to get merged, is there any more left to do for this effort? How can I help?

[ehashman]

I have merged support for manylinux2010 in auditwheel. Will be cutting a release candidate today.

Note that auditwheel will default to assigning the strictest possible tag, hence it is possible for a binary built inside the manylinux2010 image to receive a manylinux1 tag given that the wheel's symbols match the stricter manylinux1 policy.

[di]

The last remaining item here is:

Nominate a timeline and/or "percentage of PyPI downloads by manylinux2010 compatible installers" for switching the default build environment and auditwheel output to manylinux2010

Given @ehashman's comment:

Note that auditwheel will default to assigning the strictest possible tag, hence it is possible for a binary built inside the manylinux2010 image to receive a manylinux1 tag given that the wheel's symbols match the stricter manylinux1 policy.

I think we actually want to retain the current behavior, and this item is invalid.

If no-one disagrees, I'm going to call this issue resolved and manylinux2010 officially "rolled out".

[ehashman]

I think I agree this is done, but for a slightly different reason...

Nominate a timeline ... for switching the default build environment ... to manylinux2010

I think this is already our recommendation due to the security issues with building in the manylinux1 docker environment. We don't really need to worry about a percentage of PyPI downloads for this item.

[zackw]

@di I agree that auditwheel's behavior of assigning the strictest possible tag is correct.

I hesitate to call manylinux2010 "rolled out" until we have some evidence that the people building wheels really are doing so in the manylinux2010 (or 2014) build environment. This is almost entirely a messaging issue: for instance, do the top five google, bing, duckduckgo, weibo, etc. search results still tell people to use the manylinux1 build environment? if so, we still have some work to do.

[JonasVautherin]

One thing that is not clear to me is whether I should move to manylinux2010 or not. Intuitively, it sounds like if my project builds with manylinux1, then it is "more compatible" than if it only built with manylinux2010. So anyway I still want to keep manylinux1 to support as many linux flavours as possible. Is that wrong?

[mayeut]

@JonasVautherin,

Intuitively, it sounds like if my project builds with manylinux1, then it is "more compatible" than if it only built with manylinux2010.

That's true, however

So anyway I still want to keep manylinux1 to support as many linux flavours as possible. Is that wrong?

It's probably wrong. I'd say most of linux flavors that came out before 2010 are now EOL meaning they do not receive any security updates. Time to move on. (manylinux2010 base OS, which is centos6 will reach end of life in November of 2020)

[mayeut]

A partial overview of manylinux2010 adoption through quay.io stats (yes, it's partial but can give an estimate which might not be too biased):

• https://quay.io/repository/pypa/manylinux2010_x86_64: very roughly, 300 pulls a day
• https://quay.io/repository/pypa/manylinux1_x86_64: very roughly, 1000 pulls a day

[snowman2]

Needing pip>=19.0.0 is causing some troubles in pyproj in this issue pyproj4/pyproj#477. Thoughts?

[di]

@snowman2 Unfortunately we can't go back in time and add manylinux2010 support to earlier pip versions. Can the project target manylinux1 instead? It's not clear to me from that issue what the problem is.

[snowman2]

Thanks for the information!

It's not clear to me from that issue what the problem is.

Mostly just relaying the issues so y'all are aware.

Can the project target manylinux1

Are there issues doing both?

[veblush]

Since gRPC Python started distributing both manylinux1 and manylinux2010, it might be useful to see download stats for gRPC 1.25 as of today:

• manylinux2010: 83%
• manylinux1: 17%

Definitely, manylinux2010 is a primary platform now for gRPC users. For manylinux1 case, it's not because their environment are not compliant with manylinux2010 but because their pip is older than 19.

[veblush]

Please update the bullet point for "Distinguish manylinux2010 and manylinux1 images in dockercross" since they have manylinux1, manylinux2010, and manylinux2014 now.

[di]

Please update the bullet point for "Distinguish manylinux2010 and manylinux1 images in dockercross" since they have manylinux1, manylinux2010, and manylinux2014 now.

Thanks! Done.

[ehashman]

Will this issue ever die? Surely we can call the manylinux2010 rollout complete, now.

[di]

I agree!

[jbarlow83]

...can we do this? Can we really close this? Shouldn't this albatross hang over us for all time???????? ... Great work everyone.

…

On Fri., Feb. 7, 2020, 16:36 Dustin Ingram, ***@***.***> wrote: Closed #179 <#179>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#179?email_source=notifications&email_token=AAN5YMYML6HNIPAROFNCEVDRBX5A5A5CNFSM4E2TRGTKYY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOWP5EICI#event-3019523081>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN5YM5WDRFGQWHBHUEZZPLRBX5A5ANCNFSM4E2TRGTA> .

[brainwane]

Thanks all!

Discussed some further documentation/message-spreading opportunities on Discourse.