Lack of wheel in distribution causes it to break builds targeting separate platforms

[cmancone]

Granted, some of this may be specific to my process, but the lack of a wheel in your distributions is rather unusual, so I figured I'd raise an issue for this.

I work heavily in Lambdas but my build machines are typically ubuntu (e.g.: public build servers for Github actions). As a result, when packaging my applications, I have to tell pip to target a separate build. This is a normal use case for pip, and I've never had trouble with this before using jwcrypto. To reproduce this issue, dump the following line into a file called requirements.txt:

jwcrypto==1.5.0 ; python_version >= "3.8" and python_version < "4.0"

And then ask pip to build a package for this requirements.txt file (this specifically targets the platform used by AWS Lambda):

pip3 install --platform manylinux2014_x86_64 -r requirements.txt -t ./package --implementation cp --only-binary=:all:

You'll be met with the following error message:

ERROR: Could not find a version that satisfies the requirement jwcrypto==1.5.0 (from versions: 0.3.0, 0.3.1, 0.4.0, 0.4.1, 0.4.2, 0.5.0, 0.6.0, 0.7, 0.8, 0.9, 0.9.1, 1.0)
ERROR: No matching distribution found for jwcrypto==1.5.0

Which was very confusing for me for quite a while, until I went and actually looked at the files in your distributions in pypi. You can see that 1.5.0 only has a .tar.gz file:

https://pypi.org/project/jwcrypto/1.5.0/#files

While 1.0 (the last version that pip recognizes) has a wheel file:

https://pypi.org/project/jwcrypto/1.0/#files

So, it seems like there was a point in time where you followed the standard build process for your package, but it looks like that changed after 1.0. Presumably you ditched the wheel because your package is pure python, so it's not technically necessary - that is, until it is :)

[simo5]

I change how pypi publishing happens in this commit d83b316 from a manual process to an automatic one on release.

I am not sure what broke and I am not a pypi expert, so if you know how to fix it to make it better for you I will access patches here.

Note though that jwcrypto is pure python and fully depends on python cryptography which releases wheels, so I think it may make more sense to figure out if you can pull a cryptography wheel and just lay jwcrypto on top.

[cmancone]

So the root issue here is the fact that you're using sdist rather than (something like) bdist. See this article:

https://dev.to/icncsx/python-packaging-sdist-vs-bdist-5ekb

The main question I would have for you is, "how would you feel about switching to a more comprehensive build/packaging system?". Namely, poetry.

[simo5]

Quite frankly I do not care what system is used, so anything that works is theoretically fine.

However:

• I do not want to have to deal with new dependencies if avoidable, and I certainly do not want to have to become an expert in a new build system to be able to use it.
• I do not want things too young that require a lot of churn (see above) or that may break existing users.

Fundamentally a build/packaging system that doesn't get out of the way and requires constant care (which includes becoming rapidly obsoleting and requiring replacement) is not a system I care for.

[achamayou]

I don't think switching build systems is necessary for this, there is a suitable setuptools extension. As far as I can tell, the following steps work:

1. pip install wheel
2. python setup.py bdist_wheel

That creates a wheel under dist:

$ ls dist/
jwcrypto-1.5.0-py3-none-any.whl

[schinnaswamy]

Hi,

We are using AWS serverless platform and using Serverless-python-requirments plugin to create a python-requirments layer build from ManyLinux_2014_aarch64 platform and bundle all the python project dependencies (binary wheel distribution) as a layer and use the layer while running serverless AWS Lambda.

JWCrypto is one of dependent library and missing binary wheel distributions and the deployment is currently failing due to below PYPI error

Its same issue reported by @cmancone

Stderr: ERROR: Could not find a version that satisfies the requirement jwcrypto==1.5.4 (from versions: 0.3.0, 0.3.1, 0.4.0, 0.4.1, 0.4.2, 0.5.0, 0.6.0, 0.7, 0.8, 0.9, 0.9.1, 1.0)
ERROR: No matching distribution found for jwcrypto==1.5.4

Could you release the binary distribution as well as a part of your recent release 1.5.4

[simo5]

the only binaries that would make sense to distribute would be the ones already compiled by pyca/cryptography, are you perhaps missing a require there?
I am not sure it would make sense for jwcrypto to release a custom binary wheel.

[simo5]

Ok so this change when I added auto-publishing in 1,0
I can try to make a new version of the github action so that it creates a binary wheel, but I am not very experienced with wheels, so YMMV

[cmancone]

So on my end I fixed this by changing my build command from only binary to prefer binary. Here's my working build command (used to deploy to AWS Lambda):

pip3 install --platform manylinux2014_x86_64 -r requirements.txt -t ./package --implementation cp --prefer-binary --no-deps

[simo5]

Auto-publishing is a double-edged sword as I can't test that the change I made in #347 will actually work as desired on the first try ...
So hopefully 1.5.5 will do what's needed correctly, but it may cause another churn over versions, if something goes wrong ...
cross your fingers ...

[simo5]

And first attempt failed ...

[simo5]

Fixed in https://github.com/latchset/jwcrypto/releases/tag/v1.5.5
Wheel available in https://pypi.org/project/jwcrypto/1.5.5/

[schinnaswamy]

Great. Thanks @simo5 for fulfilling the request.