Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

macOS apps aren't portable across x86_64/M1 unless all binary wheels are universal2 #1217

Closed
glyph opened this issue Apr 24, 2023 · 2 comments · Fixed by #1475
Closed

macOS apps aren't portable across x86_64/M1 unless all binary wheels are universal2 #1217

glyph opened this issue Apr 24, 2023 · 2 comments · Fixed by #1475
Labels
bug A crash or error in behavior. macOS The issue relates to Apple macOS support.

Comments

@glyph
Copy link

glyph commented Apr 24, 2023

What is the problem or limitation you are having?

Many Python libraries ship separate x86_64 and arm64 binary wheels for macOS. For example, see https://pypi.org/project/numpy/1.24.3/#files . Even if the project does ship universal2 wheels, pip won't install them because the platform tags are in the wrong order. You cannot even require pip to install universal2 wheels; it will prefer the more specific platform regardless of configuration.

Describe the solution you'd like

When building an application "for release" (not sure if that has a precise definition within Briefcase but really I mean "codesigned & notarized for public distribution"), briefcase should:

  • ensure that your interpreter is universal2 and error out with a clear error message explaining that distribution binaries should be built only with an appropriate interpreter. (I have a strong bias towards recommending python.org's builds for this purpose, but YMMV)
  • download all the official universal2 wheels it can find
  • discover all dependencies with separate x86_64 and arm64 wheels
  • use delocate-fuse to smash together the relevant wheel, synthesizing the platform tag.

Describe alternatives you've considered

Encrust has an incredibly janky solution to this problem that requires the building computer be an M1, because it just invokes pip install twice, once under arch -x86_64. This isn't generally usable because x86_64 CI (ahem, github actions) is still pretty popular.

See https://github.com/glyph/Encrust/blob/91b51faf299ad5967d1853a0651c27666107a156/src/encrust/_architecture.py

It uses this hack because discovering the proper set of platform tags to pass to pip download is kind of annoying and fiddly, since you can't just say universal2, you need to exactly call out base SDK versions as well. For example, pip download --only-binary :all: numpy --platform=macosx_10_12_x86_64 for x86_64, but pip download --only-binary :all: numpy --platform=macosx_11_0_arm64 for arm.

Additional context

Delocate is open to a PR that would fix the retagging issue on their end, so it is probably best to do that part upstream. matthew-brett/delocate#174

iOS has a similar problem. Given that Beeware is at the forefront of the ecosystem for iOS wheels and thus might have more influence there, perhaps we could establish some norms about building & uploading (i.e., do not support single-architecture uploads to PyPI at all, there would be no point)

@glyph glyph added the enhancement New features, or improvements to existing features. label Apr 24, 2023
@davidfokkema
Copy link
Contributor

It would be great to have a working solution in place. I had two GitHub actions running, one for Windows, one for Intel Mac which can also run in Rosetta. I wanted to switch to a native app but now I have to build two apps, one for Intel Mac and one for Apple Silicon Mac, which I currently do locally on two separate laptops 🫤.

@freakboy3742 freakboy3742 added macOS The issue relates to Apple macOS support. bug A crash or error in behavior. and removed enhancement New features, or improvements to existing features. labels Sep 14, 2023
@freakboy3742 freakboy3742 changed the title briefcase should enforce multi-architecture binary wheels on multi-architecture targets, so as not to ship partially-broken app bundles that won't launch on either Intel or Apple Silicon macOS apps aren't portable across x86_64/M1 unless all binary wheels are universal2 Sep 14, 2023
@freakboy3742
Copy link
Member

For those that are open to testing a draft PR, #1475 implements a fix for this. It works with Pillow, Numpy and Pandas in my testing; more data points would be helpful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A crash or error in behavior. macOS The issue relates to Apple macOS support.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants