Recently built macOS amd64 packages fail to get installed on mac MI

[dvarrazzo]

Hello,

I am facing a problem that I cannot solve myself with psycopg packages.

Since a series of CI updates (especially cibuildwheel 2.17.0 -> 2.18.1 and the introduction of MACOSX_DEPLOYMENT_TARGET=14.0 to make delocate happy, but maybe just having moved this packages' build to github, whereas previously they were built on a Scaleway machine), users are reporting failure to install packages on Apple M1. See psycopg/psycopg#858

I have no experience with the MacOS deployment target and have no idea about why psycopg_binary-3.1.19-cp311-cp311-macosx_11_0_arm64.whl from 3.1.19 was installable but psycopg_binary-3.2.1-cp311-cp311-macosx_14_0_arm64.whl from 3.2.1 isn't.

What is the problem here? What makes macosx_14 packages not as usable as macosx_11? System updates on the receiver's laptop? Is there a way to produce more usable packages? I don't even know if the packages don't work for just a number of people or for everyone.

Help is appreciated. Thank you very much 🙂

[Czaki]

macos_11 means Big Sur, macos_14 means Sonoma. You cannot install a package compiled against the more modern macOS version that you use.

So user problem comes from using not latest macOS version, when you bump the minimum required version to Sonoma.
I do not know which files produced you warning by delocate happy, but to pass this check all dependencies need to be built with MACOSX_DEPLOYMENT_TARGET at most as targeted system for wheels.

[mayeut]

It seems some dependencies are coming from homebrew.
As mentioned in https://cibuildwheel.pypa.io/en/stable/faq/#missing-dependencies / https://cibuildwheel.pypa.io/en/stable/faq/#macos-library-dependencies-do-not-satisfy-target-macos, those homebrew dependencies are bumping the minimum macOS target you can achieve.
If the scalaway machine was also using homebrew dependencies, then it's likely it was running an older version of macOS and the move to macOS 14 on GHA means you're not compatible with older OS anymore.

You'll need to rebuild dependencies from sources or find another source than homebrew for binaries if you want to achieve compatibility with OS older than macOS-14 (arm64) / macOS-12 (x86_64).

[dvarrazzo]

Forcing a build with MACOSX_DEPLOYMENT_TARGET=11.0 results in:

  delocate.libsana.DelocationError: Library dependencies do not satisfy target MacOS version 11.0:
  /private/var/folders/h9/l1shxhdd69nct08ylzq0n6q00000gn/T/tmp1ki89fxh/wheel/psycopg_binary/.dylibs/libkrb5support.1.1.dylib has a minimum target of 14.0
  /private/var/folders/h9/l1shxhdd69nct08ylzq0n6q00000gn/T/tmp1ki89fxh/wheel/psycopg_binary/.dylibs/libintl.8.dylib has a minimum target of 14.0
  /private/var/folders/h9/l1shxhdd69nct08ylzq0n6q00000gn/T/tmp1ki89fxh/wheel/psycopg_binary/.dylibs/libcrypto.3.dylib has a minimum target of 14.0
  /private/var/folders/h9/l1shxhdd69nct08ylzq0n6q00000gn/T/tmp1ki89fxh/wheel/psycopg_binary/.dylibs/libpq.5.dylib has a minimum target of 14.0
  /private/var/folders/h9/l1shxhdd69nct08ylzq0n6q00000gn/T/tmp1ki89fxh/wheel/psycopg_binary/.dylibs/libkrb5.3.3.dylib has a minimum target of 14.0
  /private/var/folders/h9/l1shxhdd69nct08ylzq0n6q00000gn/T/tmp1ki89fxh/wheel/psycopg_binary/.dylibs/libcom_err.3.0.dylib has a minimum target of 14.0
  /private/var/folders/h9/l1shxhdd69nct08ylzq0n6q00000gn/T/tmp1ki89fxh/wheel/psycopg_binary/.dylibs/libgssapi_krb5.2.2.dylib has a minimum target of 14.0
  /private/var/folders/h9/l1shxhdd69nct08ylzq0n6q00000gn/T/tmp1ki89fxh/wheel/psycopg_binary/.dylibs/libk5crypto.3.1.dylib has a minimum target of 14.0
  /private/var/folders/h9/l1shxhdd69nct08ylzq0n6q00000gn/T/tmp1ki89fxh/wheel/psycopg_binary/.dylibs/libssl.3.dylib has a minimum target of 14.0

Does it mean, in practice, that it's impossible to build macos arm64 wheels on Github infrastructure that can be used on MacOS 11, 12, 13 (unless building all the above libs from source)?

[dvarrazzo]

Looks like using scaleway wouldn't be better:

$ ./tools/build/scaleway_m1.sh ensure
Creating new server.
{
  "id": "...",
  "type": "M1-M",
  "name": "mac-m1-psycopg",
  ...
  "os": {
    "id": "e08d1e5d-b4b9-402a-9f9a-97732d17e374",
    "name": "macos-sonoma-14",
    "label": "macOS Sonoma 14",
    "image_url": "https://scw-apple-silicon.s3.fr-par.scw.cloud/scw-console/os/macos-sonoma.png",
    "family": "Sonoma",
    "is_beta": false,
    "xcode_version": "15",
    "version": "14.4",
    "compatible_server_types": [
      "M1-M",
      "M2-M",
      "M2-L"
    ]
  },
  "status": "starting",
  "created_at": "2024-07-08T20:05:30.211634Z",
  "updated_at": "2024-07-08T20:05:30.211634Z",
  "deletable_at": "2024-07-09T20:05:30.211634Z",
  "zone": "fr-par-3"
}

So scaleway can't be used to produce macosx_11_0_arm64 either.

[Czaki]

Does it mean, in practice, that it's impossible to build macos arm64 wheels on Github infrastructure that can be used on MacOS 11, 12, 13 (unless building all the above libs from source)?

It is possible, but you need to compile them manually, not install from brew.