Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Add aarch64 wheels to build workflow #13938

Closed
MichaIng opened this issue Sep 28, 2022 · 3 comments · Fixed by #14212
Closed

Add aarch64 wheels to build workflow #13938

MichaIng opened this issue Sep 28, 2022 · 3 comments · Fixed by #14212
Labels
A-Arm64 Running on an ARM64 CPU, including Apple's M1 and M2 A-Packaging Our Debian packages, docker images; or issues relevant to downstream packagers O-Uncommon Most users are unlikely to come across this or unexpected workflow S-Tolerable Minor significance, cosmetic issues, low or no impact to users. T-Enhancement New features, changes in functionality, improvements in performance, or user-facing enhancements.

Comments

@MichaIng
Copy link
Contributor

Description:

Would it be feasible to add aarch64 wheels to the build workflow to be uploaded to PyPI? ARM play an increasing role among servers and pre-compiled wheels save users a lot of compile time and build dependencies, since Rust is required now with v1.68.0.

@DMRobertson DMRobertson added A-Packaging Our Debian packages, docker images; or issues relevant to downstream packagers S-Tolerable Minor significance, cosmetic issues, low or no impact to users. T-Enhancement New features, changes in functionality, improvements in performance, or user-facing enhancements. O-Uncommon Most users are unlikely to come across this or unexpected workflow A-Arm64 Running on an ARM64 CPU, including Apple's M1 and M2 labels Sep 29, 2022
@DMRobertson
Copy link
Contributor

IIRC we produce aarch64 docker images, so I don't see why we wouldn't also produce wheels for that platform.

cibuildwheel apparently supports aarch64, judging by the table at https://cibuildwheel.readthedocs.io/en/stable/#what-does-it-do

I wonder why manylinux musllinux aarch64 doesn't show up in the binaries we upload. Tests are being skipped here due to *musl* but I don't think that should prevent builds?

- name: Build wheels
run: python -m cibuildwheel --output-dir wheelhouse
env:
# Skip testing for platforms which various libraries don't have wheels
# for, and so need extra build deps.
CIBW_TEST_SKIP: pp39-* *i686* *musl* pp37-macosx*

@erikjohnston any ideas here?

@DMRobertson
Copy link
Contributor

I guess cibuildwheel only builds what it can on a given platform. Would probably have to add a matrix entry to

build-wheels:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-20.04, macos-10.15]
is_pr:
- ${{ startsWith(github.ref, 'refs/pull/') }}
exclude:
# Don't build macos wheels on PR CI.
- is_pr: true
os: "macos-10.15"

and a step below that conditionally runs the rest under QEMU, like here:

- name: Set up QEMU
id: qemu
uses: docker/setup-qemu-action@v1
with:
platforms: arm64

@MichaIng
Copy link
Contributor Author

Finally: #14212

That was a hassle with rust-lang/cargo#10583.

  • First I tried to pass CARGO_NET_GIT_FETCH_WITH_CLI it via CIBW_ENVIRONMENT and was wondering why suddenly within the CIBW container the Rust compiler couldn't be found, event that Rust was definitely installed.
  • In attempt to fix this, I tried to set CIBW_BEFORE_ALL_LINUX and wondered why since then really no Rust was installed anymore.
  • Finally I recognised that these both variables, if not set, are internally set to install Rust (via CIBW_BEFORE_ALL) and to add Rust to PATH (via CIBW_ENVIRONMENT). Setting any of both overrides the default completely, there is no way to only append to the default (AFAIK).
  • So the only way to pass CARGO_NET_GIT_FETCH_WITH_CLI without breaking or needing to manually add Rust related steps and variables, is CIBW_ENVIRONMENT_PASS_LINUX, which has no internal default.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A-Arm64 Running on an ARM64 CPU, including Apple's M1 and M2 A-Packaging Our Debian packages, docker images; or issues relevant to downstream packagers O-Uncommon Most users are unlikely to come across this or unexpected workflow S-Tolerable Minor significance, cosmetic issues, low or no impact to users. T-Enhancement New features, changes in functionality, improvements in performance, or user-facing enhancements.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants