Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix manylinux wheel builds #1099

Merged
merged 4 commits into from
Nov 15, 2024
Merged

Fix manylinux wheel builds #1099

merged 4 commits into from
Nov 15, 2024

Conversation

sklam
Copy link
Member

@sklam sklam commented Nov 13, 2024

llvmdev:

  • bump to llvm15
  • drop zstd due to conda package uses too new of cxxabi. If needed, we'll need to build it for manylinux ourselves.

docker image:

  • x86-64 uses manylinux2014 (which is glibc2.17)
  • aarch64 uses manylinux_2_28 since conda doesn't have glibc2.17 builds

misc:

  • renamed old directory that assumes manylinux_2_17

@sklam sklam marked this pull request as ready for review November 13, 2024 22:27
@sklam sklam added 3 - Ready for Review BuildFarm Passed For PRs that have been through the buildfarm and passed labels Nov 13, 2024
@sklam
Copy link
Member Author

sklam commented Nov 13, 2024

BFID llvmlite_wheel_70

@sklam sklam added this to the v0.44.0 milestone Nov 13, 2024
@sklam sklam mentioned this pull request Nov 13, 2024
25 tasks
Copy link
Member

@esc esc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've given this a first pass, there are numerous questions and suggestions,mostly related to code cleanliness, overall coherence and adherence to conventions. One they are resolved I will do a functional review and actually test the code to ensure it behaves as advertised.

buildscripts/manylinux/README.md Outdated Show resolved Hide resolved
buildscripts/manylinux/README.md Outdated Show resolved Hide resolved
buildscripts/manylinux/README.md Outdated Show resolved Hide resolved
#!/bin/bash
set -xe
cd $(dirname $0)
source ./prepare_miniconda.sh $1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is $1 supposed to be and does the script need to fail if $1 is empty?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added error checking in 7ba4e73

buildscripts/manylinux/build_llvmlite.sh Show resolved Hide resolved
conda-recipes/llvmdev_manylinux/meta.yaml Outdated Show resolved Hide resolved
conda-recipes/llvmdev_manylinux/meta.yaml Outdated Show resolved Hide resolved
conda-recipes/llvmdev_manylinux/meta.yaml Outdated Show resolved Hide resolved
conda-recipes/llvmdev_manylinux/meta.yaml Outdated Show resolved Hide resolved
conda-recipes/llvmdev_manylinux/meta.yaml Outdated Show resolved Hide resolved
@@ -0,0 +1,116 @@
#!/bin/bash

# based on https://github.com/AnacondaRecipes/llvmdev-feedstock/blob/master/recipe/build.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PS: Maybe include a link to the commit hash or version here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is a copy of ../llvmdev/build.sh with minor changes. I'll add file level comments to document this.

Main changes are improving error handling, documentation
@sklam
Copy link
Member Author

sklam commented Nov 14, 2024

Addressed all review comments. The scripts have room for improvements but we should just do them later as part of updating all the CI/release infrastructure.

@sklam
Copy link
Member Author

sklam commented Nov 14, 2024

Tested build farm. Manylinux llvmlite wheel still works

@sklam sklam requested a review from esc November 14, 2024 18:24
@esc
Copy link
Member

esc commented Nov 15, 2024

Addressed all review comments. The scripts have room for improvements but we should just do them later as part of updating all the CI/release infrastructure.

I looked at all the changes you made and am happy with the response.

Copy link
Member

@esc esc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spot checked this using docker on mac.

First I ran:

./buildscripts/manylinux/docker_run_x64.sh build_llvmdev.sh

This produced the llvmdev package locally.

Then I applied the patch:

diff --git i/buildscripts/manylinux/build_llvmlite.sh w/buildscripts/manylinux/build_llvmlite.sh
index 582b35c991..73f1c35313 100755
--- i/buildscripts/manylinux/build_llvmlite.sh
+++ w/buildscripts/manylinux/build_llvmlite.sh
@@ -21,9 +21,11 @@ conda activate $envname
 # Install llvmdev

 if [[ $(uname -m) == "aarch64" ]] ; then
-    conda install -y numba/label/manylinux_2_28::llvmdev --no-deps
+    #conda install -y numba/label/manylinux_2_28::llvmdev --no-deps
+    conda install -y -c file://$outputdir llvmdev --no-deps
 elif [[ $(uname -m) == "x86_64" ]] ; then
-    conda install -y numba/label/manylinux_2_17::llvmdev --no-deps
+    #conda install -y numba/label/manylinux_2_17::llvmdev --no-deps
+    conda install -y -c file://$outputdir llvmdev --no-deps
 else
     echo "Error: Unsupported architecture: $(uname -m)"
     exit 1

And then I ran:

./buildscripts/manylinux/docker_run_x64.sh build_llvmlite.sh cp313-cp313

To build and test llvmlite against a locally build llvmdev.

All tests passed! 🎉

This concludes my functional spot test and this PR now LGTM!

@kc611 kc611 merged commit e9f12c6 into numba:main Nov 15, 2024
25 checks passed
kc611 added a commit that referenced this pull request Nov 15, 2024
Backport #1099 to release branch: Fix manylinux wheel builds
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to merge BuildFarm Passed For PRs that have been through the buildfarm and passed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants