python3Packages.llvmlite: make it statically link to llvm #335416
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wrvsrx
force-pushed
the
fix-llvmlite-link
branch
from
f8d730e to
86c7f54
Compare
|
I think we should make this an option, and cleanup the outdated patching. --- a/pkgs/development/python-modules/llvmlite/default.nix
+++ b/pkgs/development/python-modules/llvmlite/default.nix
@@ -5,13 +5,11 @@
buildPythonPackage,
isPyPy,
pythonAtLeast,
-
- # build-system
llvm,
+ libxml2,
setuptools,
-
- # tests
python,
+ withStaticLLVM ? true,
}:
buildPythonPackage rec {
@@ -28,22 +26,16 @@ buildPythonPackage rec {
hash = "sha256-5QBSRDb28Bui9IOhGofj+c7Rk7J5fNv5nPksEPY/O5o=";
};
- nativeBuildInputs = [
- llvm
- setuptools
- ];
+ build-system = [ setuptools ];
- # Disable static linking
- # https://github.com/numba/llvmlite/issues/93
- postPatch = ''
- substituteInPlace ffi/Makefile.linux --replace "-static-libstdc++" ""
+ buildInputs = [ llvm ] ++ lib.optionals withStaticLLVM [ libxml2.dev ];
- substituteInPlace llvmlite/tests/test_binding.py --replace "test_linux" "nope"
- '';
+ env.LLVM_CONFIG = "${llvm.dev}/bin/llvm-config";
- # Set directory containing llvm-config binary
- preConfigure = ''
- export LLVM_CONFIG=${llvm.dev}/bin/llvm-config
+ postPatch = lib.optionalString withStaticLLVM ''
+ substituteInPlace ffi/build.py \
+ --replace-fail 'llvm_config, "--system-libs --libs all"' \
+ 'llvm_config, "--system-libs --libs all --link-static"'
'';
checkPhase = ''
The output size is 2.3M when dynamic linking, but 110M when static linking. |
wrvsrx
force-pushed
the
fix-llvmlite-link
branch
from
86c7f54 to
3c4c667
Compare
|
@NickCao Done. |
wrvsrx
force-pushed
the
fix-llvmlite-link
branch
from
3c4c667 to
39c414b
Compare
|
Does successfully build for me on macos 12 x86_64 & no new reverse dependencies fail (with it enabled) |
NickCao
reviewed
Aug 19, 2024
wrvsrx
force-pushed
the
fix-llvmlite-link
branch
from
39c414b to
1f2a9fb
Compare
|
This pull request has been mentioned on NixOS Discourse. There might be relevant details there: https://discourse.nixos.org/t/prs-ready-for-review/3032/4468 |
wrvsrx
force-pushed
the
fix-llvmlite-link
branch
from
1f2a9fb to
33c3507
Compare
ofborg
bot
added
the
2.status: merge conflict
|
Please resolve the merge conflict. |
wrvsrx
force-pushed
the
fix-llvmlite-link
branch
from
33c3507 to
c9bd743
Compare
ofborg
bot
removed
the
2.status: merge conflict
Llvmlite doc says that they link to llvm statically, but it's not. Check - numba/llvmlite#1027. - https://github.com/numba/llvmlite/blob/ca123c3ae2a6f7db865661ae509862277ec5d692/docs/source/faqs.rst#why-static-linking-to-llvm This behavior causes problems when python uses another version of llvm, for example, mesa llvmpipe. This PR force llvmlite statically link to llvm. Besides, this PR also use llvm as buildInputs rather than nativeBuildInputs since it's a library that llvmlite use rather than a part of build system. This PR also cleans outdated patches.
wrvsrx
force-pushed
the
fix-llvmlite-link
branch
from
c9bd743 to
62fae87
Compare
|
@NickCao Done. |
NickCao
approved these changes
Sep 7, 2024
There was a problem hiding this comment.
Result of nixpkgs-review pr 335416 run on x86_64-linux 1
10 packages marked as broken and skipped:
- khoj
- khoj.dist
- piper-train
- piper-train.dist
- python311Packages.bambi
- python311Packages.bambi.dist
- python311Packages.clifford
- python311Packages.clifford.dist
- python312Packages.clifford
- python312Packages.clifford.dist
12 packages failed to build:
- python311Packages.ffcv
- python311Packages.ffcv.dist
- python311Packages.vllm
- python311Packages.vllm.dist
- python311Packages.ydata-profiling
- python311Packages.ydata-profiling.dist
- python312Packages.gumath
- python312Packages.gumath.dist
- python312Packages.vllm
- python312Packages.vllm.dist
- python312Packages.ydata-profiling
- python312Packages.ydata-profiling.dist
233 packages built:
- backgroundremover
- backgroundremover.dist
- molbar (python312Packages.molbar)
- molbar.dist (python312Packages.molbar.dist)
- openai-whisper (python312Packages.openai-whisper)
- openai-whisper.dist (python312Packages.openai-whisper.dist)
- pianotrans
- pianotrans.dist
- pitivi
- python311Packages.aeppl
- python311Packages.aeppl.dist
- python311Packages.aesara
- python311Packages.aesara.dist
- python311Packages.apricot-select
- python311Packages.apricot-select.dist
- python311Packages.array-api-compat
- python311Packages.array-api-compat.dist
- python311Packages.arviz
- python311Packages.arviz.dist
- python311Packages.asteroid-filterbanks
- python311Packages.asteroid-filterbanks.dist
- python311Packages.awkward
- python311Packages.awkward.dist
- python311Packages.coffea
- python311Packages.coffea.dist
- python311Packages.corner
- python311Packages.corner.dist
- python311Packages.correctionlib
- python311Packages.correctionlib.dist
- python311Packages.cppe
- python311Packages.cppe.dist
- python311Packages.dask-awkward
- python311Packages.dask-awkward.dist
- python311Packages.dask-glm
- python311Packages.dask-glm.dist
- python311Packages.dask-ml
- python311Packages.dask-ml.dist
- python311Packages.datashader
- python311Packages.datashader.dist
- python311Packages.dscribe
- python311Packages.dscribe.dist
- python311Packages.flowmc
- python311Packages.flowmc.dist
- python311Packages.galois
- python311Packages.galois.dist
- python311Packages.graphrag
- python311Packages.graphrag.dist
- python311Packages.graspologic
- python311Packages.graspologic.dist
- python311Packages.gumath
- python311Packages.gumath.dist
- python311Packages.hyppo
- python311Packages.hyppo.dist
- python311Packages.librosa
- python311Packages.librosa.dist
- python311Packages.llvmlite
- python311Packages.llvmlite.dist
- python311Packages.miasm
- python311Packages.miasm.dist
- python311Packages.molbar
- python311Packages.molbar.dist
- python311Packages.mplhep
- python311Packages.mplhep.dist
- python311Packages.numba
- python311Packages.numba-scipy
- python311Packages.numba-scipy.dist
- python311Packages.numba.dist
- python311Packages.numbaWithCuda
- python311Packages.numbaWithCuda.dist
- python311Packages.numpy-groupies
- python311Packages.numpy-groupies.dist
- python311Packages.openai-whisper
- python311Packages.openai-whisper.dist
- python311Packages.outlines
- python311Packages.outlines.dist
- python311Packages.piano-transcription-inference
- python311Packages.piano-transcription-inference.dist
- python311Packages.pyannote-audio
- python311Packages.pyannote-audio.dist
- python311Packages.pygmo
- python311Packages.pymatting
- python311Packages.pymatting.dist
- python311Packages.pymc
- python311Packages.pymc.dist
- python311Packages.pymoo
- python311Packages.pymoo.dist
- python311Packages.pynndescent
- python311Packages.pynndescent.dist
- python311Packages.pyscf
- python311Packages.pyscf.dist
- python311Packages.pytensor
- python311Packages.pytensor.dist
- python311Packages.quaternion
- python311Packages.quaternion.dist
- python311Packages.resampy
- python311Packages.resampy.dist
- python311Packages.scikit-tda
- python311Packages.scikit-tda.dist
- python311Packages.shap
- python311Packages.shap.dist
- python311Packages.simple-dftd3
- python311Packages.simple-dftd3.dist
- python311Packages.sparse
- python311Packages.sparse.dist
- python311Packages.stumpy
- python311Packages.stumpy.dist
- python311Packages.stytra
- python311Packages.stytra.dist
- python311Packages.timezonefinder
- python311Packages.timezonefinder.dist
- python311Packages.tinygrad
- python311Packages.tinygrad.dist
- python311Packages.torch-audiomentations
- python311Packages.torch-audiomentations.dist
- python311Packages.torchcrepe
- python311Packages.torchcrepe.dist
- python311Packages.torchlibrosa
- python311Packages.torchlibrosa.dist
- python311Packages.trackpy
- python311Packages.trackpy.dist
- python311Packages.tsfresh
- python311Packages.tsfresh.dist
- python311Packages.umap-learn
- python311Packages.umap-learn.dist
- python311Packages.uproot
- python311Packages.uproot.dist
- python311Packages.vector
- python311Packages.vector.dist
- python311Packages.xarray-einstats
- python311Packages.xarray-einstats.dist
- python312Packages.apricot-select
- python312Packages.apricot-select.dist
- python312Packages.array-api-compat
- python312Packages.array-api-compat.dist
- python312Packages.arviz
- python312Packages.arviz.dist
- python312Packages.asteroid-filterbanks
- python312Packages.asteroid-filterbanks.dist
- python312Packages.awkward
- python312Packages.awkward.dist
- python312Packages.coffea
- python312Packages.coffea.dist
- python312Packages.corner
- python312Packages.corner.dist
- python312Packages.correctionlib
- python312Packages.correctionlib.dist
- python312Packages.cppe
- python312Packages.cppe.dist
- python312Packages.dask-awkward
- python312Packages.dask-awkward.dist
- python312Packages.dask-glm
- python312Packages.dask-glm.dist
- python312Packages.dask-ml
- python312Packages.dask-ml.dist
- python312Packages.dscribe
- python312Packages.dscribe.dist
- python312Packages.galois
- python312Packages.galois.dist
- python312Packages.graphrag
- python312Packages.graphrag.dist
- python312Packages.graspologic
- python312Packages.graspologic.dist
- python312Packages.hyppo
- python312Packages.hyppo.dist
- python312Packages.librosa
- python312Packages.librosa.dist
- python312Packages.llvmlite
- python312Packages.llvmlite.dist
- python312Packages.miasm
- python312Packages.miasm.dist
- python312Packages.mplhep
- python312Packages.mplhep.dist
- python312Packages.numba
- python312Packages.numba-scipy
- python312Packages.numba-scipy.dist
- python312Packages.numba.dist
- python312Packages.numbaWithCuda
- python312Packages.numbaWithCuda.dist
- python312Packages.numpy-groupies
- python312Packages.numpy-groupies.dist
- python312Packages.outlines
- python312Packages.outlines.dist
- python312Packages.piano-transcription-inference
- python312Packages.piano-transcription-inference.dist
- python312Packages.pyannote-audio
- python312Packages.pyannote-audio.dist
- python312Packages.pygmo
- python312Packages.pymatting
- python312Packages.pymatting.dist
- python312Packages.pymoo
- python312Packages.pymoo.dist
- python312Packages.pynndescent
- python312Packages.pynndescent.dist
- python312Packages.pyscf
- python312Packages.pyscf.dist
- python312Packages.quaternion
- python312Packages.quaternion.dist
- python312Packages.resampy
- python312Packages.resampy.dist
- python312Packages.scikit-tda
- python312Packages.scikit-tda.dist
- python312Packages.shap
- python312Packages.shap.dist
- python312Packages.simple-dftd3
- python312Packages.simple-dftd3.dist
- python312Packages.sparse
- python312Packages.sparse.dist
- python312Packages.stumpy
- python312Packages.stumpy.dist
- python312Packages.stytra
- python312Packages.stytra.dist
- python312Packages.timezonefinder
- python312Packages.timezonefinder.dist
- python312Packages.tinygrad
- python312Packages.tinygrad.dist
- python312Packages.torch-audiomentations
- python312Packages.torch-audiomentations.dist
- python312Packages.torchcrepe
- python312Packages.torchcrepe.dist
- python312Packages.torchlibrosa
- python312Packages.torchlibrosa.dist
- python312Packages.trackpy
- python312Packages.trackpy.dist
- python312Packages.umap-learn
- python312Packages.umap-learn.dist
- python312Packages.uproot
- python312Packages.uproot.dist
- python312Packages.vector
- python312Packages.vector.dist
- python312Packages.xarray-einstats
- python312Packages.xarray-einstats.dist
- whisper-ctranslate2
- whisper-ctranslate2.dist
All failures already present on master.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of changes
Llvmlite doc says that they link to llvm statically, but it's not. Check
This behavior causes problems when python uses another version of llvm, for example, mesa llvmpipe. This PR force llvmlite statically link to llvm.
Besides, this PR also use llvm as buildInputs rather than nativeBuildInputs since it's a library that llvmlite use rather than a part of build system.
Things done
nix.conf? (See Nix manual)sandbox = relaxedsandbox = truenix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage./result/bin/)Add a 👍 reaction to pull requests you find important.