Support building against llvm15 #1035

yashssh · 2024-03-13T17:11:20Z

Add support for llvmlite to be built against llvm15 while keeping support for llvm14 as well.

1) Drop 2 passes no longer supported by old PM
2) Update domTree passes to their wrapper passes
3) Update getAddress() api
4) Disable opaque pointer mode
5) Give warning when build with llvm15

All the changes are put inside llvm version check guard so that it builds with both llvm14 and llvm15

1) Drop 2 passes no longer supported by old PM 2) Update domTree passes to their wrapper passes 3) Update getAddress() api 4) Disable opaque pointer mode 5) Give warning when build with llvm15 All the changes are put inside llvm version check guard so that it builds with both llvm14 and llvm15

yashssh · 2024-03-13T17:13:46Z

Removing draft tag as it won't allow me to add reviewers. Adding "draft" to the title instead.

Edit: Turns out I can't add reviewers. @gmarkall please add yourself and other reviewers who might be interested in this.

ffi/Makefile.linux

gmarkall · 2024-03-13T21:05:38Z

ffi/core.cpp

+LLVMPY_GetGlobalContext() {
+  auto context = LLVMGetGlobalContext();
+#if LLVM_VERSION_MAJOR > 14
+  LLVMContextSetOpaquePointers(context, false);


Note to other reviewers - this does result in opaque pointers being disabled for the global context multiple times, but this operation is idempotent.

gmarkall · 2024-03-13T21:06:10Z

ffi/core.h

@@ -2,6 +2,11 @@
 #define LLVMPY_CORE_H_

 #include "llvm-c/Core.h"
+
+// Needed for macros that control version-specific behaviour - included here so
+// that they are available in all ffi translation units


In particular, this defines the LLVM_VERSION_MAJOR macro referenced in various places in this PR.

gmarkall · 2024-03-13T21:08:11Z

llvmlite/tests/test_binding.py

@@ -653,7 +654,7 @@ def break_up_asm(self, asm):
    def test_rv32d_ilp32(self):
        self.check_riscv_target()
        llmod = self.fpadd_ll_module()
-        target = self.riscv_target_machine(features="+f,+d")
+        target = self.riscv_target_machine(features="+f,+d", abiname="ilp32")


LLVM 14 defaults to the soft-float ABI, LLVM 15 defaults to the hard-float ABI. We explicitly specify the soft-float ABI for consistency with the expected output. Note that hard float and hard double ABIs are tested in the subsequent two tests below.

llvmlite/tests/test_binding.py

gmarkall · 2024-03-13T21:09:27Z

llvmlite/tests/test_binding.py

-        self.assertIn(patch, range(10))
+        valid = (14, 15)
+        self.assertIn(major, valid)
+        self.assertIn(patch, range(8))


(Max patch versions are 14.0.6 and 15.0.7)

This includes changes: - Adding the Windows LLVM 15 config to the Azure setup. - Converting the commands in the Azure pipeline file to a call to the setup_conda_environment.cmd script (it's easier to edit a script rather than individual lines in the Azure file). - Installation of LLVM 15 from conda-forge. - A workaround / supporting change for the conda-forge solution to conda-forge/llvmdev-feedstock#175.

gmarkall

The code changes here have my approval - I've already done a couple of rounds of review internally on this PR, and I made a few edits for a couple of left over changes that needed removing (adjustments to the Makefile, and a test that was accidentally skipped).

I added the CI config so we can test with LLVM 15 from conda-forge on each platform - someone else might want to review those changes.

I ran the Numba test suite with this PR locally, and didn't encounter any issues - the two passes that aren't available with the legacy pass manager anymore (arg promotion and loop unswitch) don't seem to be used by Numba.

As I see it, this PR provides us a way to give experimental support for LLVM 15 once merged, with no further blockers. Following that, I think we can:

Build an LLVM 15 llvmdev package and move testing to LLVM 15 exclusively, with the aim of officially moving to 15.
Move to the new pass manager, and re-enable the passes that have to be disabled with LLVM 15 at present.
Switch to opaque pointers (at least in the binding layer)
Move to LLVM 16+

Some discussion of the ordering / dependencies of these is warranted though - perhaps at the next office hours.

modiking

Thanks for putting this up!

modiking · 2024-03-14T18:32:02Z

ffi/passmanagers.cpp

@@ -161,8 +161,13 @@ LLVMPY_AddCallGraphDOTPrinterPass(LLVMPassManagerRef PM) {

 API_EXPORT(void)
 LLVMPY_AddDotDomPrinterPass(LLVMPassManagerRef PM, bool showBody) {
+#if LLVM_VERSION_MAJOR <= 14


Suggest being consistent with the #ifs, either all "<= 14" or all "< 15"

modiking · 2024-03-14T18:38:59Z

ffi/passmanagers.cpp

@@ -333,12 +345,14 @@ LLVMPY_AddLoopUnrollAndJamPass(LLVMPassManagerRef PM) {
    LLVMAddLoopUnrollAndJamPass(PM);
 }

+#if LLVM_VERSION_MAJOR < 15
 API_EXPORT(void)
 LLVMPY_AddLoopUnswitchPass(LLVMPassManagerRef PM, bool optimizeForSize,


For reference the loop unswitch pass has been renamed to SimpleLoopUnswitchPass in the new pass manager (https://reviews.llvm.org/D124376). Naturally to enable this we need to move to NPM so it makes sense that this is disabled for now but it can come back with NPM.

Actually createSimpleLoopUnswitchLegacyPass still exists in LLVM-15 so it can be dropped in here without issue.

API_EXPORT(void) LLVMPY_AddLoopUnswitchPass(LLVMPassManagerRef PM, bool optimizeForSize, bool hasBranchDivergence) { #if LLVM_VERSION_MAJOR < 15 unwrap(PM)->add( createLoopUnswitchPass(optimizeForSize, hasBranchDivergence)); #else unwrap(PM)->add( createSimpleLoopUnswitchLegacyPass(optimizeForSize)); #endif }

Thanks for finding this, I have updated the code.

modiking · 2024-03-14T18:40:32Z

ffi/passmanagers.cpp

@@ -244,10 +254,12 @@ LLVMPY_AddAlwaysInlinerPass(LLVMPassManagerRef PM, bool insertLifetime) {
    unwrap(PM)->add(llvm::createAlwaysInlinerLegacyPass(insertLifetime));
 }

+#if LLVM_VERSION_MAJOR < 15
 API_EXPORT(void)
 LLVMPY_AddArgPromotionPass(LLVMPassManagerRef PM, unsigned int maxElements) {


Ditto with ArgumentPromotionPass in NPM as below

Did you mean legacy version of this pass is available as well? I couldn't find it.

Sorry I meant the new pass version, I wrote both of these comments together then saw the legacy pass still existed.

On that note, the NPM setup slots in differently (using pipeline tuning options) so we can’t replace these in-situ. No needed for this change though.

…on checking condtions

gmarkall

Re-approving following latest changes.

modiking · 2024-03-18T18:15:15Z

LGTM

gmarkall · 2024-03-19T14:42:41Z

Added "Pending Buildfarm" to make sure this is all OK on all other platforms.

sklam · 2024-03-29T15:17:48Z

BFID: llvmlite_275

sklam · 2024-03-29T16:18:38Z

llvmlite_275 passed

sklam · 2024-03-30T09:10:38Z

Also passed buildfarm test against numba main (numba_yaml_448)

yashssh added 2 commits March 12, 2024 16:41

get rid of -flto

2a0f69b

yashssh marked this pull request as ready for review March 13, 2024 17:13

yashssh changed the title ~~Support llvmlite to building against llvm15~~ [Draft] Support llvmlite to building against llvm15 Mar 13, 2024

yashssh commented Mar 13, 2024

View reviewed changes

ffi/Makefile.linux Outdated Show resolved Hide resolved

Update ffi/Makefile.linux

36ae6cd

gmarkall reviewed Mar 13, 2024

View reviewed changes

llvmlite/tests/test_binding.py Outdated Show resolved Hide resolved

Update llvmlite/tests/test_binding.py

34406e8

gmarkall reviewed Mar 13, 2024

View reviewed changes

Update assembly.cpp

6206a41

gmarkall changed the title ~~[Draft] Support llvmlite to building against llvm15~~ Support building against llvm15 Mar 13, 2024

yashssh and others added 3 commits March 14, 2024 10:31

fix python formatting

247fa16

run clang format

64bb499

Run CI with LLVM 15 on Linux and macOS

bec8eec

esc added 2 - In Progress 4 - Waiting on reviewer and removed 2 - In Progress labels Mar 14, 2024

gmarkall added 2 commits March 14, 2024 13:42

Windows build: work around llvm/llvm-project#83802

a585985

gmarkall force-pushed the llvm14to15 branch from c361f27 to cdd4c5f Compare March 14, 2024 13:50

gmarkall previously approved these changes Mar 14, 2024

View reviewed changes

gmarkall added 4 - Waiting on second reviewer and removed 4 - Waiting on reviewer labels Mar 14, 2024

modiking reviewed Mar 14, 2024

View reviewed changes

Use legacysimpleloopunswitch with old PM; more consistency with versi…

2d5d8ee

…on checking condtions

yashssh dismissed gmarkall’s stale review via 2d5d8ee March 18, 2024 06:52

gmarkall approved these changes Mar 18, 2024

View reviewed changes

sklam self-requested a review March 19, 2024 14:41

sklam added the Pending BuildFarm For PRs that have been reviewed but pending a push through our buildfarm label Mar 19, 2024

gmarkall added the 4 - Waiting on CI label Mar 19, 2024

sklam added BuildFarm Passed For PRs that have been through the buildfarm and passed and removed Pending BuildFarm For PRs that have been reviewed but pending a push through our buildfarm labels Mar 30, 2024

sklam approved these changes Mar 30, 2024

View reviewed changes

sklam added 5 - Ready to merge and removed 4 - Waiting on CI 4 - Waiting on second reviewer labels Mar 30, 2024

sklam merged commit 42b9834 into numba:main Mar 30, 2024
23 checks passed

modiking added a commit to modiking/llvmlite that referenced this pull request Apr 24, 2024

Following numba#1035 on updating pipeline files

7376017

sklam added this to the v0.43.0-rc1 milestone Apr 29, 2024

This was referenced May 9, 2024

LLVM update plan? #1048

Open

LLVM 15 conda recipe #1036

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support building against llvm15 #1035

Support building against llvm15 #1035

yashssh commented Mar 13, 2024

yashssh commented Mar 13, 2024 •

edited

Loading

gmarkall Mar 13, 2024

gmarkall Mar 13, 2024

gmarkall Mar 13, 2024

gmarkall Mar 13, 2024

gmarkall left a comment

modiking left a comment

modiking Mar 14, 2024

modiking Mar 14, 2024

modiking Mar 14, 2024

modiking Mar 14, 2024

yashssh Mar 18, 2024

modiking Mar 14, 2024

yashssh Mar 18, 2024

modiking Mar 18, 2024

gmarkall left a comment

modiking commented Mar 18, 2024

gmarkall commented Mar 19, 2024

sklam commented Mar 29, 2024

sklam commented Mar 29, 2024

sklam commented Mar 30, 2024

Support building against llvm15 #1035

Support building against llvm15 #1035

Conversation

yashssh commented Mar 13, 2024

yashssh commented Mar 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gmarkall left a comment

Choose a reason for hiding this comment

modiking left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gmarkall left a comment

Choose a reason for hiding this comment

modiking commented Mar 18, 2024

gmarkall commented Mar 19, 2024

sklam commented Mar 29, 2024

sklam commented Mar 29, 2024

sklam commented Mar 30, 2024

yashssh commented Mar 13, 2024 •

edited

Loading