Flesh out CPack packaging for releases. #5135

alexreinking · 2020-07-23T00:01:58Z

Main changes:

Adds CPack scripts for Windows, Mac, and Linux to create release packages easily.
a. Linux packages include Debug/Release x Static/Shared
b. macOS packages include Release x Static/Shared
c. Windows packages include Debug/Release x Shared
Starts semantic versioning for Halide at 10.0.0
Renames WITH_EXCEPTIONS macro to HALIDE_WITH_EXCEPTIONS
Renamed / introduced CMake variables:
a. LLVM_USE_SHARED_LLVM_LIBS -> Halide_SHARED_LLVM. This was never an LLVM option, so we need to stay out of their namespace.
b. Halide_BUNDLE_LLVM -> On all major platforms, unpacks LLVM's static libraries and includes their objects in the Halide static library.

When using find_package, users can now request either the static or shared versions of Halide. This works in one of several ways (in order of preference):

A user may explicitly request one or the other as a component. Ie. find_package(Halide REQUIRED static). If it cannot load the requested libraries, it dies.
A user may set the Halide_SHARED_LIBS variable. When true-y, it loads the shared libs, else the static ones. If it cannot load the requested libraries, it dies.
When no library-type component is specified and Halide_SHARED_LIBS is not defined, it imitates the behavior of FetchContent by examining the BUILD_SHARED_LIBS variable. If it is true-y or not defined, it tries to load the shared libraries first. Otherwise it tries to load the static libraries first. Finally, it falls back to the other type.

As an advanced use-case, the user may request both static and shared by specifying both components. In this case, the targets Halide::shared::{Halide,Generator,RunGenMain} and Halide::static::{Halide,Generator,RunGenMain} are created, but Halide::Halide is unavailable. As a consequence, the convenience method add_halide_library is also not usable.

Fixes #3971
Fixes #3972
Fixes #4254

abadams · 2020-07-23T00:20:17Z

We generally don't break user code without a release with a deprecation warning first, so I'm not sure '0.' is necessary. We'll just bump the major every time we delete some deprecated APIs or backends. It happens infrequently enough.

steven-johnson · 2020-07-23T00:35:59Z

Makefile

@@ -199,7 +199,7 @@ RISCV_CXX_FLAGS=$(if $(WITH_RISCV), -DWITH_RISCV, )
 RISCV_LLVM_CONFIG_LIB=$(if $(WITH_RISCV), riscv, )

 INTROSPECTION_CXX_FLAGS=$(if $(WITH_INTROSPECTION), -DWITH_INTROSPECTION, )
-EXCEPTIONS_CXX_FLAGS=$(if $(WITH_EXCEPTIONS), -DWITH_EXCEPTIONS -fexceptions, )
+EXCEPTIONS_CXX_FLAGS=$(if $(WITH_EXCEPTIONS), -DHALIDE_WITH_EXCEPTIONS -fexceptions, )


Why is this renaming necessary?

Because the behavior of Error.cpp changes depending on whether or not Halide was built with exceptions, so our users might need to know whether it was enabled. The old name, WITH_EXCEPTIONS, is almost guaranteed to conflict with another project so I prefixed it. It also now matches HALIDE_ENABLE_RTTI.

I don't think this macro is ever exported though. It's only used in Error.cpp, and no headers. This is now inconsistent with all the other WITH flags in the makefile.

If we think clients might need to check it dynamically should we add an api like we did for llvm_version?

The use cases for checking it would seem to need to be compile-time mostly, I would think (e.g. so that you can wrap try/catch or not).

So perhaps it should be something exported by the Halide package in cmake (not sure of correct terminology here) as the variable HALIDE_WITH_EXCEPTIONS? But this is independent of what the #define is called.

Linking with varying exception-handling capabilities is generally risky, as is linking with varying RTTI info, which is why packages that allow users to disable them have to propagate that information at build time.

abadams · 2020-07-23T19:26:47Z

Question: What is the expectation of build compatibility across versions? E.g. we're talking about renaming cmake config vars in another thread. Does this require bumping the major because it requires downstream users to adjust their build setup?

alexreinking · 2020-07-23T19:32:17Z

Question: What is the expectation of build compatibility across versions? E.g. we're talking about renaming cmake config vars in another thread. Does this require bumping the major because it requires downstream users to adjust their build setup?

That's a good question. Semantic versioning is fundamentally about link-time compatibility. The hard requirement is that you should always be able to load a newer library of the same major version than the one you linked to. I think we should extend that to our build API. However, major version 0 has no such requirements and we should feel free to make breaking changes in the CMake until a numbered release is issued.

abadams · 2020-07-23T19:52:53Z

Suggestion for versioning from offline discussion: Just match the version number of the included LLVM, and try to release at the same cadence (6 months)

steven-johnson · 2020-07-23T19:55:32Z

Suggestion for versioning from offline discussion: Just match the version number of the included LLVM, and try to release at the same cadence (6 months)

SGTM.

alexreinking · 2020-07-24T20:34:54Z

Question: do we have any need to support the following scenarios:

A single project uses both static Halide and shared Halide
A single project uses multiple versions of Halide

abadams · 2020-07-24T21:20:43Z

By "Halide" you mean the compiler, right? Not Halide-generated code?

Assuming you mean the compiler, I don't think we should expend any effort in supporting that.

alexreinking · 2020-07-24T21:35:22Z

Yes, I mean the compiler / library, not the generator outputs.

abadams · 2020-07-24T22:27:23Z

Yeah, these multiple version issues normally only come up for large projects with multiple subteams, and large projects right now typically use Halide as an AOT compiler, so multiple Halide versions wouldn't ever end up in the same process.

steven-johnson · 2020-07-27T17:10:54Z

(Monday morning check: is this still pending more work to do, or is it ready for review?)

alexreinking · 2020-07-27T21:04:52Z

(Monday morning check: is this still pending more work to do, or is it ready for review?)

Still a couple TODOs:

Change version number from 0.1.0 -> 10.0.0 (easy)
Bundle static libs (hard... won't have time until mid-week, sorry)
macOS packaging

alexreinking · 2020-07-27T23:59:03Z

A mac mini just arrived today, so I'll be able to test on mac imminently.

alexreinking · 2020-08-04T00:02:26Z

PR now includes version 10.0 bump and macOS packaging script (using zsh since I did everything on Catalina).

Still TODO: bundling static libs on Linux / macOS. This will likely be a hack.

…RTTI

1. Support selecting library type in `find_package`. 2. Support linking to shared LLVM or bundling static LLVM in libHalide.a 3. Add packaging scripts.

…ing.

alexreinking · 2020-08-07T19:37:15Z

All the major things are here. It would be nice to get the -e o for multi target feature working and some integration tests written.

alexreinking · 2020-08-12T20:09:50Z

Can I bump this? I think it's in a merge-able state.

steven-johnson · 2020-08-12T20:19:41Z

All the major things are here. It would be nice to get the -e o for multi target feature working and some integration tests written.

Is that on my plate to get done, and is it a blocker for landing this?

alexreinking · 2020-08-12T22:13:14Z

You touched that code most recently, right? But it is not a blocker, no.

steven-johnson · 2020-08-12T22:19:20Z

You touched that code most recently, right? But it is not a blocker, no.

Yep. Working on it now.

steven-johnson · 2020-08-13T18:45:36Z

#5183 is ready to review if you want -e o before landing this. (If not, I'm OK with landing this as-is.)

alexreinking · 2020-08-13T20:06:28Z

#5183 is ready to review if you want -e o before landing this. (If not, I'm OK with landing this as-is.)

We can go ahead and land this now. The -e o changes support an orthogonal feature.

alexreinking changed the title ~~Cpack~~ Flesh out CPack packaging for releases. Jul 23, 2020

alexreinking requested a review from steven-johnson July 23, 2020 00:02

alexreinking force-pushed the cpack branch from eeb8290 to 6110725 Compare July 23, 2020 00:05

alexreinking force-pushed the cpack branch from 6110725 to e2e8b28 Compare July 23, 2020 00:24

steven-johnson reviewed Jul 23, 2020

View reviewed changes

alexreinking force-pushed the cpack branch from 9f19e1b to 2fadc5c Compare July 24, 2020 18:16

alexreinking mentioned this pull request Jul 28, 2020

Package manager status #4660

Open

alexreinking added 6 commits August 5, 2020 16:33

Run CMake formatter

b89ec19

Add VS output path to .gitignore, fix IDEA.

cc4f8d1

Rename WITH_EXCEPTIONS to HALIDE_WITH_EXCEPTIONS, like HALIDE_ENABLE_…

7ffc2e5

…RTTI

Set library version to 10.0.0

ad1f4fa

Fix export names for plugins and Python bindings

a95e858

Major packaging improvements.

47f608b

1. Support selecting library type in `find_package`. 2. Support linking to shared LLVM or bundling static LLVM in libHalide.a 3. Add packaging scripts.

alexreinking force-pushed the cpack branch from e001c17 to 47f608b Compare August 5, 2020 23:46

alexreinking added 3 commits August 5, 2020 20:41

Improve readability.

73b8560

Something with WABT is broken... trying to fix.

e6d35ee

Fixing WASM by explicitly importing LLD. Consolidating LLVM deps find…

ad9feb5

…ing.

alexreinking mentioned this pull request Aug 6, 2020

Use / instead of \ in LLVM_DIR on all platforms halide/build_bot#78

Merged

alexreinking added 2 commits August 6, 2020 17:54

Add Windows support for bundling LLVM.

62814b1

Merge branch 'master' into cpack

cfc125d

alexreinking mentioned this pull request Aug 7, 2020

Drop support for LLVM9 #5121

Merged

alexreinking marked this pull request as ready for review August 7, 2020 19:15

This was referenced Aug 7, 2020

Should include doxygen docs in every binary release #4254

Closed

Python bindings should be included in the distrib/ folder #4368

Open

Installs files into wrong locations #4952

Closed

LLVM Custom Module Linking: Add to CMake or remove from Make #4969

Open

steven-johnson approved these changes Aug 13, 2020

View reviewed changes

alexreinking merged commit 1d49c70 into master Aug 13, 2020

alexreinking deleted the cpack branch January 14, 2021 21:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flesh out CPack packaging for releases. #5135

Flesh out CPack packaging for releases. #5135

alexreinking commented Jul 23, 2020 •

edited

Loading

abadams commented Jul 23, 2020

steven-johnson Jul 23, 2020

alexreinking Jul 23, 2020 •

edited

Loading

abadams Jul 23, 2020

abadams Jul 23, 2020

steven-johnson Jul 23, 2020

abadams Jul 23, 2020

alexreinking Jul 23, 2020

abadams commented Jul 23, 2020

alexreinking commented Jul 23, 2020

abadams commented Jul 23, 2020

steven-johnson commented Jul 23, 2020

alexreinking commented Jul 24, 2020

abadams commented Jul 24, 2020

alexreinking commented Jul 24, 2020

abadams commented Jul 24, 2020

steven-johnson commented Jul 27, 2020

alexreinking commented Jul 27, 2020 •

edited

Loading

alexreinking commented Jul 27, 2020

alexreinking commented Aug 4, 2020

alexreinking commented Aug 7, 2020

alexreinking commented Aug 12, 2020

steven-johnson commented Aug 12, 2020

alexreinking commented Aug 12, 2020

steven-johnson commented Aug 12, 2020

steven-johnson commented Aug 13, 2020

alexreinking commented Aug 13, 2020

Flesh out CPack packaging for releases. #5135

Flesh out CPack packaging for releases. #5135

Conversation

alexreinking commented Jul 23, 2020 • edited Loading

abadams commented Jul 23, 2020

steven-johnson Jul 23, 2020

Choose a reason for hiding this comment

alexreinking Jul 23, 2020 • edited Loading

Choose a reason for hiding this comment

abadams Jul 23, 2020

Choose a reason for hiding this comment

abadams Jul 23, 2020

Choose a reason for hiding this comment

steven-johnson Jul 23, 2020

Choose a reason for hiding this comment

abadams Jul 23, 2020

Choose a reason for hiding this comment

alexreinking Jul 23, 2020

Choose a reason for hiding this comment

abadams commented Jul 23, 2020

alexreinking commented Jul 23, 2020

abadams commented Jul 23, 2020

steven-johnson commented Jul 23, 2020

alexreinking commented Jul 24, 2020

abadams commented Jul 24, 2020

alexreinking commented Jul 24, 2020

abadams commented Jul 24, 2020

steven-johnson commented Jul 27, 2020

alexreinking commented Jul 27, 2020 • edited Loading

alexreinking commented Jul 27, 2020

alexreinking commented Aug 4, 2020

alexreinking commented Aug 7, 2020

alexreinking commented Aug 12, 2020

steven-johnson commented Aug 12, 2020

alexreinking commented Aug 12, 2020

steven-johnson commented Aug 12, 2020

steven-johnson commented Aug 13, 2020

alexreinking commented Aug 13, 2020

alexreinking commented Jul 23, 2020 •

edited

Loading

alexreinking Jul 23, 2020 •

edited

Loading

alexreinking commented Jul 27, 2020 •

edited

Loading