[bootstrap] Add support for building gcc and libgccjit #125419

GuillaumeGomez · 2024-05-22T19:15:11Z

As @eholk summarized below:

From my understanding, this change would add libgccjit as an optional component to the Rust distribution. This library is licensed under GPLv2 and currently we do not have any other components under that license so it would be a new license, and one that is generally more restrictive than the other licenses we use.

It'll greatly improve the experience for anyone wanting to work on the GCC backend from the compiler.

Should help with #124172.
Will unblock #124353.

r? @Kobzol

rustbot · 2024-05-22T19:15:19Z

⚠️ Warning ⚠️

These commits modify submodules.

rustbot · 2024-05-22T19:15:21Z

This PR changes how LLVM is built. Consider updating src/bootstrap/download-ci-llvm-stamp.

GuillaumeGomez · 2024-05-23T13:33:04Z

From what I can see, it's not possible to tell reuse to allow deprecated licenses. So in this case, I have no clue how we can go around this issue. If someone has an idea?

Kobzol · 2024-05-23T15:57:19Z

@davidtwco Any suggestions on what to do about the GPL-2.0 license of GCC?

davidtwco · 2024-05-28T13:01:45Z

@davidtwco Any suggestions on what to do about the GPL-2.0 license of GCC?

Will get back to you about this

davidtwco · 2024-05-28T14:28:28Z

@wesleywiser and I discussed this and we're not sure about how to get REUSE to play ball (@pietroalbini might know), but we've forwarded this on to t-compiler's council rep to double-check that adding gcc like this is okay, license-wise.

pietroalbini · 2024-06-05T09:20:40Z

From what I can see, it's not possible to tell reuse to allow deprecated licenses. So in this case, I have no clue how we can go around this issue. If someone has an idea?

Note that REUSE doesn't mark the GPL 2.0 license as deprecated. What is deprecated is referring to it as GPL-2.0. You should refer to the license as either GPL-2.0-only or GPL-2.0-or-later, depending on what is appropriate.

Also, the warning doesn't come from REUSE itself, but from the SPDX License List (you'll find GPL-2.0 as deprecated in there).

eholk · 2024-06-07T16:44:49Z

We discussed this at the Council meeting today and we don't have any concerns with this change. However, we would like to confirm with the Foundation. @abibroom - do you know of any concerns from the Foundation side (feel free to ping me on Zulip or reach out via email if you'd rather not discuss this publicly.

From my understanding, this change would add libgccjit as an optional component to the Rust distribution. This library is licensed under GPLv2 and currently we do not have any other components under that license so it would be a new license, and one that is generally more restrictive than the other licenses we use.

Kobzol · 2024-06-09T12:59:45Z

@bors try

Just to see what happens on Linux dist.

Kobzol · 2024-06-09T12:58:42Z

src/bootstrap/src/core/build_steps/dist.rs

@@ -2365,6 +2366,10 @@ impl Step for RustDev {
        // just broadly useful to be able to link against the bundled LLVM.
        tarball.add_dir(builder.llvm_out(target).join("include"), "include");

+        tarball.add_dir(builder.gcc_out(target).join("install/lib/libgccjit.so"), "libgccjit.so");


What will this do on Windows? 🤔 Does the gcc build also work on non-Linux OSes, in general?

No it doesn't, good catch!

bors · 2024-06-09T13:00:58Z

⌛ Trying commit f302cfd with merge 9e6f6bf...

[bootstrap] Add gcc to dist generation As `@eholk` summarized below: > From my understanding, this change would add libgccjit as an optional component to the Rust distribution. This library is licensed under GPLv2 and currently we do not have any other components under that license so it would be a new license, and one that is generally more restrictive than the other licenses we use. It'll greatly improve the experience for anyone wanting to work on the GCC backend from the compiler. Should help with rust-lang#124172. Will unblock rust-lang#124353. r? `@Kobzol`

nikic · 2024-06-09T13:18:59Z

Can this be done without adding gcc as a submodule?

bors · 2024-06-09T14:35:57Z

💔 Test failed - checks-actions

abibroom · 2024-06-10T12:53:59Z

We discussed this at the Council meeting today and we don't have any concerns with this change. However, we would like to confirm with the Foundation. @abibroom - do you know of any concerns from the Foundation side (feel free to ping me on Zulip or reach out via email if you'd rather not discuss this publicly.

I'd like to be able to say this is all fine, but want to run it past Foundation legal counsel first just to make sure. Please bear with me whilst that happens!

abibroom · 2024-06-13T15:45:35Z

We discussed this at the Council meeting today and we don't have any concerns with this change. However, we would like to confirm with the Foundation. @abibroom - do you know of any concerns from the Foundation side (feel free to ping me on Zulip or reach out via email if you'd rather not discuss this publicly.

I'd like to be able to say this is all fine, but want to run it past Foundation legal counsel first just to make sure. Please bear with me whilst that happens!

We should be good to go with this. Legal raised two points, neither of which I believe to be blocking concerns:

Compliance with the GPLv2 license, specifically, the requirement to provide source code as well as binary distributions. This has hopefully already been thoroughly considered before deciding to add the component.
Enabling downstream compliance: "When it comes to downstream distribution, anyone providing a binary distribution of Rust incorporating this component will need to provide source code for the whole distribution. This is ultimately their problem, but if you're aware of anyone who's doing this currently, it's probably worth talking through those cases and figuring out how to provide notice to them and other potential distributors of their additional obligations with respect to this component."

eholk · 2024-06-14T17:48:05Z

Enabling downstream compliance: "When it comes to downstream distribution, anyone providing a binary distribution of Rust incorporating this component will need to provide source code for the whole distribution. This is ultimately their problem, but if you're aware of anyone who's doing this currently, it's probably worth talking through those cases and figuring out how to provide notice to them and other potential distributors of their additional obligations with respect to this component."

@abibroom - just to make sure, if someone wanted to make a downstream binary distribution without releasing source code (which strikes me as rather unlikely), they could still do so if they chose not to include the libgccjit component?

I want to make sure this change doesn't effectively mean we have to relicense Rust as GPL2.

Kobzol · 2024-09-06T17:11:26Z

Ah, crap. On Linux CI, we're not allowed to write to the source directory. Hmm, then I would probably only do the "download dependencies" step if we're not on CI.

Kobzol · 2024-09-06T17:11:52Z

That being said, why is this step executed on CI..? 😕 That shouldn't happen.

Kobzol · 2024-09-13T12:17:31Z

src/bootstrap/src/core/build_steps/dist.rs

@@ -2273,6 +2273,9 @@ impl Step for RustDev {
        tarball.permit_symlinks(true);

        builder.ensure(crate::core::build_steps::llvm::Llvm { target });
+        if target.contains("linux") && target.contains("x86_64") {


Oh this was kept in, we forgot to remove it. Please revert the changes to dist to make this PR only affect local builds.

GuillaumeGomez · 2024-09-13T15:15:20Z

Removed changes to dist.rs.

Kobzol · 2024-09-13T15:58:47Z

@bors r+

bors · 2024-09-13T15:58:50Z

📌 Commit e7fa03b has been approved by Kobzol

It is now in the queue for this repository.

bors · 2024-09-13T18:13:32Z

⌛ Testing commit e7fa03b with merge e0ebd86...

…bzol [bootstrap] Add support for building gcc and libgccjit As `@eholk` summarized below: > From my understanding, this change would add libgccjit as an optional component to the Rust distribution. This library is licensed under GPLv2 and currently we do not have any other components under that license so it would be a new license, and one that is generally more restrictive than the other licenses we use. It'll greatly improve the experience for anyone wanting to work on the GCC backend from the compiler. Should help with rust-lang#124172. Will unblock rust-lang#124353. r? `@Kobzol`

rust-log-analyzer · 2024-09-13T19:20:00Z

The job x86_64-msvc-ext failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

bors · 2024-09-13T19:20:31Z

💔 Test failed - checks-actions

Kobzol · 2024-09-13T19:25:09Z

@bors retry

bors · 2024-09-14T00:26:07Z

⌛ Testing commit e7fa03b with merge 23b04c0...

bors · 2024-09-14T02:50:06Z

☀️ Test successful - checks-actions
Approved by: Kobzol
Pushing 23b04c0 to master...

rust-timer · 2024-09-14T04:09:02Z

Finished benchmarking commit (23b04c0): comparison URL.

Overall result: ✅ improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.6%	[-0.6%, -0.6%]	1
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results (primary -2.9%, secondary -1.8%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.9%	[-2.9%, -2.9%]	1
Improvements ✅ (secondary)	-1.8%	[-1.8%, -1.8%]	1
All ❌✅ (primary)	-2.9%	[-2.9%, -2.9%]	1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 756.859s -> 756.265s (-0.08%)
Artifact size: 341.11 MiB -> 341.19 MiB (0.02%)

rustbot assigned Kobzol May 22, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels May 22, 2024