Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rustbuild] Implement testing and refactor configuration #48521

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
1c55459
[VIC] Prevent bootstrap test from overwriting initial compile
Mark-Simulacrum Feb 11, 2018
e72b895
Remove ONLY_BUILD_TARGETS.
Mark-Simulacrum Feb 11, 2018
f61d354
Remove ONLY_BUILD.
Mark-Simulacrum Feb 11, 2018
3124b25
Refactor run_host_only to have the proper effect.
Mark-Simulacrum Feb 11, 2018
48923dd
[VIC] Remove --host and --target arguments to configure
Mark-Simulacrum Feb 11, 2018
9321fba
Refactor to improve readability.
Mark-Simulacrum Feb 11, 2018
3ae9c7a
Avoid only-hosts to change what is built.
Mark-Simulacrum Feb 11, 2018
db0d96d
Remove unused fields on Crate struct
Mark-Simulacrum Feb 14, 2018
9a8af65
[VIC] Fix cross-compiling rustdoc.
Mark-Simulacrum Feb 15, 2018
7f1426d
Print out the sysroot and libdir on verbose builds.
Mark-Simulacrum Feb 15, 2018
23109dc
Pull paths out of Subcommand enum
Mark-Simulacrum Feb 16, 2018
5cd0002
Move StepDescription::run to Builder::run_step_descriptions.
Mark-Simulacrum Feb 16, 2018
67e6482
Deny warnings
Mark-Simulacrum Feb 16, 2018
ba7a588
Remove src from Build
Mark-Simulacrum Feb 16, 2018
b11dae6
Remove out from Build
Mark-Simulacrum Feb 17, 2018
ca2e204
Remove initial_rustc and initial_cargo from Build
Mark-Simulacrum Feb 17, 2018
7826a74
Remove local_rebuild from Build
Mark-Simulacrum Feb 17, 2018
ca059b9
Move is_sudo to Config
Mark-Simulacrum Feb 17, 2018
cb447a7
Remove hosts from Build
Mark-Simulacrum Feb 17, 2018
ff5117b
Remove targets from Build
Mark-Simulacrum Feb 17, 2018
13d97f3
Remove build from Build
Mark-Simulacrum Feb 17, 2018
9a695be
Remove verbosity from Build
Mark-Simulacrum Feb 17, 2018
f0185ec
Remove fail_fast from Build
Mark-Simulacrum Feb 17, 2018
9fe9348
Build is default-able
Mark-Simulacrum Feb 17, 2018
2b0ce18
Refactor default values for configuration into Default
Mark-Simulacrum Feb 18, 2018
ed4d1e0
Cleanup to config
Mark-Simulacrum Feb 18, 2018
aca2d80
Stop copying llvm configuration from Llvm struct
Mark-Simulacrum Feb 18, 2018
d4e7d53
Cleanup target creation
Mark-Simulacrum Feb 18, 2018
e276ba7
Stop copying install configuration from Install struct
Mark-Simulacrum Feb 18, 2018
a096036
Remove doc_tests from Build
Mark-Simulacrum Feb 18, 2018
483e099
Stop copying dist configuration from Dist struct
Mark-Simulacrum Feb 18, 2018
15a90fa
Stop copying rust configuration from Rust struct
Mark-Simulacrum Feb 19, 2018
2c3860e
Regroup Config
Mark-Simulacrum Feb 19, 2018
c492b71
Stop copying build configuration from Build struct
Mark-Simulacrum Feb 19, 2018
13a4184
Prepare initial_{cargo,rustc} for removal from Config
Mark-Simulacrum Feb 20, 2018
34d1be2
Cleanup hosts/targets handling
Mark-Simulacrum Feb 20, 2018
7ea1d36
Let bootstrap.py handle build triple configuration
Mark-Simulacrum Feb 20, 2018
ff40909
Simplify internment
Mark-Simulacrum Feb 21, 2018
db2187d
Use Intern::intern instead of intern_{string,str,path}
Mark-Simulacrum Feb 21, 2018
6d922e9
Permit ser/deserializing Interned values
Mark-Simulacrum Feb 21, 2018
423aae7
Cleanup Interned impls to remove 'static bounds
Mark-Simulacrum Feb 21, 2018
6a22f0b
Implement PartialOrd, Ord for Interned
Mark-Simulacrum Feb 21, 2018
4f2a860
Refactor {build,host,target} loading
Mark-Simulacrum Feb 21, 2018
3744f53
Refactor initial_{rustc,cargo} defaults into Build::default
Mark-Simulacrum Feb 21, 2018
75ee753
Create Config directly instead of through Default
Mark-Simulacrum Feb 21, 2018
5147b1e
Remove Default impls for Config and Build
Mark-Simulacrum Feb 21, 2018
c310e4d
Polish config loading
Mark-Simulacrum Feb 22, 2018
d2c266a
Set stage default of 2
Mark-Simulacrum Feb 22, 2018
8ae4f7d
Remove out from Config
Mark-Simulacrum Feb 23, 2018
14b227b
Remove initial_{cargo,rustc} from Config
Mark-Simulacrum Feb 23, 2018
d76bf80
Remove hosts, targets from Config
Mark-Simulacrum Feb 23, 2018
de8ee0d
Remove build from Config
Mark-Simulacrum Feb 23, 2018
7385641
Format via rustfmt
Mark-Simulacrum Feb 23, 2018
16d5fe3
Exclusively utilize Builder in methods
Mark-Simulacrum Feb 23, 2018
7503faa
Clear out build directories from Builder::cargo
Mark-Simulacrum Feb 24, 2018
8e6593b
Call {std,test,rustc}_cargo in Builder::cargo
Mark-Simulacrum Feb 24, 2018
20ded30
[VIC] Move codegen backend mangling to copy step
Mark-Simulacrum Feb 24, 2018
98ce89f
Move run_cargo to Builder
Mark-Simulacrum Feb 24, 2018
2bdbb32
Refactor to prevent passing stamp to run_cargo
Mark-Simulacrum Feb 24, 2018
c2a46fc
Move some of prepare_tool_cargo to Builder
Mark-Simulacrum Feb 24, 2018
a68a8e5
[VIC] Add TestTool and RustcTool modes
Mark-Simulacrum Feb 24, 2018
699689b
[VIC] Build rustc libraries for compiler host for RustcTool
Mark-Simulacrum Feb 24, 2018
a1c22f6
Make cargo invocation start with subcommand
Mark-Simulacrum Feb 24, 2018
b33594a
[VIC] Add tests to Rustbuild
Mark-Simulacrum Feb 25, 2018
2fe3ede
Test extended builds
Mark-Simulacrum Feb 25, 2018
1f43cfa
Use fs::{read,read_string,write} instead of File
Mark-Simulacrum Feb 25, 2018
36849ec
Permit specifying build.build in config.toml
Mark-Simulacrum Feb 25, 2018
022ce70
Write dist-src configure to dist.src-tarball
Mark-Simulacrum Feb 25, 2018
3831685
Add more asserts to test cases
Mark-Simulacrum Feb 25, 2018
a174d0e
Fix tidy
Mark-Simulacrum Feb 25, 2018
d7e8200
Clean up Config::libdir_relative
Mark-Simulacrum Feb 25, 2018
b82a798
Fix compilation error
Mark-Simulacrum Feb 25, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions src/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions src/bootstrap/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,6 @@ serde_json = "1.0.2"
toml = "0.4"
lazy_static = "0.2"
time = "0.1"

[dev-dependencies]
pretty_assertions = "0.5"
2 changes: 1 addition & 1 deletion src/bootstrap/bin/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ extern crate bootstrap;

use std::env;

use bootstrap::{Config, Build};
use bootstrap::{Build, Config};

fn main() {
let args = env::args().skip(1).collect::<Vec<_>>();
Expand Down
53 changes: 33 additions & 20 deletions src/bootstrap/bin/rustc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,15 @@ fn main() {
// Dirty code for borrowing issues
let mut new = None;
if let Some(current_as_str) = args[i].to_str() {
if (&*args[i - 1] == "-C" && current_as_str.starts_with("metadata")) ||
current_as_str.starts_with("-Cmetadata") {
if (&*args[i - 1] == "-C" && current_as_str.starts_with("metadata"))
|| current_as_str.starts_with("-Cmetadata")
{
new = Some(format!("{}-{}", current_as_str, s));
}
}
if let Some(new) = new { args[i] = new.into(); }
if let Some(new) = new {
args[i] = new.into();
}
}
}

Expand Down Expand Up @@ -95,19 +98,21 @@ fn main() {
let rustc = env::var_os(rustc).unwrap_or_else(|| panic!("{:?} was not set", rustc));
let libdir = env::var_os(libdir).unwrap_or_else(|| panic!("{:?} was not set", libdir));
let mut dylib_path = bootstrap::util::dylib_path();
dylib_path.insert(0, PathBuf::from(libdir));
dylib_path.insert(0, PathBuf::from(&libdir));

let mut cmd = Command::new(rustc);
cmd.args(&args)
.arg("--cfg")
.arg(format!("stage{}", stage))
.env(bootstrap::util::dylib_path_var(),
env::join_paths(&dylib_path).unwrap());
.env(
bootstrap::util::dylib_path_var(),
env::join_paths(&dylib_path).unwrap(),
);

if let Some(target) = target {
// The stage0 compiler has a special sysroot distinct from what we
// actually downloaded, so we just always pass the `--sysroot` option.
cmd.arg("--sysroot").arg(sysroot);
cmd.arg("--sysroot").arg(&sysroot);

// When we build Rust dylibs they're all intended for intermediate
// usage, so make sure we pass the -Cprefer-dynamic flag instead of
Expand All @@ -130,9 +135,7 @@ fn main() {
cmd.arg(format!("-Clinker={}", target_linker));
}

let crate_name = args.windows(2)
.find(|a| &*a[0] == "--crate-name")
.unwrap();
let crate_name = args.windows(2).find(|a| &*a[0] == "--crate-name").unwrap();
let crate_name = &*crate_name[1];

// If we're compiling specifically the `panic_abort` crate then we pass
Expand All @@ -147,8 +150,7 @@ fn main() {
// `compiler_builtins` are unconditionally compiled with panic=abort to
// workaround undefined references to `rust_eh_unwind_resume` generated
// otherwise, see issue https://github.com/rust-lang/rust/issues/43095.
if crate_name == "panic_abort" ||
crate_name == "compiler_builtins" && stage != "0" {
if crate_name == "panic_abort" || crate_name == "compiler_builtins" && stage != "0" {
cmd.arg("-C").arg("panic=abort");
}

Expand All @@ -160,7 +162,11 @@ fn main() {
cmd.arg("-Cdebuginfo=1");
}
let debug_assertions = match env::var("RUSTC_DEBUG_ASSERTIONS") {
Ok(s) => if s == "true" { "y" } else { "n" },
Ok(s) => if s == "true" {
"y"
} else {
"n"
},
Err(..) => "n",
};

Expand All @@ -169,7 +175,8 @@ fn main() {
if crate_name == "compiler_builtins" {
cmd.arg("-C").arg("debug-assertions=no");
} else {
cmd.arg("-C").arg(format!("debug-assertions={}", debug_assertions));
cmd.arg("-C")
.arg(format!("debug-assertions={}", debug_assertions));
}

if let Ok(s) = env::var("RUSTC_CODEGEN_UNITS") {
Expand All @@ -182,10 +189,12 @@ fn main() {
// Emit save-analysis info.
if env::var("RUSTC_SAVE_ANALYSIS") == Ok("api".to_string()) {
cmd.arg("-Zsave-analysis");
cmd.env("RUST_SAVE_ANALYSIS_CONFIG",
"{\"output_file\": null,\"full_docs\": false,\
\"pub_only\": true,\"reachable_only\": false,\
\"distro_crate\": true,\"signatures\": false,\"borrow_data\": false}");
cmd.env(
"RUST_SAVE_ANALYSIS_CONFIG",
"{\"output_file\": null,\"full_docs\": false,\
\"pub_only\": true,\"reachable_only\": false,\
\"distro_crate\": true,\"signatures\": false,\"borrow_data\": false}",
);
}

// Dealing with rpath here is a little special, so let's go into some
Expand Down Expand Up @@ -216,7 +225,6 @@ fn main() {
// to change a flag in a binary?
if env::var("RUSTC_RPATH") == Ok("true".to_string()) {
let rpath = if target.contains("apple") {

// Note that we need to take one extra step on macOS to also pass
// `-Wl,-instal_name,@rpath/...` to get things to work right. To
// do that we pass a weird flag to the compiler to get it to do
Expand Down Expand Up @@ -244,7 +252,10 @@ fn main() {
}

// When running miri tests, we need to generate MIR for all libraries
if env::var("TEST_MIRI").ok().map_or(false, |val| val == "true") {
if env::var("TEST_MIRI")
.ok()
.map_or(false, |val| val == "true")
{
cmd.arg("-Zalways-encode-mir");
if stage != "0" {
cmd.arg("-Zmiri");
Expand Down Expand Up @@ -280,6 +291,8 @@ fn main() {

if verbose > 1 {
eprintln!("rustc command: {:?}", cmd);
eprintln!("sysroot: {:?}", sysroot);
eprintln!("libdir: {:?}", libdir);
}

// Actually run the compiler!
Expand Down
16 changes: 11 additions & 5 deletions src/bootstrap/bin/rustdoc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,10 @@ fn main() {
.arg("dox")
.arg("--sysroot")
.arg(sysroot)
.env(bootstrap::util::dylib_path_var(),
env::join_paths(&dylib_path).unwrap());
.env(
bootstrap::util::dylib_path_var(),
env::join_paths(&dylib_path).unwrap(),
);

// Force all crates compiled by this compiler to (a) be unstable and (b)
// allow the `rustc_private` feature to link to other unstable crates
Expand All @@ -55,16 +57,20 @@ fn main() {
cmd.arg("-Z").arg("force-unstable-if-unmarked");
}
if let Some(linker) = env::var_os("RUSTC_TARGET_LINKER") {
cmd.arg("--linker").arg(linker).arg("-Z").arg("unstable-options");
cmd.arg("--linker")
.arg(linker)
.arg("-Z")
.arg("unstable-options");
}

// Bootstrap's Cargo-command builder sets this variable to the current Rust version; let's pick
// it up so we can make rustdoc print this into the docs
if let Some(version) = env::var_os("RUSTDOC_CRATE_VERSION") {
// This "unstable-options" can be removed when `--crate-version` is stabilized
cmd.arg("-Z")
.arg("unstable-options")
.arg("--crate-version").arg(version);
.arg("unstable-options")
.arg("--crate-version")
.arg(version);
}

if verbose > 1 {
Expand Down
12 changes: 6 additions & 6 deletions src/bootstrap/bin/sccache-plus-cl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@ fn main() {
env::remove_var("CXX");
let mut cfg = cc::Build::new();
cfg.cargo_metadata(false)
.out_dir("/")
.target(&target)
.host(&target)
.opt_level(0)
.warnings(false)
.debug(false);
.out_dir("/")
.target(&target)
.host(&target)
.opt_level(0)
.warnings(false)
.debug(false);
let compiler = cfg.get_compiler();

// Invoke sccache with said compiler
Expand Down
1 change: 1 addition & 0 deletions src/bootstrap/bootstrap.py
Original file line number Diff line number Diff line change
Expand Up @@ -751,6 +751,7 @@ def bootstrap():
env["SRC"] = build.rust_root
env["BOOTSTRAP_PARENT_ID"] = str(os.getpid())
env["BOOTSTRAP_PYTHON"] = sys.executable
env["BUILD_DIR"] = build.build_dir
run(args, env=env, verbose=build.verbose)


Expand Down
Loading