From a8ddb68b8d7bf67e58012c2504527e3661c561d1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Sun, 30 Jun 2024 21:34:05 +0200 Subject: [PATCH 1/3] Document `bootstrap` integration with `rustc-perf` --- src/SUMMARY.md | 1 + src/profiling/with_perf.md | 39 +++++++------------------------- src/profiling/with_rustc_perf.md | 35 ++++++++++++++++++++++++++++ src/tests/perf.md | 3 +++ 4 files changed, 47 insertions(+), 31 deletions(-) create mode 100644 src/profiling/with_rustc_perf.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 618591c89..3ff6cfbb7 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -34,6 +34,7 @@ - [Profiling the compiler](./profiling.md) - [with the linux perf tool](./profiling/with_perf.md) - [with Windows Performance Analyzer](./profiling/wpa_profiling.md) + - [with the Rust benchmark suite](./profiling/with_rustc_perf.md) - [crates.io Dependencies](./crates-io.md) # Contributing to Rust diff --git a/src/profiling/with_perf.md b/src/profiling/with_perf.md index 469e237b9..ea3a051a0 100644 --- a/src/profiling/with_perf.md +++ b/src/profiling/with_perf.md @@ -54,46 +54,23 @@ you made in the beginning. But there are some things to be aware of: ### Gathering a perf profile from a `perf.rust-lang.org` test -Often we want to analyze a specific test from `perf.rust-lang.org`. To -do that, the first step is to clone -[the rustc-perf repository][rustc-perf-gh]: +Often we want to analyze a specific test from `perf.rust-lang.org`. +The easiest way to do that is to use the [rustc-perf][rustc-perf] +benchmarking suite, this approach is described [here](with_rustc_perf.md). -```bash -git clone https://github.com/rust-lang/rustc-perf -``` - -[rustc-perf-gh]: https://github.com/rust-lang/rustc-perf - -#### Doing it the easy way - -Once you've cloned the repo, you can use the `collector` executable to -do profiling for you! You can find -[instructions in the rustc-perf readme][rustc-perf-readme]. - -[rustc-perf-readme]: https://github.com/rust-lang/rustc-perf/blob/master/collector/README.md#profiling - -For example, to measure the clap-rs test, you might do: +Instead of using the benchmark suite CLI, you can also profile the benchmarks manually. First, +you need to clone the [rustc-perf][rustc-perf] repository: ```bash -./target/release/collector \ - --output-repo /path/to/place/output \ - profile perf-record \ - --rustc /path/to/rustc/executable/from/your/build/directory \ - --cargo `which cargo` \ - --filter clap-rs \ - --builds Check \ +$ git clone https://github.com/rust-lang/rustc-perf ``` -You can also use that same command to use cachegrind or other profiling tools. - -#### Doing it the hard way - -If you prefer to run things manually, that is also possible. You first -need to find the source for the test you want. Sources for the tests +and then find the source code of the test that you want to profile. Sources for the tests are found in [the `collector/compile-benchmarks` directory][compile-time dir] and [the `collector/runtime-benchmarks` directory][runtime dir]. So let's go into the directory of a specific test; we'll use `clap-rs` as an example: +[rustc-perf]: https://github.com/rust-lang/rustc-perf [compile-time dir]: https://github.com/rust-lang/rustc-perf/tree/master/collector/compile-benchmarks [runtime dir]: https://github.com/rust-lang/rustc-perf/tree/master/collector/runtime-benchmarks diff --git a/src/profiling/with_rustc_perf.md b/src/profiling/with_rustc_perf.md new file mode 100644 index 000000000..42b6845ea --- /dev/null +++ b/src/profiling/with_rustc_perf.md @@ -0,0 +1,35 @@ +# Profiling with rustc-perf + +The [Rust benchmark suite][rustc-perf] provides a comprehensive way of profiling and benchmarking +the Rust compiler. You can find instructions on how to use the suite in its [manual][rustc-perf-readme]. + +However, using the suite manually can be a bit cumbersome. To make this easier for `rustc` contributors, +the compiler build system (`bootstrap`) also provides built-in integration with the benchmarking suite, +which will download and build the suite for you, build a local compiler toolchain and let you profile or +benchmark it using a simplified command-line interface. + +You can use the `./x perf -- [options]` command to use this integration. + +> Note that you need to specify arguments after `--` in the `x perf` command! You will not be able to pass arguments without the double dashes. + +You can use normal bootstrap flags for this command, such as `--stage 1` or `--stage 2`, for example to modify the stage of the created sysroot. It might also be useful to configure `config.toml` to better support profiling, e.g. set `rust.debuginfo-level = 1` to add source line information to the built compiler. + +`x perf` currently supports the following commands: +- `eprintln`: Just run the compiler and capture its `stderr` output. +- `samply`: Profile the compiler using the [samply][samply] sampling profiler. +- `cachegrind`: Use [Cachegrind][cachegrind] to generated a detailed simulated trace of the compiler's execution. + +> You can find a more detailed description of these profilers in the [`rustc-perf` manual][rustc-perf-readme-profilers]. + +You can use the following options for the `x perf` command, which mirror the corresponding options of the +`profile_local` and `bench_local` commands that you can use in the suite: + +- `--include`: Select benchmarks which should be profiled/benchmarked. +- `--profiles`: Select profiles (`Check`, `Debug`, `Opt`, `Doc`) which should be profiled/benchmarked. +- `--scenarios`: Select scenarios (`Full`, `IncrFull`, `IncrPatched`, `IncrUnchanged`) which should be profiled/benchmarked. + +[samply]: https://github.com/mstange/samply +[cachegrind]: https://www.cs.cmu.edu/afs/cs.cmu.edu/project/cmt-40/Nice/RuleRefinement/bin/valgrind-3.2.0/docs/html/cg-manual.html +[rustc-perf]: https://github.com/rust-lang/rustc-perf +[rustc-perf-readme]: https://github.com/rust-lang/rustc-perf/blob/master/collector/README.md +[rustc-perf-readme-profilers]: https://github.com/rust-lang/rustc-perf/blob/master/collector/README.md#profiling-local-builds diff --git a/src/tests/perf.md b/src/tests/perf.md index d704a2497..4581ed234 100644 --- a/src/tests/perf.md +++ b/src/tests/perf.md @@ -16,6 +16,9 @@ Different configurations include "fresh builds", builds with incremental compila The result of a perf run is a comparison between two versions of the compiler (by their commit hashes). +You can also use `rustc-perf` to manually benchmark and profile the compiler +[locally](../profiling/with_rustc_perf.md). + ### Automatic perf runs After every PR is merged, a suite of benchmarks are run against the compiler. From f28748038979eec0a9d30cd3b2249428369ba14a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Thu, 18 Jul 2024 21:53:25 +0200 Subject: [PATCH 2/3] Clarify that benchmarking is not available yet and how the `eprintln` profiler works --- src/profiling/with_rustc_perf.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/profiling/with_rustc_perf.md b/src/profiling/with_rustc_perf.md index 42b6845ea..5120f89ab 100644 --- a/src/profiling/with_rustc_perf.md +++ b/src/profiling/with_rustc_perf.md @@ -5,8 +5,10 @@ the Rust compiler. You can find instructions on how to use the suite in its [man However, using the suite manually can be a bit cumbersome. To make this easier for `rustc` contributors, the compiler build system (`bootstrap`) also provides built-in integration with the benchmarking suite, -which will download and build the suite for you, build a local compiler toolchain and let you profile or -benchmark it using a simplified command-line interface. +which will download and build the suite for you, build a local compiler toolchain and let you profile it using a +simplified command-line interface. + +> This integration currently only supports profiling, benchmarking is not integrated yet. You can use the `./x perf -- [options]` command to use this integration. @@ -15,7 +17,8 @@ You can use the `./x perf -- [options]` command to use this integratio You can use normal bootstrap flags for this command, such as `--stage 1` or `--stage 2`, for example to modify the stage of the created sysroot. It might also be useful to configure `config.toml` to better support profiling, e.g. set `rust.debuginfo-level = 1` to add source line information to the built compiler. `x perf` currently supports the following commands: -- `eprintln`: Just run the compiler and capture its `stderr` output. +- `eprintln`: Just run the compiler and capture its `stderr` output. Note that the compiler normally does not print + anything to `stderr`, you might want to add some `eprintln!` calls to get any output. - `samply`: Profile the compiler using the [samply][samply] sampling profiler. - `cachegrind`: Use [Cachegrind][cachegrind] to generated a detailed simulated trace of the compiler's execution. From ce0e15f24aa69887b95d566124866e82afba8858 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jakub=20Ber=C3=A1nek?= Date: Tue, 13 Aug 2024 18:03:03 +0200 Subject: [PATCH 3/3] Add mention of the benchmark commands --- src/profiling/with_rustc_perf.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/src/profiling/with_rustc_perf.md b/src/profiling/with_rustc_perf.md index 5120f89ab..87d205d9e 100644 --- a/src/profiling/with_rustc_perf.md +++ b/src/profiling/with_rustc_perf.md @@ -5,10 +5,7 @@ the Rust compiler. You can find instructions on how to use the suite in its [man However, using the suite manually can be a bit cumbersome. To make this easier for `rustc` contributors, the compiler build system (`bootstrap`) also provides built-in integration with the benchmarking suite, -which will download and build the suite for you, build a local compiler toolchain and let you profile it using a -simplified command-line interface. - -> This integration currently only supports profiling, benchmarking is not integrated yet. +which will download and build the suite for you, build a local compiler toolchain and let you profile it using a simplified command-line interface. You can use the `./x perf -- [options]` command to use this integration. @@ -17,12 +14,14 @@ You can use the `./x perf -- [options]` command to use this integratio You can use normal bootstrap flags for this command, such as `--stage 1` or `--stage 2`, for example to modify the stage of the created sysroot. It might also be useful to configure `config.toml` to better support profiling, e.g. set `rust.debuginfo-level = 1` to add source line information to the built compiler. `x perf` currently supports the following commands: +- `benchmark `: Benchmark the compiler and store the results under the passed `id`. +- `compare `: Compare the benchmark results of two compilers with the two passed `id`s. - `eprintln`: Just run the compiler and capture its `stderr` output. Note that the compiler normally does not print anything to `stderr`, you might want to add some `eprintln!` calls to get any output. - `samply`: Profile the compiler using the [samply][samply] sampling profiler. -- `cachegrind`: Use [Cachegrind][cachegrind] to generated a detailed simulated trace of the compiler's execution. +- `cachegrind`: Use [Cachegrind][cachegrind] to generate a detailed simulated trace of the compiler's execution. -> You can find a more detailed description of these profilers in the [`rustc-perf` manual][rustc-perf-readme-profilers]. +> You can find a more detailed description of the profilers in the [`rustc-perf` manual][rustc-perf-readme-profilers]. You can use the following options for the `x perf` command, which mirror the corresponding options of the `profile_local` and `bench_local` commands that you can use in the suite: