Skip to content

Commit

Permalink
Fix test coverage documentation (#4064)
Browse files Browse the repository at this point in the history
  • Loading branch information
daxpedda authored Aug 12, 2024
1 parent d3872f1 commit 5d45bd6
Showing 1 changed file with 38 additions and 17 deletions.
55 changes: 38 additions & 17 deletions guide/src/wasm-bindgen-test/coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,11 @@ Currently it is particularly difficult to [deliver compile-line arguments to pro

Make sure you are using `RUSTFLAGS=-Cinstrument-coverage -Zno-profiler-runtime`.

Due to the current limitation of `llvm-cov`, we can't collect profiling symbols from the generated `.wasm` files. Instead, we can grab them from the LLVM IR with `--emit=llvm-ir` by using Clang. Additionally, the emitted LLVM IR files by Rust contain invalid code that can't be parsed by Clang, so they need to be adjusted. Clang must use the same LLVM version that Rustc is using, which can be checkd by calling `rustc +nightly -vV`.
Due to the current limitation of `llvm-cov`, we can't collect profiling symbols from the generated `.wasm` files. Instead, we can grab them from the LLVM IR with `--emit=llvm-ir` by using Clang. Additionally, the emitted LLVM IR files by Rust contain invalid code that can't be parsed by Clang, so they need to be adjusted.

At the time of writing Rust Nightly uses LLVM v19, however [minicov] only supports LLVM v18. Usage of Clang or any LLVM tools must match the version used by [minicov].

[minicov]: https://crates.io/crates/minicov

### Arguments to the test runner

Expand All @@ -43,6 +47,8 @@ This feature relies on the [minicov] crate, which provides a profiling runtime f

### Example

This adapts code taken from the [Rustc book], see that for more examples and general information on test coverage as well.

```sh
# Run the tests:
# - `CARGO_HOST_RUSTFLAGS` to pass the configuration to `wasm-bindgen-macro`.
Expand All @@ -53,26 +59,41 @@ RUSTFLAGS="-Cinstrument-coverage -Zno-profiler-runtime --emit=llvm-ir --cfg=wasm
CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_RUNNER=wasm-bindgen-test-runner \
cargo +nightly test -Ztarget-applies-to-host -Zhost-config --tests
# Adjust the LLVM IR and compile to object files:
# - Replaces every function body with `unreachable`.
# - Removes Rust-specific `range` annotations from function signatures.
name=name_of_the_tested_crate_in_snake_case; \
for file in `ls target/wasm32-unknown-unknown/debug/deps/$name-*.ll`; \
do \
perl -i -p0e 's/(^define.*?$).*?^}/$1\nstart:\n unreachable\n}/gms' $file && \
perl -i -p0e 's/(?<=noundef) range\(.*?\)//g' $file && \
clang $file -Wno-override-module -c; \
# - Extract a list of compiled artifacts from Cargo and filter them with `jq`.
# - Figure out the path to the LLVM IR file corresponding to an artifact.
# - Replace every function body with `unreachable`.
# - Remove Rust-specific `range` annotations from function signatures.
# - Compile to object file with Clang and store for later usage with `llvm-cov`.
crate_name=name_of_the_tested_crate_in_snake_case
objects=()
IFS=$'\n'
for file in $(
CARGO_HOST_RUSTFLAGS=--cfg=wasm_bindgen_unstable_test_coverage \
RUSTFLAGS="-Cinstrument-coverage -Zno-profiler-runtime --emit=llvm-ir --cfg=wasm_bindgen_unstable_test_coverage" \
cargo +nightly test -Ztarget-applies-to-host -Zhost-config --tests --no-run --message-format=json | \
jq -r "select(.reason == \"compiler-artifact\") | (select(.target.kind == [\"test\"]) // select(.target.name == \"$crate_name\")) | .filenames[0]"
)
do
if [[ ${file##*.} == "rlib" ]]; then
base=$(basename $file .rlib)
file=$(dirname $file)/${base#"lib"}.ll
else
file=$(dirname $file)/$(basename $file .wasm).ll
fi

perl -i -p0e 's/(^define.*?$).*?^}/$1\nstart:\n unreachable\n}/gms' $file
perl -i -p0e 's/(^define( [^ ]+)*) range\(.*?\)/$1/gm' $file

output = $(basename $file .ll).o
clang-18 $input -Wno-override-module -c -o $output
objects+=(-object $output)
done
# Merge all generated raw profiling data.
# This uses `cargo-binutils` which uses LLVM tools shipped by Rust to make sure there is no LLVM version discrepancy.
# But `llvm-profdata` can be used directly as well.
# See <https://crates.io/crates/cargo-binutils>.
rust-profdata merge -sparse ./*.profraw -o coverage.profdata
# Produce test coverage data in the HTML format.
rust-cov show --instr-profile=coverage.profdata --object ./*.o --format=html --Xdemangler=rust-demangler --sources src --output-dir coverage
llvm-profdata-18 merge -sparse *.profraw -o coverage.profdata
# Produce test coverage data in the HTML format and pass the object files we generated earlier.
llvm-cov-18 show -show-instantiations=false -Xdemangler=rustfilt -output-dir coverage -format=html -instr-profile=coverage.profdata ${objects[@]} -sources src
```

The [rustc book] has a lot more exapmles and information on test coverage as well.

[rustc book]: https://doc.rust-lang.org/nightly/rustc/instrument-coverage.html

## Attribution
Expand Down

0 comments on commit 5d45bd6

Please sign in to comment.