From 0097f8c6dc3b94669db25ca8b515589df96b6fdb Mon Sep 17 00:00:00 2001 From: Steve Klabnik Date: Fri, 7 Oct 2016 14:05:34 -0400 Subject: [PATCH 01/18] set up --- src/SUMMARY.md | 10 ++++++---- src/chXX-00-testing.md | 1 + src/chXX-01-testing.md | 1 + src/chXX-02-integration-testing.md | 1 + src/chXX-03-documentation-tests.md | 1 + 5 files changed, 10 insertions(+), 4 deletions(-) create mode 100644 src/chXX-00-testing.md create mode 100644 src/chXX-01-testing.md create mode 100644 src/chXX-02-integration-testing.md create mode 100644 src/chXX-03-documentation-tests.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 8d4def8ead..c61ec0d05c 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -56,10 +56,12 @@ - [`std::path`]() - [`std::env`]() -- [Testing]() - - [Unit Tests]() - - [Integration Tests]() - - [Test Attribute]() +- [Testing](chXX-00-testing.md) + - [Unit testing](chXX-01-testing.md) + - [Integration testing](chXX-02-integration-testing.md) + - [Documentation Tests](chXX-03-documentation-tests.md) + +- [Debugging]() ## Thinking in Rust diff --git a/src/chXX-00-testing.md b/src/chXX-00-testing.md new file mode 100644 index 0000000000..f00b526a98 --- /dev/null +++ b/src/chXX-00-testing.md @@ -0,0 +1 @@ +# Testing diff --git a/src/chXX-01-testing.md b/src/chXX-01-testing.md new file mode 100644 index 0000000000..4ca7244589 --- /dev/null +++ b/src/chXX-01-testing.md @@ -0,0 +1 @@ +# Unit testing diff --git a/src/chXX-02-integration-testing.md b/src/chXX-02-integration-testing.md new file mode 100644 index 0000000000..39fe096416 --- /dev/null +++ b/src/chXX-02-integration-testing.md @@ -0,0 +1 @@ +# Integration testing diff --git a/src/chXX-03-documentation-tests.md b/src/chXX-03-documentation-tests.md new file mode 100644 index 0000000000..ead2b6059f --- /dev/null +++ b/src/chXX-03-documentation-tests.md @@ -0,0 +1 @@ +# Documentation Tests From a85af065beebc7acf8ccf970360cab6b02734f2b Mon Sep 17 00:00:00 2001 From: Steve Klabnik Date: Fri, 7 Oct 2016 15:16:04 -0400 Subject: [PATCH 02/18] initial import/re-write of testing chapter --- src/chXX-00-testing.md | 279 ++++++++++++++++++++++++++ src/chXX-01-testing.md | 130 +++++++++++++ src/chXX-02-integration-testing.md | 162 ++++++++++++++++ src/chXX-03-documentation-tests.md | 301 +++++++++++++++++++++++++++++ 4 files changed, 872 insertions(+) diff --git a/src/chXX-00-testing.md b/src/chXX-00-testing.md index f00b526a98..aa3a639d1a 100644 --- a/src/chXX-00-testing.md +++ b/src/chXX-00-testing.md @@ -1 +1,280 @@ # Testing + +> Program testing can be a very effective way to show the presence of bugs, but +> it is hopelessly inadequate for showing their absence. +> +> Edsger W. Dijkstra, "The Humble Programmer" (1972) + +Rust is a programming language that cares a lot about correctness. But +correctness is a complex topic, and isn't exactly easy to get right. Rust +places a lot of weight on its type system to help ensure that our programs do +what we intend, but it cannot help with everything. As such, Rust also includes +support for writing software tests in the language itself. + +Testing is a skill, and we cannot hope to learn everything about how to write +good tests in one chapter of a book. What we can learn, however, are the +mechanics of Rust's testing facilities. That's what we'll focus on in this +chapter. + +## The `test` attribute + +At its simplest, a test in Rust is a function that's annotated with the `test` +attribute. Let's make a new project with Cargo called `adder`: + +```bash +$ cargo new adder + Created library `adder` project +$ cd adder +``` + +Cargo will automatically generate a simple test when you make a new project. +Here's the contents of `src/lib.rs`: + +```rust +#[cfg(test)] +mod tests { + #[test] + fn it_works() { + } +} +``` + +For now, let's remove the `mod` bit, and focus on just the function: + +```rust +#[test] +fn it_works() { +} +``` + +Note the `#[test]`. This attribute indicates that this is a test function. It +currently has no body. That's good enough to pass! We can run the tests with +`cargo test`: + +```bash +$ cargo test + Compiling adder v0.1.0 (file:///projects/adder) + Finished debug [unoptimized + debuginfo] target(s) in 0.22 secs + Running target/debug/deps/adder-ce99bcc2479f4607 + +running 1 test +test it_works ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Doc-tests adder + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` + +Cargo compiled and ran our tests. There are two sets of output here: one +for the test we wrote, and another for documentation tests. We'll talk about +documentation tests later. For now, see this line: + +```text +test it_works ... ok +``` + +Note the `it_works`. This comes from the name of our function: + +```rust +fn it_works() { +# } +``` + +We also get a summary line: + +```text +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured +``` + +## The `assert!` macro + +So why does our do-nothing test pass? Any test which doesn't `panic!` passes, +and any test that does `panic!` fails. Let's make our test fail: + +```rust +#[test] +fn it_works() { + assert!(false); +} +``` + +`assert!` is a macro provided by Rust which takes one argument: if the argument +is `true`, nothing happens. If the argument is `false`, it will `panic!`. Let's +run our tests again: + +```bash +$ cargo test + Compiling adder v0.1.0 (file:///projects/adder) + Finished debug [unoptimized + debuginfo] target(s) in 0.22 secs + Running target/debug/deps/adder-ce99bcc2479f4607 + +running 1 test +test it_works ... FAILED + +failures: + +---- it_works stdout ---- + thread 'it_works' panicked at 'assertion failed: false', src/lib.rs:5 +note: Run with `RUST_BACKTRACE=1` for a backtrace. + + +failures: + it_works + +test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured + +error: test failed +``` + +Rust indicates that our test failed: + +```text +test it_works ... FAILED +``` + +And that's reflected in the summary line: + +```text +test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured +``` + +## Inverting failure with `should_panic` + +We can invert our test's failure with another attribute: `should_panic`: + +```rust +#[test] +#[should_panic] +fn it_works() { + assert!(false); +} +``` + +This test will now succeed if we `panic!` and fail if we complete. + +`should_panic` tests can be fragile, as it's hard to guarantee that the test +didn't fail for an unexpected reason. To help with this, an optional `expected` +parameter can be added to the `should_panic` attribute. The test harness will +make sure that the failure message contains the provided text. A safer version +of the example above would be: + +```rust +#[test] +#[should_panic(expected = "assertion failed")] +fn it_works() { + assert!(false); +} +``` + +## Testing equality + +Rust provides a pair of macros, `assert_eq!` and `assert_ne!`, that compares +two arguments for equality: + +```rust +#[test] +fn it_works() { + assert_eq!("Hello", "Hello"); + + assert_ne!("Hello", "world"); +} +``` + +These macros expand to something like this: + +```rust,ignore +// assert_eq +if left_val == right_val { + panic!("message goes here") +} + +// assert_ne +if left_val =! right_val { + panic!("message goes here") +} +``` + +But they're a bit more convenient than writing this out by hand. These macros +are often used to call some function with some known arguments and compare it +to the expected output, like this: + +```rust +pub fn add_two(a: i32) -> i32 { + a + 2 +} + +#[test] +fn it_works() { + assert_eq!(4, add_two(2)); +} +``` + +## The `ignore` attribute + +Sometimes a few specific tests can be very time-consuming to execute. These +can be disabled by default by using the `ignore` attribute: + +```rust +pub fn add_two(a: i32) -> i32 { + a + 2 +} + +#[test] +fn it_works() { + assert_eq!(4, add_two(2)); +} + +#[test] +#[ignore] +fn expensive_test() { + // code that takes an hour to run +} +``` + +Now we run our tests and see that `it_works` is run, but `expensive_test` is +not: + +```bash +$ cargo test + Compiling adder v0.1.0 (file:///projects/adder) + Finished debug [unoptimized + debuginfo] target(s) in 0.24 secs + Running target/debug/deps/adder-ce99bcc2479f4607 + +running 2 tests +test expensive_test ... ignored +test it_works ... ok + +test result: ok. 1 passed; 0 failed; 1 ignored; 0 measured + + Doc-tests adder + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` + +The expensive tests can be run explicitly using `cargo test -- --ignored`: + +```bash +$ cargo test -- --ignored + Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs + Running target/debug/deps/adder-ce99bcc2479f4607 + +running 1 test +test expensive_test ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Doc-tests adder + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` + +The `--ignored` argument is an argument to the test binary, and not to Cargo, +which is why the command is `cargo test -- --ignored`. diff --git a/src/chXX-01-testing.md b/src/chXX-01-testing.md index 4ca7244589..19bb7a2128 100644 --- a/src/chXX-01-testing.md +++ b/src/chXX-01-testing.md @@ -1 +1,131 @@ # Unit testing + +As we mentioned before, testing is a large discipline, and so different people +can sometimes use different terminology. For our purposes, we tend to place +tests into two main categories: *unit tests* and *integration tests*. Unit +tests tend to be smaller, and more focused. In Rust, they can also test +non-public interfaces. Let's talk more about how to do unit testing in Rust. + +## The tests module and `cfg(test)` + +Remember when we generated our new project in the last section? Cargo had +generated some stuff for us: + +```rust +#[cfg(test)] +mod tests { + #[test] + fn it_works() { + } +} +``` + +We deleted the module stuff so we could learn more about the mechanics of +tests. But there's a reason that Cargo generated this module for us: it's the +idiomatic way to organize unit tests in Rust. That is, unit tests are: + +* Stored inside of the same tree as your source code. +* Placed inside their own module. + +For a more realistic example of how this works, consider our `add_two` function +from before: + +```rust +pub fn add_two(a: i32) -> i32 { + a + 2 +} + +#[cfg(test)] +mod tests { + use add_two; + + #[test] + fn it_works() { + assert_eq!(4, add_two(2)); + } +} +``` + +First of all, there's a new attribute, `cfg`. The `cfg` attribute lets us +declare that something should only be included given a certain configuration. +Rust provides the `test` configuration when compiling and running tests. By +using this attribute, Cargo only compiles our test code if we're currently +trying to run the tests. Given that they're not compiled at all during a +regular `cargo build`, this can save compile time. It also ensures that our +tests are entirely left out of the binary, saving space in a non-testing +context. + +You'll notice one more change: the `use` declaration. The `tests` module is +only a convention, it's nothing that Rust understands directly. As such, we +have to follow the usual visibility rules. Because we're in an inner module, +we need to bring our test function into scope. This can be annoying if you have +a large module, and so this is a common use of globs. Let's change our +`src/lib.rs` to make use of it: + +```rust,ignore +pub fn add_two(a: i32) -> i32 { + a + 2 +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn it_works() { + assert_eq!(4, add_two(2)); + } +} +``` + +Note the different `use` line. Now we run our tests: + +```bash +$ cargo test + Compiling adder v0.1.0 (file:///projects/adder) + Finished debug [unoptimized + debuginfo] target(s) in 0.27 secs + Running target/debug/deps/adder-ce99bcc2479f4607 + +running 1 test +test tests::it_works ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Doc-tests adder + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` + +It works! + +## Testing internal functions + +There's controversy within the testing community about unit testing private +functions. Regardless of which testing ideology you adhere to, Rust does allow +you to test them, due to the way that the privacy rules work. Consider this: + +```rust +pub fn add_two(a: i32) -> i32 { + internal_adder(a, 2) +} + +fn internal_adder(a: i32, b: i32) -> i32 { + a + b +} + +#[cfg(test)] +mod tests { + use internal_adder; + + #[test] + fn internal() { + assert_eq!(4, internal_adder(2, 2)); + } +} +``` + +In this scenario, we have a non-`pub` function, `internal_adder`. Because tests +are just Rust code, and the `tests` module is just another module, we can +import and call `internal_adder` in a test just fine. diff --git a/src/chXX-02-integration-testing.md b/src/chXX-02-integration-testing.md index 39fe096416..2d642bf88c 100644 --- a/src/chXX-02-integration-testing.md +++ b/src/chXX-02-integration-testing.md @@ -1 +1,163 @@ # Integration testing + +In the last section, we talked about unit tests. But there's still that other +category: integration testing. In Rust, an integration test is a test that is +entirely external to your library. It uses it in the same way any other code +would. + +Cargo has support for integration tests through the `tests` directory. If you +make one, and put `.rs` files inside, Cargo will compile each of them as an +individual crate. Let's give it a try! First, make a `tests` directory at the +top level of your project, next to `src`. Then, make a new file, +`tests/integration_test.rs`, and put this inside: + +```rust,ignore +extern crate adder; + +#[test] +fn it_works() { + assert_eq!(4, adder::add_two(2)); +} +``` + +There's some small changes from our previous tests. We now have an `extern +crate adder` at the top. This is because each test in the `tests` directory is +an entirely separate crate, and so we need to import our library. This is also +why `tests` is a suitable place to write integration-style tests: they use the +library like any other consumer of it would. + +Let's run them: + +```bash +$ cargo test + Compiling adder v0.1.0 (file:///home/steve/tmp/adder) + Running target/adder-91b3e234d4ed382a + +running 1 test +test tests::it_works ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Running target/lib-c18e7d3494509e74 + +running 1 test +test it_works ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Doc-tests adder + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` + +Now we have three sections: our previous test is also run, as well as our new +one. + +That's all there is to the `tests` directory. The `tests` module isn't needed +here, since the whole thing is focused on tests. + +## Submodules in integration tests + +As your integration tests grow, you may want to make more than one file in the +`tests` directory. As we mentioned before, that works well, given that Cargo +treats every file as its own crate. But there's one small trap that can happen. + +Imagine we wanted some common helper functions to be shared across our tests. +So we change our test to have a `common` module: + +```rust,ignore +extern crate adder; + +mod common; + +#[test] +fn it_works() { + common::helper(); + + assert_eq!(4, adder::add_two(2)); +} +``` + +And then, we create a `tests/common.rs` file to hold our common helpers: + +```rust +pub fn helper() { + // no implementation for now +} +``` + +Let's try running this: + +```bash +$ cargo test + Compiling adder v0.1.0 (file:///home/steve/tmp/adder) + Finished debug [unoptimized + debuginfo] target(s) in 0.25 secs + Running target/debug/deps/adder-ce99bcc2479f4607 + +running 1 test +test tests::internal ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Running target/debug/common-c3635c69f3aeef92 + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured + + Running target/debug/integration_tests-6d6e12b4680b0368 + +running 1 test +test it_works ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Doc-tests adder + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` + +Wait a minute. Now we have four sections? + +```text + Running target/debug/common-c3635c69f3aeef92 +``` + +Because `common.rs` is in our `tests` directory, Cargo is also compiling it as +its own crate. Because `common.rs` is so simple, we didn't get an error, but +with more complex code, this might not work. So what can we do? + +The key is, always use the `common/mod.rs` form over the `common.rs` form when +making modules in integration tests. If we move `tests/common.rs` to +`tests/common/mod.rs`, we'll go back to our expected output: + +```bash +$ mkdir tests/common +$ mv tests/common.rs tests/common/mod.rs +$ cargo test + Compiling adder v0.1.0 (file:///home/steve/tmp/adder) + Finished debug [unoptimized + debuginfo] target(s) in 0.24 secs + Running target/debug/deps/adder-ce99bcc2479f4607 + +running 1 test +test tests::internal ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Running target/debug/integration_tests-6d6e12b4680b0368 + +running 1 test +test it_works ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Doc-tests adder + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` diff --git a/src/chXX-03-documentation-tests.md b/src/chXX-03-documentation-tests.md index ead2b6059f..c78b708e9c 100644 --- a/src/chXX-03-documentation-tests.md +++ b/src/chXX-03-documentation-tests.md @@ -1 +1,302 @@ # Documentation Tests + +Nothing is better than documentation with examples. Nothing is worse than +examples that don't actually work, because the code has changed since the +documentation has been written. To this end, Rust supports automatically +running examples in your documentation for library crates. Here's a fleshed-out +`src/lib.rs` with examples: + +```rust +//! The `adder` crate provides functions that add numbers to other numbers. +//! +//! # Examples +//! +//! ``` +//! assert_eq!(4, adder::add_two(2)); +//! ``` + +/// This function adds two to its argument. +/// +/// # Examples +/// +/// ``` +/// use adder::add_two; +/// +/// assert_eq!(4, add_two(2)); +/// ``` +pub fn add_two(a: i32) -> i32 { + a + 2 +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn it_works() { + assert_eq!(4, add_two(2)); + } +} +``` + +Note the module-level documentation with `//!` and the function-level +documentation with `///`. Rust's documentation supports Markdown in comments, +and so triple graves mark code blocks. It is conventional to include the +`# Examples` section, exactly like that, with examples following. + +Let's run the tests again: + +```bash +$ cargo test + Compiling adder v0.1.0 (file:///projects/adder) + Running target/adder-91b3e234d4ed382a + +running 1 test +test tests::it_works ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Running target/lib-c18e7d3494509e74 + +running 1 test +test it_works ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Doc-tests adder + +running 2 tests +test add_two_0 ... ok +test _0 ... ok + +test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured +``` + +Now we have all three kinds of tests running! Note the names of the +documentation tests: the `_0` is generated for the module test, and `add_two_0` +for the function test. These will auto increment with names like `add_two_1` as +you add more examples. + +## Automatic `main` insertion + +Let's discuss our sample example documentation: + +```rust +/// ``` +/// println!("Hello, world"); +/// ``` +# fn foo() {} +``` + +You'll notice that you don't need a `fn main()` or anything here. `rustdoc` +will automatically add a `main()` wrapper around your code, using heuristics to +attempt to put it in the right place. For example: + +```rust +/// ``` +/// use std::rc::Rc; +/// +/// let five = Rc::new(5); +/// ``` +# fn foo() {} +``` + +This will end up testing: + +```rust +fn main() { + use std::rc::Rc; + let five = Rc::new(5); +} +``` + +Here's the full algorithm rustdoc uses to preprocess examples: + +1. Any leading `#![foo]` attributes are left intact as crate attributes. +2. Some common `allow` attributes are inserted, including + `unused_variables`, `unused_assignments`, `unused_mut`, + `unused_attributes`, and `dead_code`. Small examples often trigger + these lints. +3. If the example does not contain `extern crate`, then `extern crate + ;` is inserted (note the lack of `#[macro_use]`). +4. Finally, if the example does not contain `fn main`, the remainder of the + text is wrapped in `fn main() { your_code }`. + +This generated `fn main` can be a problem! If you have `extern crate` or a +`mod` statements in the example code that are referred to by `use` statements, +they will fail to resolve unless you include at least `fn main() {}` to inhibit +step 4. `#[macro_use] extern crate` also does not work except at the crate +root, so when testing macros an explicit `main` is always required. It doesn't +have to clutter up your docs, though — keep reading! + +## Hiding extraneous code with `#` + +Sometimes this algorithm isn't enough, though. For example, all of these code +samples with `///` we've been talking about? The raw text: + +```text +/// Some documentation. +# fn foo() {} +``` + +looks different than the output: + +```rust +/// Some documentation. +# fn foo() {} +``` + +Yes, that's right: we can add lines that start with `# `, and they will be +hidden from the output, but will be used when compiling our code. We can use +this to our advantage. In this case, documentation comments need to apply to +some kind of function, so if We want to show off a documentation comment, I +need to add a little function definition below it. At the same time, it's only +there to satisfy the compiler, so hiding it makes the example more clear. We +can use this technique to explain longer examples in detail, while still +preserving the testability of our documentation. + +For example, imagine that we wanted to document this code: + +```rust +let x = 5; +let y = 6; +println!("{}", x + y); +``` + +We might want the documentation to end up looking like this: + +> First, we set `x` to five: +> +> ```rust +> let x = 5; +> # let y = 6; +> # println!("{}", x + y); +> ``` +> +> Next, we set `y` to six: +> +> ```rust +> # let x = 5; +> let y = 6; +> # println!("{}", x + y); +> ``` +> +> Finally, we print the sum of `x` and `y`: +> +> ```rust +> # let x = 5; +> # let y = 6; +> println!("{}", x + y); +> ``` + +To keep each code block testable, we want the whole program in each block, but +we don't want the reader to see every line every time. Here's what we put in +our source code: + +```text + First, we set `x` to five: + + ```rust + let x = 5; + # let y = 6; + # println!("{}", x + y); + ``` + + Next, we set `y` to six: + + ```rust + # let x = 5; + let y = 6; + # println!("{}", x + y); + ``` + + Finally, we print the sum of `x` and `y`: + + ```rust + # let x = 5; + # let y = 6; + println!("{}", x + y); + ``` +``` + +By repeating all parts of the example, we can ensure that our example still +compiles, while only showing the parts that are relevant to that part of our +explanation. + +Another case where the use of `#` is handy is when you want to ignore +error handling. Lets say you want the following, + +```rust,ignore +/// use std::io; +/// let mut input = String::new(); +/// try!(io::stdin().read_line(&mut input)); +``` + +The problem is that `try!` returns a `Result` and test functions +don't return anything so this will give a mismatched types error. + +```rust,ignore +/// A doc test using try! +/// +/// ``` +/// use std::io; +/// # fn foo() -> io::Result<()> { +/// let mut input = String::new(); +/// try!(io::stdin().read_line(&mut input)); +/// # Ok(()) +/// # } +/// ``` +# fn foo() {} +``` + +You can get around this by wrapping the code in a function. This catches +and swallows the `Result` when running tests on the docs. This +pattern appears regularly in the standard library. + +## Adding attributes to control documentation testing. + +In the first part of the chapter, we talked about attributes that help with +testing: + +```rust +#[test] +#[ignore] +fn it_works() { +} + +#[should_panic] +fn it_works() { + assert!(false); +} +``` + +We can use these annotations in documentation tests as well: + +```rust +/// ```rust,ignore +/// fn foo() { +/// ``` +fn foo() {} + +/// ```rust,should_panic +/// assert!(false); +/// ``` +fn bar() {} +``` + +## The `no_run` attribute + +There's one attribute that's specific to documentation tests: + +```rust +/// ```rust,no_run +/// loop { +/// println!("Hello, world"); +/// } +/// ``` +# fn foo() {} +``` + +The `no_run` attribute will compile your code, but not run it. This is +important for examples such as "Here's how to start up a network service," +which you would want to make sure compile, but might run in an infinite loop! From 62bfe3ad4ccd689e6c812fbf54d745293948934a Mon Sep 17 00:00:00 2001 From: Steve Klabnik Date: Tue, 8 Nov 2016 16:00:45 -0500 Subject: [PATCH 03/18] rename to unit testing --- src/SUMMARY.md | 2 +- src/{chXX-01-testing.md => chXX-01-unit-testing.md} | 0 2 files changed, 1 insertion(+), 1 deletion(-) rename src/{chXX-01-testing.md => chXX-01-unit-testing.md} (100%) diff --git a/src/SUMMARY.md b/src/SUMMARY.md index c61ec0d05c..746c9c5d93 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -57,7 +57,7 @@ - [`std::env`]() - [Testing](chXX-00-testing.md) - - [Unit testing](chXX-01-testing.md) + - [Unit testing](chXX-01-unit-testing.md) - [Integration testing](chXX-02-integration-testing.md) - [Documentation Tests](chXX-03-documentation-tests.md) diff --git a/src/chXX-01-testing.md b/src/chXX-01-unit-testing.md similarity index 100% rename from src/chXX-01-testing.md rename to src/chXX-01-unit-testing.md From 5428b7f5ba4e35cc904e230b1c0565895d41be30 Mon Sep 17 00:00:00 2001 From: Steve Klabnik Date: Tue, 8 Nov 2016 16:07:20 -0500 Subject: [PATCH 04/18] add error message for assert_eq --- src/chXX-00-testing.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/chXX-00-testing.md b/src/chXX-00-testing.md index aa3a639d1a..b44780155c 100644 --- a/src/chXX-00-testing.md +++ b/src/chXX-00-testing.md @@ -198,6 +198,16 @@ if left_val =! right_val { } ``` +And if the test fails, we get an error that looks like this: + +```text +---- failing_test stdout ---- + thread 'failing_test' panicked at 'assertion failed: `(left == right)` (left: `"hello"`, right: `"world"`)', failing_test.rs:4 +``` + +One other thing to notice is that they're named "left" and "right" rather than +"expected" vs "actual"; the order isn't particularly important. + But they're a bit more convenient than writing this out by hand. These macros are often used to call some function with some known arguments and compare it to the expected output, like this: From aefa195dd9867a11c534e810a6d02e062bfd4798 Mon Sep 17 00:00:00 2001 From: Steve Klabnik Date: Tue, 8 Nov 2016 16:07:48 -0500 Subject: [PATCH 05/18] Remove 'documentation tests' bit that goes in the 'creating a library' section --- src/SUMMARY.md | 1 - src/chXX-03-documentation-tests.md | 302 ----------------------------- 2 files changed, 303 deletions(-) delete mode 100644 src/chXX-03-documentation-tests.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 746c9c5d93..eeddf89657 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -59,7 +59,6 @@ - [Testing](chXX-00-testing.md) - [Unit testing](chXX-01-unit-testing.md) - [Integration testing](chXX-02-integration-testing.md) - - [Documentation Tests](chXX-03-documentation-tests.md) - [Debugging]() diff --git a/src/chXX-03-documentation-tests.md b/src/chXX-03-documentation-tests.md deleted file mode 100644 index c78b708e9c..0000000000 --- a/src/chXX-03-documentation-tests.md +++ /dev/null @@ -1,302 +0,0 @@ -# Documentation Tests - -Nothing is better than documentation with examples. Nothing is worse than -examples that don't actually work, because the code has changed since the -documentation has been written. To this end, Rust supports automatically -running examples in your documentation for library crates. Here's a fleshed-out -`src/lib.rs` with examples: - -```rust -//! The `adder` crate provides functions that add numbers to other numbers. -//! -//! # Examples -//! -//! ``` -//! assert_eq!(4, adder::add_two(2)); -//! ``` - -/// This function adds two to its argument. -/// -/// # Examples -/// -/// ``` -/// use adder::add_two; -/// -/// assert_eq!(4, add_two(2)); -/// ``` -pub fn add_two(a: i32) -> i32 { - a + 2 -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn it_works() { - assert_eq!(4, add_two(2)); - } -} -``` - -Note the module-level documentation with `//!` and the function-level -documentation with `///`. Rust's documentation supports Markdown in comments, -and so triple graves mark code blocks. It is conventional to include the -`# Examples` section, exactly like that, with examples following. - -Let's run the tests again: - -```bash -$ cargo test - Compiling adder v0.1.0 (file:///projects/adder) - Running target/adder-91b3e234d4ed382a - -running 1 test -test tests::it_works ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Running target/lib-c18e7d3494509e74 - -running 1 test -test it_works ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Doc-tests adder - -running 2 tests -test add_two_0 ... ok -test _0 ... ok - -test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured -``` - -Now we have all three kinds of tests running! Note the names of the -documentation tests: the `_0` is generated for the module test, and `add_two_0` -for the function test. These will auto increment with names like `add_two_1` as -you add more examples. - -## Automatic `main` insertion - -Let's discuss our sample example documentation: - -```rust -/// ``` -/// println!("Hello, world"); -/// ``` -# fn foo() {} -``` - -You'll notice that you don't need a `fn main()` or anything here. `rustdoc` -will automatically add a `main()` wrapper around your code, using heuristics to -attempt to put it in the right place. For example: - -```rust -/// ``` -/// use std::rc::Rc; -/// -/// let five = Rc::new(5); -/// ``` -# fn foo() {} -``` - -This will end up testing: - -```rust -fn main() { - use std::rc::Rc; - let five = Rc::new(5); -} -``` - -Here's the full algorithm rustdoc uses to preprocess examples: - -1. Any leading `#![foo]` attributes are left intact as crate attributes. -2. Some common `allow` attributes are inserted, including - `unused_variables`, `unused_assignments`, `unused_mut`, - `unused_attributes`, and `dead_code`. Small examples often trigger - these lints. -3. If the example does not contain `extern crate`, then `extern crate - ;` is inserted (note the lack of `#[macro_use]`). -4. Finally, if the example does not contain `fn main`, the remainder of the - text is wrapped in `fn main() { your_code }`. - -This generated `fn main` can be a problem! If you have `extern crate` or a -`mod` statements in the example code that are referred to by `use` statements, -they will fail to resolve unless you include at least `fn main() {}` to inhibit -step 4. `#[macro_use] extern crate` also does not work except at the crate -root, so when testing macros an explicit `main` is always required. It doesn't -have to clutter up your docs, though — keep reading! - -## Hiding extraneous code with `#` - -Sometimes this algorithm isn't enough, though. For example, all of these code -samples with `///` we've been talking about? The raw text: - -```text -/// Some documentation. -# fn foo() {} -``` - -looks different than the output: - -```rust -/// Some documentation. -# fn foo() {} -``` - -Yes, that's right: we can add lines that start with `# `, and they will be -hidden from the output, but will be used when compiling our code. We can use -this to our advantage. In this case, documentation comments need to apply to -some kind of function, so if We want to show off a documentation comment, I -need to add a little function definition below it. At the same time, it's only -there to satisfy the compiler, so hiding it makes the example more clear. We -can use this technique to explain longer examples in detail, while still -preserving the testability of our documentation. - -For example, imagine that we wanted to document this code: - -```rust -let x = 5; -let y = 6; -println!("{}", x + y); -``` - -We might want the documentation to end up looking like this: - -> First, we set `x` to five: -> -> ```rust -> let x = 5; -> # let y = 6; -> # println!("{}", x + y); -> ``` -> -> Next, we set `y` to six: -> -> ```rust -> # let x = 5; -> let y = 6; -> # println!("{}", x + y); -> ``` -> -> Finally, we print the sum of `x` and `y`: -> -> ```rust -> # let x = 5; -> # let y = 6; -> println!("{}", x + y); -> ``` - -To keep each code block testable, we want the whole program in each block, but -we don't want the reader to see every line every time. Here's what we put in -our source code: - -```text - First, we set `x` to five: - - ```rust - let x = 5; - # let y = 6; - # println!("{}", x + y); - ``` - - Next, we set `y` to six: - - ```rust - # let x = 5; - let y = 6; - # println!("{}", x + y); - ``` - - Finally, we print the sum of `x` and `y`: - - ```rust - # let x = 5; - # let y = 6; - println!("{}", x + y); - ``` -``` - -By repeating all parts of the example, we can ensure that our example still -compiles, while only showing the parts that are relevant to that part of our -explanation. - -Another case where the use of `#` is handy is when you want to ignore -error handling. Lets say you want the following, - -```rust,ignore -/// use std::io; -/// let mut input = String::new(); -/// try!(io::stdin().read_line(&mut input)); -``` - -The problem is that `try!` returns a `Result` and test functions -don't return anything so this will give a mismatched types error. - -```rust,ignore -/// A doc test using try! -/// -/// ``` -/// use std::io; -/// # fn foo() -> io::Result<()> { -/// let mut input = String::new(); -/// try!(io::stdin().read_line(&mut input)); -/// # Ok(()) -/// # } -/// ``` -# fn foo() {} -``` - -You can get around this by wrapping the code in a function. This catches -and swallows the `Result` when running tests on the docs. This -pattern appears regularly in the standard library. - -## Adding attributes to control documentation testing. - -In the first part of the chapter, we talked about attributes that help with -testing: - -```rust -#[test] -#[ignore] -fn it_works() { -} - -#[should_panic] -fn it_works() { - assert!(false); -} -``` - -We can use these annotations in documentation tests as well: - -```rust -/// ```rust,ignore -/// fn foo() { -/// ``` -fn foo() {} - -/// ```rust,should_panic -/// assert!(false); -/// ``` -fn bar() {} -``` - -## The `no_run` attribute - -There's one attribute that's specific to documentation tests: - -```rust -/// ```rust,no_run -/// loop { -/// println!("Hello, world"); -/// } -/// ``` -# fn foo() {} -``` - -The `no_run` attribute will compile your code, but not run it. This is -important for examples such as "Here's how to start up a network service," -which you would want to make sure compile, but might run in an infinite loop! From a49d644aadcba1166966adb74491fbc7d2558f3b Mon Sep 17 00:00:00 2001 From: Steve Klabnik Date: Tue, 8 Nov 2016 16:16:38 -0500 Subject: [PATCH 06/18] add stuff about running subsets of tests --- src/chXX-01-unit-testing.md | 94 +++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) diff --git a/src/chXX-01-unit-testing.md b/src/chXX-01-unit-testing.md index 19bb7a2128..db58a7f69d 100644 --- a/src/chXX-01-unit-testing.md +++ b/src/chXX-01-unit-testing.md @@ -129,3 +129,97 @@ mod tests { In this scenario, we have a non-`pub` function, `internal_adder`. Because tests are just Rust code, and the `tests` module is just another module, we can import and call `internal_adder` in a test just fine. + +## Running a subset of tests + +Sometimes, running a full test suite can take a long time. `cargo test` takes +an argument that allows you to only run certain tests, if you'd prefer to do that. +Let's say we had two tests of `add_two`: + +```rust +pub fn add_two(a: i32) -> i32 { + a + 2 +} + +#[cfg(test)] +mod tests { + use add_two; + + #[test] + fn add_two_and_two() { + assert_eq!(4, add_two(2)); + } + + #[test] + fn add_three_and_two() { + assert_eq!(5, add_two(3)); + } + + #[test] + fn one_hundred() { + assert_eq!(102, add_two(100)); + } +} +``` + +Running with different arguments will run different subsets of the tests. +No arguments, as we've already seen, runs all the tests: + +```text +$ cargo test + Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs + Running target/debug/deps/lol-06a75b4a1f2515e9 + +running 3 tests +test tests::add_three_and_two ... ok +test tests::one_hundred ... ok +test tests::add_two_and_two ... ok + +test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured + + Doc-tests lol + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` + +We can pass the name of any test function to run only that test: + +```text +$ cargo test one_hundred + Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs + Running target/debug/deps/lol-06a75b4a1f2515e9 + +running 1 test +test tests::one_hundred ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Doc-tests lol + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` + +We can also pass part of a name, and `cargo test` will run all tests +that match: + +```text +$ cargo test add + Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs + Running target/debug/deps/lol-06a75b4a1f2515e9 + +running 2 tests +test tests::add_three_and_two ... ok +test tests::add_two_and_two ... ok + +test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured + + Doc-tests lol + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` From 2f4c61dc06c6a41319d5bcd3d2067816bb895409 Mon Sep 17 00:00:00 2001 From: Steve Klabnik Date: Tue, 8 Nov 2016 16:18:56 -0500 Subject: [PATCH 07/18] remove myself --- src/chXX-02-integration-testing.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/chXX-02-integration-testing.md b/src/chXX-02-integration-testing.md index 2d642bf88c..6c91f72acd 100644 --- a/src/chXX-02-integration-testing.md +++ b/src/chXX-02-integration-testing.md @@ -30,7 +30,7 @@ Let's run them: ```bash $ cargo test - Compiling adder v0.1.0 (file:///home/steve/tmp/adder) + Compiling adder v0.1.0 (file:///projects/tmp/adder) Running target/adder-91b3e234d4ed382a running 1 test @@ -92,7 +92,7 @@ Let's try running this: ```bash $ cargo test - Compiling adder v0.1.0 (file:///home/steve/tmp/adder) + Compiling adder v0.1.0 (file:///projects/tmp/adder) Finished debug [unoptimized + debuginfo] target(s) in 0.25 secs Running target/debug/deps/adder-ce99bcc2479f4607 @@ -139,7 +139,7 @@ making modules in integration tests. If we move `tests/common.rs` to $ mkdir tests/common $ mv tests/common.rs tests/common/mod.rs $ cargo test - Compiling adder v0.1.0 (file:///home/steve/tmp/adder) + Compiling adder v0.1.0 (file:///projects/tmp/adder) Finished debug [unoptimized + debuginfo] target(s) in 0.24 secs Running target/debug/deps/adder-ce99bcc2479f4607 From 71b87e03d772ba7a54504e2ab0f83abc8aee6147 Mon Sep 17 00:00:00 2001 From: steveklabnik Date: Mon, 14 Nov 2016 14:37:44 -0500 Subject: [PATCH 08/18] swap order of sections to resolve @carols10cents @jonathandturner's concerns --- src/chXX-00-testing.md | 57 +++++++++++++++++++++--------------------- 1 file changed, 29 insertions(+), 28 deletions(-) diff --git a/src/chXX-00-testing.md b/src/chXX-00-testing.md index b44780155c..ef2275b335 100644 --- a/src/chXX-00-testing.md +++ b/src/chXX-00-testing.md @@ -142,34 +142,6 @@ And that's reflected in the summary line: test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured ``` -## Inverting failure with `should_panic` - -We can invert our test's failure with another attribute: `should_panic`: - -```rust -#[test] -#[should_panic] -fn it_works() { - assert!(false); -} -``` - -This test will now succeed if we `panic!` and fail if we complete. - -`should_panic` tests can be fragile, as it's hard to guarantee that the test -didn't fail for an unexpected reason. To help with this, an optional `expected` -parameter can be added to the `should_panic` attribute. The test harness will -make sure that the failure message contains the provided text. A safer version -of the example above would be: - -```rust -#[test] -#[should_panic(expected = "assertion failed")] -fn it_works() { - assert!(false); -} -``` - ## Testing equality Rust provides a pair of macros, `assert_eq!` and `assert_ne!`, that compares @@ -223,6 +195,35 @@ fn it_works() { } ``` +## Inverting failure with `should_panic` + +We can invert our test's failure with another attribute: `should_panic`: + +```rust +#[test] +#[should_panic] +fn it_works() { + assert!(false); +} +``` + +This test will now succeed if we `panic!` and fail if we complete. + +`should_panic` tests can be fragile, as it's hard to guarantee that the test +didn't fail for an unexpected reason. To help with this, an optional `expected` +parameter can be added to the `should_panic` attribute. The test harness will +make sure that the failure message contains the provided text. A safer version +of the example above would be: + +```rust +#[test] +#[should_panic(expected = "assertion failed")] +fn it_works() { + assert!(false); +} +``` + + ## The `ignore` attribute Sometimes a few specific tests can be very time-consuming to execute. These From 182fa3da0dab0b949db8ac3a1577f2fbb55a7b13 Mon Sep 17 00:00:00 2001 From: steveklabnik Date: Mon, 14 Nov 2016 14:43:42 -0500 Subject: [PATCH 09/18] Add notes about parallelism --- src/chXX-00-testing.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/src/chXX-00-testing.md b/src/chXX-00-testing.md index ef2275b335..08f1b46b3f 100644 --- a/src/chXX-00-testing.md +++ b/src/chXX-00-testing.md @@ -289,3 +289,33 @@ test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured The `--ignored` argument is an argument to the test binary, and not to Cargo, which is why the command is `cargo test -- --ignored`. + +## Parallelism + +One thing that is important to note when writing tests are run in parallel +using threads. For this reason you should take care that your tests are written +in such a way as to not depend on each other, or on any shared state. "Shared +state" can also include the environment, such as the current working directory, +or environment variables. + +If you don't want this behavior, or if you want more fine-grained control over +the number of threads used, you can use the `--test-threads` flag: + +```bash +$ cargo test -- --test-threads=1 # one thread means no parallelism +``` + +Note the `--`, which is important. `--test-threads` is an argument to our +tests, not to `cargo test` directly. + +## Test output + +By default Rust's test library captures and discards output to standard +out/error, e.g. output from `println!()`. This can be controlled using a flag: + +```bash +$ cargo test -- --nocapture +``` + +Note the `--`, which is important. `--nocapture` is an argument to our tests, +not to `cargo test` directly. From 97ffe9fdd95293fcc66ddfb05f95d6a5a91b3f9d Mon Sep 17 00:00:00 2001 From: "Carol (Nichols || Goulding)" Date: Wed, 23 Nov 2016 16:35:20 -0500 Subject: [PATCH 10/18] Remove the Debugging chapter that snuck back in --- src/SUMMARY.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/src/SUMMARY.md b/src/SUMMARY.md index eeddf89657..e55c0c40bb 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -60,8 +60,6 @@ - [Unit testing](chXX-01-unit-testing.md) - [Integration testing](chXX-02-integration-testing.md) -- [Debugging]() - ## Thinking in Rust - [Composition]() From d1107eccf55bd6b04c1202f2902eafb6de0e7051 Mon Sep 17 00:00:00 2001 From: "Carol (Nichols || Goulding)" Date: Wed, 23 Nov 2016 16:35:27 -0500 Subject: [PATCH 11/18] Carol's edits to the testing chapter Reorganize a bit, make some of the examples a bit more concrete --- src/SUMMARY.md | 8 +- src/ch11-00-testing.md | 30 +++ src/ch11-01-writing-tests.md | 247 +++++++++++++++++++ src/ch11-02-running-tests.md | 177 ++++++++++++++ src/ch11-03-test-organization.md | 366 +++++++++++++++++++++++++++++ src/chXX-00-testing.md | 321 ------------------------- src/chXX-01-unit-testing.md | 225 ------------------ src/chXX-02-integration-testing.md | 163 ------------- 8 files changed, 825 insertions(+), 712 deletions(-) create mode 100644 src/ch11-00-testing.md create mode 100644 src/ch11-01-writing-tests.md create mode 100644 src/ch11-02-running-tests.md create mode 100644 src/ch11-03-test-organization.md delete mode 100644 src/chXX-00-testing.md delete mode 100644 src/chXX-01-unit-testing.md delete mode 100644 src/chXX-02-integration-testing.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index e55c0c40bb..bbec4d8f50 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -50,15 +50,17 @@ - [Traits](ch10-02-traits.md) - [Lifetime syntax](ch10-03-lifetime-syntax.md) +- [Testing](ch11-00-testing.md) + - [Writing tests](ch11-01-writing-tests.md) + - [Running tests](ch11-02-running-tests.md) + - [Test Organization](ch11-03-test-organization.md) + - [I/O]() - [`Read` & `Write`]() - [`std::fs`]() - [`std::path`]() - [`std::env`]() -- [Testing](chXX-00-testing.md) - - [Unit testing](chXX-01-unit-testing.md) - - [Integration testing](chXX-02-integration-testing.md) ## Thinking in Rust diff --git a/src/ch11-00-testing.md b/src/ch11-00-testing.md new file mode 100644 index 0000000000..233093b7a5 --- /dev/null +++ b/src/ch11-00-testing.md @@ -0,0 +1,30 @@ +# Testing + +> Program testing can be a very effective way to show the presence of bugs, but +> it is hopelessly inadequate for showing their absence. +> +> Edsger W. Dijkstra, "The Humble Programmer" (1972) + +Rust is a programming language that cares a lot about correctness, but +correctness is a complex topic and isn't easy to prove. Rust places a lot of +weight on its type system to help ensure that our programs do what we intend, +but it cannot help with everything. As such, Rust also includes support for +writing software tests in the language itself. + +For example, we can write a function called `add_two` with a signature that +accepts an integer as an argument and returns an integer as a result. We can +implement and compile that function, and Rust can do all the type checking and +borrow checking that we've seen it's capable of doing. What Rust *can't* check +for us is that we've implemented this function to return the argument plus two +and not the argument plus 10 or the argument minus 50! That's where tests come +in: we can write tests that, for example, pass `3` to the `add_two` function +and check that we get `5` back. We can run the tests whenever we make changes +to our code to make sure we didn't change any existing behavior from what the +tests specify it should be. + +Testing is a skill, and we cannot hope to cover everything about how to write +good tests in one chapter of a book. What we can discuss, however, are the +mechanics of Rust's testing facilities. We'll talk about the annotations and +macros available to you when writing your tests, the default behavior and +options provided for running your tests, and how to organize tests into unit +tests and integration tests. diff --git a/src/ch11-01-writing-tests.md b/src/ch11-01-writing-tests.md new file mode 100644 index 0000000000..bed1ff8ae0 --- /dev/null +++ b/src/ch11-01-writing-tests.md @@ -0,0 +1,247 @@ +## Writing Tests + +Tests are Rust functions that use particular features and are written in such a +way as to verify that non-test code is functioning in the expected manner. +Everything we've discussed about Rust code applies to Rust tests as well! Let's +look at the features Rust provides specifically for writing tests: the `test` +attribute, a few macros, and the `should_panic` attribute. + +### The `test` attribute + +At its simplest, a test in Rust is a function that's annotated with the `test` +attribute. Let's make a new library project with Cargo called `adder`: + +```text +$ cargo new adder + Created library `adder` project +$ cd adder +``` + +Cargo will automatically generate a simple test when you make a new library +project. Here's the contents of `src/lib.rs`: + +Filename: src/lib.rs + +```rust +#[cfg(test)] +mod tests { + #[test] + fn it_works() { + } +} +``` + +For now, let's ignore the `tests` module and the `#[cfg(test)]` annotation in +order to focus on just the function. Note the `#[test]` before it: this +attribute indicates this is a test function. The function currently has no +body; that's good enough to pass! We can run the tests with `cargo test`: + +```text +$ cargo test + Compiling adder v0.1.0 (file:///projects/adder) + Finished debug [unoptimized + debuginfo] target(s) in 0.22 secs + Running target/debug/deps/adder-ce99bcc2479f4607 + +running 1 test +test it_works ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Doc-tests adder + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` + +Cargo compiled and ran our tests. There are two sets of output here; we're +going to focus on the first set in this chapter. The second set of output is +for documentation tests, which we'll talk about in Chapter 14. For now, note +this line: + +```text +test it_works ... ok +``` + +The `it_works` text comes from the name of our function. + +We also get a summary line that tells us the aggregate results of all the +tests that we have: + +```text +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured +``` + +### The `assert!` macro + +The empty test function passes because any test which doesn't `panic!` passes, +and any test that does `panic!` fails. Let's make the test fail by using the +`assert!` macro: + +Filename: src/lib.rs + +```rust +#[test] +fn it_works() { + assert!(false); +} +``` + +The `assert!` macro is provided by the standard library, and it takes one +argument. If the argument is `true`, nothing happens. If the argument is +`false`, the macro will `panic!`. Let's run our tests again: + +```text +$ cargo test + Compiling adder v0.1.0 (file:///projects/adder) + Finished debug [unoptimized + debuginfo] target(s) in 0.22 secs + Running target/debug/deps/adder-ce99bcc2479f4607 + +running 1 test +test it_works ... FAILED + +failures: + +---- it_works stdout ---- + thread 'it_works' panicked at 'assertion failed: false', src/lib.rs:5 +note: Run with `RUST_BACKTRACE=1` for a backtrace. + + +failures: + it_works + +test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured + +error: test failed +``` + +Rust indicates that our test failed: + +```text +test it_works ... FAILED +``` + +And shows that the test failed because the `assert!` macro in `src/lib.rs` on +line 5 got a `false` value: + +```text +thread 'it_works' panicked at 'assertion failed: false', src/lib.rs:5 +``` + +The test failure is also reflected in the summary line: + +```text +test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured +``` + +### Testing equality with the `assert_eq!` and `assert_ne!` macros + +A common way to test functionality is to compare the result of the code under +test to the value you expect it to be, and check that they're equal. You can do +this using the `assert!` macro by passing it an expression using the `==` +macro. This is so common, though, that the standard library provides a pair of +macros to do this for convenience: `assert_eq!` and `assert_ne!`. These macros +compare two arguments for equality or inequality, respectively. The other +advantage of using these macros is they will print out what the two values +actually are if the assertion fails so that it's easier to see *why* the test +failed, whereas the `assert!` macro would just print out that it got a `false` +value for the `==` expression. + +Here's an example test that uses each of these macros and will pass: + +Filename: src/lib.rs + +```rust +#[test] +fn it_works() { + assert_eq!("Hello", "Hello"); + + assert_ne!("Hello", "world"); +} +``` + +You can also specify an optional third argument to each of these macros, which +is a custom message that you'd like to be added to the failure message. The +macros expand to logic similar to this: + +```rust,ignore +// assert_eq +if left_val == right_val { + panic!("your optional custom message") +} + +// assert_ne +if left_val != right_val { + panic!("your optional custom message") +} +``` + +Let's take a look at a test that will fail becasue `hello` is not equal to +`world`. We've also added a custom error message, `greeting operation failed`: + +Filename: src/lib.rs + +```rust +#[test] +fn a_simple_case() { + let result = "hello"; // this value would come from running your code + assert_eq!(result, "world", "greeting operation failed"); +} +``` + +Running this indeed fails, and the output we get explains why the test failed +and includes the custom error message we specified: + +```text +---- a_simple_case stdout ---- + thread 'a_simple_case' panicked at 'assertion failed: `(left == right)` (left: `"hello"`, right: `"world"`): greeting operation failed', src/main.rs:4 +``` + +The two arguments to `assert_eq!` are named "left" and "right" rather than +"expected" and "actual"; the order of the value that comes from your code and +the value hardcoded into your test isn't important. + +## Test for failure with `should_panic` + +We can invert our test's failure with another attribute: `should_panic`. This +is useful when we want to test that calling a particular function will cause an +error. For example, let's test something that we know will panic from Chapter +8: attempting to create a slice using range syntax with byte indices that +aren't on character boundaries. Add the `#[should_panic]` attribute before the +function like the `#[test]` attribute: + +Filename: src/lib.rs + +```rust +#[test] +#[should_panic] +fn slice_not_on_char_boundaries() { + let s = "Здравствуйте"; + &s[0..1]; +} +``` + +This test will succeed, since the code panics and we said that it should. If +this code happened to run and did not cause a `panic!`, this test would fail. + +`should_panic` tests can be fragile, as it's hard to guarantee that the test +didn't fail for a different reason than the one you were expecting. To help +with this, an optional `expected` parameter can be added to the `should_panic` +attribute. The test harness will make sure that the failure message contains +the provided text. A safer version of the example above would be: + +Filename: src/lib.rs + +```rust +#[test] +#[should_panic(expected = "do not lie on character boundary")] +fn slice_not_on_char_boundaries() { + let s = "Здравствуйте"; + &s[0..1]; +} +``` + +Try on your own to see what happens when a `should_panic` test panics but +doesn't match the expected message: cause a `panic!` that happens for a +different reason in this test, or change the expected panic message to +something that doesn't match the character boundary panic message. diff --git a/src/ch11-02-running-tests.md b/src/ch11-02-running-tests.md new file mode 100644 index 0000000000..4800b26d71 --- /dev/null +++ b/src/ch11-02-running-tests.md @@ -0,0 +1,177 @@ +## Running tests + +Just like `cargo run` compiles your code and then runs the resulting binary, +`cargo test` compiles your code in test mode and runs the resulting test +binary. The default behavior of the binary that `cargo test` produces is to run +all the tests in parallel and to capture output generated during test runs so +that it's easier to read the output about the test results. + +The default behavior of running tests can be changed by specifying command line +options. Some of these options can be passed to `cargo test`, and some need to +be passed instead to the resulting test binary. The way to separate these +arguments is with `--`: after `cargo test`, list the arguments that go to +`cargo test`, then the separator `--`, and then the arguments that go to the +test binary. + +### Tests Run in Parallel + +Tests are run in parallel using threads. For this reason, you should take care +that your tests are written in such a way as to not depend on each other or on +any shared state. Shared state can also include the environment, such as the +current working directory or environment variables. + +If you don't want this behavior, or if you want more fine-grained control over +the number of threads used, you can send the `--test-threads` flag and the +number of threads to the test binary. Setting the number of test threads to 1 +means to not use any parallelism: + +```text +$ cargo test -- --test-threads=1 +``` + +### Tests Capture Output + +By default, Rust's test library captures and discards output to standard out +and standard error, unless the test fails. For example, if you call +`println!()` in a test and the test passes, you won't see the `println!` output +in your terminal. This behavior can be disabled by sending the `--nocapture` +flag to the test binary: + +```text +$ cargo test -- --nocapture +``` + +### Running a Subset of Tests by Name + +Sometimes, running a full test suite can take a long time. If you're only +working on code in a particular area, you might want to only run the tests +having to do with that code. `cargo test` takes an argument that allows you to +only run certain tests, specified by name. + +Let's create three tests with the following names: + +Filename: src/lib.rs + +```rust +#[test] +fn add_two_and_two() { + assert_eq!(4, 2 + 2); +} + +#[test] +fn add_three_and_two() { + assert_eq!(5, 3 + 2); +} + +#[test] +fn one_hundred() { + assert_eq!(102, 100 + 2); +} +``` + +Running with different arguments will run different subsets of the tests. No +arguments, as we've already seen, runs all the tests: + +```text +$ cargo test + Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs + Running target/debug/deps/adder-06a75b4a1f2515e9 + +running 3 tests +test add_three_and_two ... ok +test one_hundred ... ok +test add_two_and_two ... ok + +test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured +``` + +We can pass the name of any test function to run only that test: + +```text +$ cargo test one_hundred + Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs + Running target/debug/deps/adder-06a75b4a1f2515e9 + +running 1 test +test one_hundred ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured +``` + +We can also pass part of a name, and `cargo test` will run all tests that match: + +```text +$ cargo test add + Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs + Running target/debug/deps/adder-06a75b4a1f2515e9 + +running 2 tests +test add_three_and_two ... ok +test add_two_and_two ... ok + +test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured +``` + +### Ignore Some Tests Unless Specifically Requested + +Sometimes a few specific tests can be very time-consuming to execute, so during +most runs of `cargo test`, we'd like to exclude them. Instead of having to +construct an argument to `cargo test` to run all tests except these and +remember to use that argument every time, we can annotate these tests with the +`ignore` attribute: + +Filename: src/lib.rs + +```rust +#[test] +fn it_works() { + assert!(true); +} + +#[test] +#[ignore] +fn expensive_test() { + // code that takes an hour to run +} +``` + +Now if we run our tests, we'll see `it_works` is run, but `expensive_test` is +not: + +```text +$ cargo test + Compiling adder v0.1.0 (file:///projects/adder) + Finished debug [unoptimized + debuginfo] target(s) in 0.24 secs + Running target/debug/deps/adder-ce99bcc2479f4607 + +running 2 tests +test expensive_test ... ignored +test it_works ... ok + +test result: ok. 1 passed; 0 failed; 1 ignored; 0 measured + + Doc-tests adder + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` + +We can run only the expensive tests by explicitly asking to run them using +`cargo test -- --ignored`: + +```text +$ cargo test -- --ignored + Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs + Running target/debug/deps/adder-ce99bcc2479f4607 + +running 1 test +test expensive_test ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured +``` + +This way, most of the time that you run `cargo test` the results would be fast. +When you're at a point that it makes sense to check the results of the +`ignored` tests and you have time to wait for the results, you can choose to +run `cargo test -- --ignored` instead. diff --git a/src/ch11-03-test-organization.md b/src/ch11-03-test-organization.md new file mode 100644 index 0000000000..b9ef304a6b --- /dev/null +++ b/src/ch11-03-test-organization.md @@ -0,0 +1,366 @@ +## Test Organization + +As mentioned before, testing is a large discipline, and different people +sometimes use different terminology and organization. The Rust community tends +to think about tests in terms of two main categories: *unit tests* and +*integration tests*. Unit tests tend to be smaller and more focused, testing +one module in isolation at a time. They can also test private interfaces. +Integration tests are entirely external to your library. They use your code in +the same way any other code would, using only the public interface and +exercising multiple modules per test. Both kinds of tests are important to +ensure that the pieces of your library are doing what you expect them to +separately and together. + +### Unit Tests + +The purpose of unit tests is to test each unit of code in isolation from the +rest of the code, in order to be able to quickly pinpoint where code is working +as expected or not. Unit tests live in the *src* directory, in the same files +as the code they are testing. They are separated into their own `tests` module +in each file. + +#### The Tests Module and `cfg(test)` + +By placing tests in their own module and using the `cfg` annotation on the +module, we can tell Rust to only compile and run the test code when we run +`cargo test`. This saves compile time when we only want to build the library +code with `cargo build`, and saves space in the resulting compiled artifact +since the tests are not included. + +Remember when we generated the new `adder` project in the last section? Cargo +generated this code for us: + +Filename: src/lib.rs + +```rust +#[cfg(test)] +mod tests { + #[test] + fn it_works() { + } +} +``` + +We ignored the module stuff so we could concentrate on the mechanics of the +test code inside the module, but now let's focus on the code surrounding our +tests. + +First of all, there's a new attribute, `cfg`. The `cfg` attribute lets us +declare that something should only be included given a certain *configuration*. +Rust provides the `test` configuration for compiling and running tests. By +using this attribute, Cargo only compiles our test code if we're currently +trying to run the tests. + +Next, the `tests` module holds all of our test functions, while our code is +outside of the `tests` module. The name of the `tests` module is a convention; +otherwise this is a regular module that follows the usual visibility rules we +covered in Chapter 7. Because we're in an inner module, we need to bring the +code under test into scope. This can be annoying if you have a large module, so +this is a common use of globs. + +Up until now in this chapter, we've been writing tests in our `adder` project +that don't actually call any code we've written. Let's change that now! In +*src/lib.rs*, place this `add_two` function and `tests` module that has a test +function to exercise the code: + +Filename: src/lib.rs + +```rust +pub fn add_two(a: i32) -> i32 { + a + 2 +} + +#[cfg(test)] +mod tests { + use add_two; + + #[test] + fn it_works() { + assert_eq!(4, add_two(2)); + } +} +``` + +Notice in addition to the test function, we also added `use add_two;` within +the `tests` module. This brings the code we want to test into the scope of the +inner `tests` module, just like we'd need to do for any inner module. If we run +this test now with `cargo test`, it will pass: + +```text +running 1 test +test tests::it_works ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured +``` + +If we had forgotten to bring the `add_two` function into scope, we would get an +unresolved name error since the `tests` module wouldn't know anything about the +`add_two` function: + +```text +error[E0425]: unresolved name `add_two` + --> src/lib.rs:9:23 + | +9 | assert_eq!(4, add_two(2)); + | ^^^^^^^ unresolved name +``` + +If this module contained lots of code we wanted to test, it would be annoying +to list everything in the `use` statement in the tests. It's common instead to +put `use super::*;` within a module's `test` submodule in order to bring +everything into the `test` module scope at once. + +#### Testing Private Functions + +There's controversy within the testing community about whether you should write +unit tests for private functions or not. Regardless of which testing ideology +you adhere to, Rust does allow you to test private functions due to the way +that the privacy rules work. Consider this code with the private function +`internal_adder`: + +Filename: src/lib.rs + +```rust +pub fn add_two(a: i32) -> i32 { + internal_adder(a, 2) +} + +fn internal_adder(a: i32, b: i32) -> i32 { + a + b +} + +#[cfg(test)] +mod tests { + use internal_adder; + + #[test] + fn internal() { + assert_eq!(4, internal_adder(2, 2)); + } +} +``` + +Because tests are just Rust code and the `tests` module is just another module, +we can import and call `internal_adder` in a test just fine. If you don't think +private functions should be tested, there's nothing in Rust that will compel +you to do so. + +### Integration Tests + +In Rust, integration tests are tests that are entirely external to your +library. They use your library in the same way any other code would. Their +purpose is to test that many parts of your library work correctly together. +Units of code that work correctly by themselves could have problems when +integrated, so test coverage of the integrated code is important as well. + +#### The *tests* Directory + +Cargo has support for integration tests in the *tests* directory. If you make +one and put Rust files inside, Cargo will compile each of the files as an +individual crate. Let's give it a try! + +First, make a *tests* directory at the top level of your project directory, next +to *src*. Then, make a new file, *tests/integration_test.rs*, and put this +inside: + +Filename: tests/integration_test.rs + +```rust,ignore +extern crate adder; + +#[test] +fn it_adds_two() { + assert_eq!(4, adder::add_two(2)); +} +``` + +We now have `extern crate adder` at the top, which we didn't need in the unit +tests. Each test in the `tests` directory is an entirely separate crate, so we +need to import our library into each of them. This is also why `tests` is a +suitable place to write integration-style tests: they use the library like any +other consumer of it would, by importing the crate and using only the public +API. + +We also don't need a `tests` module in this file. The whole directory won't be +compiled unless we're running the tests, so we don't need to annotate any part +of it with `#[cfg(test)]`. Also, each test file is already isolated into its +own crate, so we don't need to separate the test code further. + +Let's run the integration tests, which also get run when we run `cargo test`: + +```text +$ cargo test + Compiling adder v0.1.0 (file:///projects/adder) + Running target/debug/deps/adder-91b3e234d4ed382a + +running 1 test +test tests::it_works ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Running target/debug/integration_test-952a27e0126bb565 + +running 1 test +test it_adds_two ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Doc-tests adder + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` + +Now we have three sections of output: the unit tests, the integration test, and +the doc tests. Note that adding more unit tests in any *src* file will add more +lines to the unit tests section. Adding more test functions to the integration +test file we created will add more lines to that section. If we add more +integration test *files* in the *tests* directory, there will be more +integration test sections: one for each file. + +Specifying a test function name argument with `cargo test` will also match +against test function names in any integration test file. To run all of the +tests in only one particular integration test file, use the `--test` argument +of `cargo test`: + +```text +$ cargo test --test integration_test + Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs + Running target/debug/integration_test-952a27e0126bb565 + +running 1 test +test it_adds_two ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured +``` + +#### Submodules in integration tests + +As you add more integration tests, you may want to make more than one file in +the `tests` directory in order to group the test functions by the functionality +they're testing, for example. As we mentioned before, that will work fine, +given that Cargo treats every file as its own crate. But there's one small trap +that can happen if we try to extract a file that contains common helper +functions that we want to share across all of the integration test files. + +Let's try to extract a module called `common` from our +*tests/integration_test.rs* file into its own file by following the steps we +took to extract a module in Chapter 7. The `common` module contains a function +named `helper` that takes care of some setup that all tests in the *tests* +directory need to do. So we remove the `helper` function from the +*tests/integration_test.rs* file, leaving the `mod common` declaration: + +Filename: tests/integration_test.rs + +```rust,ignore +extern crate adder; + +mod common; + +#[test] +fn it_works() { + common::helper(); + + assert_eq!(4, adder::add_two(2)); +} +``` + +Then we create a `tests/common.rs` file to hold our common helpers: + +Filename: tests/common.rs + +```rust +pub fn helper() { + // test setup code goes here +} +``` + +Let's try running the tests: + +```text +$ cargo test + Compiling adder v0.1.0 (file:///projects/tmp/adder) + Finished debug [unoptimized + debuginfo] target(s) in 0.25 secs + Running target/debug/deps/adder-ce99bcc2479f4607 + +running 1 test +test tests::internal ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Running target/debug/common-c3635c69f3aeef92 + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured + + Running target/debug/integration_tests-6d6e12b4680b0368 + +running 1 test +test it_works ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Doc-tests adder + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` + +Wait a minute. Now we have four sections: one is for our `common` module! + +```text +Running target/debug/common-c3635c69f3aeef92 +``` + +Because `common.rs` is in our `tests` directory, Cargo is also compiling it as +its own crate. `common.rs` is so simple, so we didn't get any errors, but +with more complex code, this might not work. So what can we do? + +The key is, always use the `common/mod.rs` form over the `common.rs` form when +making modules in integration tests. If we move `tests/common.rs` to +`tests/common/mod.rs`, we'll go back to our expected output: + +```text +$ mkdir tests/common +$ mv tests/common.rs tests/common/mod.rs +$ cargo test + Compiling adder v0.1.0 (file:///projects/tmp/adder) + Finished debug [unoptimized + debuginfo] target(s) in 0.24 secs + Running target/debug/deps/adder-ce99bcc2479f4607 + +running 1 test +test tests::internal ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Running target/debug/integration_tests-6d6e12b4680b0368 + +running 1 test +test it_works ... ok + +test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured + + Doc-tests adder + +running 0 tests + +test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured +``` + +## Summary + +Rust's testing features provide a way to specify how code should function to +ensure the code continues to work in the specified ways even as we make +changes. Unit tests exercise different parts of a library separately and can +test private implementation details. Integration tests cover the use of many +parts of the library working together, and use the library's public API to test +the code in the same way other code will use it. Rust's type system and +ownership rules help prevent some kinds of bugs, but tests are an important +part of reducing logic bugs having to do with how your code is expected to +behave. + +Let's put together the knowledge from this chapter and other previous chapters +and work on a project in the next chapter! diff --git a/src/chXX-00-testing.md b/src/chXX-00-testing.md deleted file mode 100644 index 08f1b46b3f..0000000000 --- a/src/chXX-00-testing.md +++ /dev/null @@ -1,321 +0,0 @@ -# Testing - -> Program testing can be a very effective way to show the presence of bugs, but -> it is hopelessly inadequate for showing their absence. -> -> Edsger W. Dijkstra, "The Humble Programmer" (1972) - -Rust is a programming language that cares a lot about correctness. But -correctness is a complex topic, and isn't exactly easy to get right. Rust -places a lot of weight on its type system to help ensure that our programs do -what we intend, but it cannot help with everything. As such, Rust also includes -support for writing software tests in the language itself. - -Testing is a skill, and we cannot hope to learn everything about how to write -good tests in one chapter of a book. What we can learn, however, are the -mechanics of Rust's testing facilities. That's what we'll focus on in this -chapter. - -## The `test` attribute - -At its simplest, a test in Rust is a function that's annotated with the `test` -attribute. Let's make a new project with Cargo called `adder`: - -```bash -$ cargo new adder - Created library `adder` project -$ cd adder -``` - -Cargo will automatically generate a simple test when you make a new project. -Here's the contents of `src/lib.rs`: - -```rust -#[cfg(test)] -mod tests { - #[test] - fn it_works() { - } -} -``` - -For now, let's remove the `mod` bit, and focus on just the function: - -```rust -#[test] -fn it_works() { -} -``` - -Note the `#[test]`. This attribute indicates that this is a test function. It -currently has no body. That's good enough to pass! We can run the tests with -`cargo test`: - -```bash -$ cargo test - Compiling adder v0.1.0 (file:///projects/adder) - Finished debug [unoptimized + debuginfo] target(s) in 0.22 secs - Running target/debug/deps/adder-ce99bcc2479f4607 - -running 1 test -test it_works ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Doc-tests adder - -running 0 tests - -test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured -``` - -Cargo compiled and ran our tests. There are two sets of output here: one -for the test we wrote, and another for documentation tests. We'll talk about -documentation tests later. For now, see this line: - -```text -test it_works ... ok -``` - -Note the `it_works`. This comes from the name of our function: - -```rust -fn it_works() { -# } -``` - -We also get a summary line: - -```text -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured -``` - -## The `assert!` macro - -So why does our do-nothing test pass? Any test which doesn't `panic!` passes, -and any test that does `panic!` fails. Let's make our test fail: - -```rust -#[test] -fn it_works() { - assert!(false); -} -``` - -`assert!` is a macro provided by Rust which takes one argument: if the argument -is `true`, nothing happens. If the argument is `false`, it will `panic!`. Let's -run our tests again: - -```bash -$ cargo test - Compiling adder v0.1.0 (file:///projects/adder) - Finished debug [unoptimized + debuginfo] target(s) in 0.22 secs - Running target/debug/deps/adder-ce99bcc2479f4607 - -running 1 test -test it_works ... FAILED - -failures: - ----- it_works stdout ---- - thread 'it_works' panicked at 'assertion failed: false', src/lib.rs:5 -note: Run with `RUST_BACKTRACE=1` for a backtrace. - - -failures: - it_works - -test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured - -error: test failed -``` - -Rust indicates that our test failed: - -```text -test it_works ... FAILED -``` - -And that's reflected in the summary line: - -```text -test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured -``` - -## Testing equality - -Rust provides a pair of macros, `assert_eq!` and `assert_ne!`, that compares -two arguments for equality: - -```rust -#[test] -fn it_works() { - assert_eq!("Hello", "Hello"); - - assert_ne!("Hello", "world"); -} -``` - -These macros expand to something like this: - -```rust,ignore -// assert_eq -if left_val == right_val { - panic!("message goes here") -} - -// assert_ne -if left_val =! right_val { - panic!("message goes here") -} -``` - -And if the test fails, we get an error that looks like this: - -```text ----- failing_test stdout ---- - thread 'failing_test' panicked at 'assertion failed: `(left == right)` (left: `"hello"`, right: `"world"`)', failing_test.rs:4 -``` - -One other thing to notice is that they're named "left" and "right" rather than -"expected" vs "actual"; the order isn't particularly important. - -But they're a bit more convenient than writing this out by hand. These macros -are often used to call some function with some known arguments and compare it -to the expected output, like this: - -```rust -pub fn add_two(a: i32) -> i32 { - a + 2 -} - -#[test] -fn it_works() { - assert_eq!(4, add_two(2)); -} -``` - -## Inverting failure with `should_panic` - -We can invert our test's failure with another attribute: `should_panic`: - -```rust -#[test] -#[should_panic] -fn it_works() { - assert!(false); -} -``` - -This test will now succeed if we `panic!` and fail if we complete. - -`should_panic` tests can be fragile, as it's hard to guarantee that the test -didn't fail for an unexpected reason. To help with this, an optional `expected` -parameter can be added to the `should_panic` attribute. The test harness will -make sure that the failure message contains the provided text. A safer version -of the example above would be: - -```rust -#[test] -#[should_panic(expected = "assertion failed")] -fn it_works() { - assert!(false); -} -``` - - -## The `ignore` attribute - -Sometimes a few specific tests can be very time-consuming to execute. These -can be disabled by default by using the `ignore` attribute: - -```rust -pub fn add_two(a: i32) -> i32 { - a + 2 -} - -#[test] -fn it_works() { - assert_eq!(4, add_two(2)); -} - -#[test] -#[ignore] -fn expensive_test() { - // code that takes an hour to run -} -``` - -Now we run our tests and see that `it_works` is run, but `expensive_test` is -not: - -```bash -$ cargo test - Compiling adder v0.1.0 (file:///projects/adder) - Finished debug [unoptimized + debuginfo] target(s) in 0.24 secs - Running target/debug/deps/adder-ce99bcc2479f4607 - -running 2 tests -test expensive_test ... ignored -test it_works ... ok - -test result: ok. 1 passed; 0 failed; 1 ignored; 0 measured - - Doc-tests adder - -running 0 tests - -test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured -``` - -The expensive tests can be run explicitly using `cargo test -- --ignored`: - -```bash -$ cargo test -- --ignored - Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs - Running target/debug/deps/adder-ce99bcc2479f4607 - -running 1 test -test expensive_test ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Doc-tests adder - -running 0 tests - -test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured -``` - -The `--ignored` argument is an argument to the test binary, and not to Cargo, -which is why the command is `cargo test -- --ignored`. - -## Parallelism - -One thing that is important to note when writing tests are run in parallel -using threads. For this reason you should take care that your tests are written -in such a way as to not depend on each other, or on any shared state. "Shared -state" can also include the environment, such as the current working directory, -or environment variables. - -If you don't want this behavior, or if you want more fine-grained control over -the number of threads used, you can use the `--test-threads` flag: - -```bash -$ cargo test -- --test-threads=1 # one thread means no parallelism -``` - -Note the `--`, which is important. `--test-threads` is an argument to our -tests, not to `cargo test` directly. - -## Test output - -By default Rust's test library captures and discards output to standard -out/error, e.g. output from `println!()`. This can be controlled using a flag: - -```bash -$ cargo test -- --nocapture -``` - -Note the `--`, which is important. `--nocapture` is an argument to our tests, -not to `cargo test` directly. diff --git a/src/chXX-01-unit-testing.md b/src/chXX-01-unit-testing.md deleted file mode 100644 index db58a7f69d..0000000000 --- a/src/chXX-01-unit-testing.md +++ /dev/null @@ -1,225 +0,0 @@ -# Unit testing - -As we mentioned before, testing is a large discipline, and so different people -can sometimes use different terminology. For our purposes, we tend to place -tests into two main categories: *unit tests* and *integration tests*. Unit -tests tend to be smaller, and more focused. In Rust, they can also test -non-public interfaces. Let's talk more about how to do unit testing in Rust. - -## The tests module and `cfg(test)` - -Remember when we generated our new project in the last section? Cargo had -generated some stuff for us: - -```rust -#[cfg(test)] -mod tests { - #[test] - fn it_works() { - } -} -``` - -We deleted the module stuff so we could learn more about the mechanics of -tests. But there's a reason that Cargo generated this module for us: it's the -idiomatic way to organize unit tests in Rust. That is, unit tests are: - -* Stored inside of the same tree as your source code. -* Placed inside their own module. - -For a more realistic example of how this works, consider our `add_two` function -from before: - -```rust -pub fn add_two(a: i32) -> i32 { - a + 2 -} - -#[cfg(test)] -mod tests { - use add_two; - - #[test] - fn it_works() { - assert_eq!(4, add_two(2)); - } -} -``` - -First of all, there's a new attribute, `cfg`. The `cfg` attribute lets us -declare that something should only be included given a certain configuration. -Rust provides the `test` configuration when compiling and running tests. By -using this attribute, Cargo only compiles our test code if we're currently -trying to run the tests. Given that they're not compiled at all during a -regular `cargo build`, this can save compile time. It also ensures that our -tests are entirely left out of the binary, saving space in a non-testing -context. - -You'll notice one more change: the `use` declaration. The `tests` module is -only a convention, it's nothing that Rust understands directly. As such, we -have to follow the usual visibility rules. Because we're in an inner module, -we need to bring our test function into scope. This can be annoying if you have -a large module, and so this is a common use of globs. Let's change our -`src/lib.rs` to make use of it: - -```rust,ignore -pub fn add_two(a: i32) -> i32 { - a + 2 -} - -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn it_works() { - assert_eq!(4, add_two(2)); - } -} -``` - -Note the different `use` line. Now we run our tests: - -```bash -$ cargo test - Compiling adder v0.1.0 (file:///projects/adder) - Finished debug [unoptimized + debuginfo] target(s) in 0.27 secs - Running target/debug/deps/adder-ce99bcc2479f4607 - -running 1 test -test tests::it_works ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Doc-tests adder - -running 0 tests - -test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured -``` - -It works! - -## Testing internal functions - -There's controversy within the testing community about unit testing private -functions. Regardless of which testing ideology you adhere to, Rust does allow -you to test them, due to the way that the privacy rules work. Consider this: - -```rust -pub fn add_two(a: i32) -> i32 { - internal_adder(a, 2) -} - -fn internal_adder(a: i32, b: i32) -> i32 { - a + b -} - -#[cfg(test)] -mod tests { - use internal_adder; - - #[test] - fn internal() { - assert_eq!(4, internal_adder(2, 2)); - } -} -``` - -In this scenario, we have a non-`pub` function, `internal_adder`. Because tests -are just Rust code, and the `tests` module is just another module, we can -import and call `internal_adder` in a test just fine. - -## Running a subset of tests - -Sometimes, running a full test suite can take a long time. `cargo test` takes -an argument that allows you to only run certain tests, if you'd prefer to do that. -Let's say we had two tests of `add_two`: - -```rust -pub fn add_two(a: i32) -> i32 { - a + 2 -} - -#[cfg(test)] -mod tests { - use add_two; - - #[test] - fn add_two_and_two() { - assert_eq!(4, add_two(2)); - } - - #[test] - fn add_three_and_two() { - assert_eq!(5, add_two(3)); - } - - #[test] - fn one_hundred() { - assert_eq!(102, add_two(100)); - } -} -``` - -Running with different arguments will run different subsets of the tests. -No arguments, as we've already seen, runs all the tests: - -```text -$ cargo test - Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs - Running target/debug/deps/lol-06a75b4a1f2515e9 - -running 3 tests -test tests::add_three_and_two ... ok -test tests::one_hundred ... ok -test tests::add_two_and_two ... ok - -test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured - - Doc-tests lol - -running 0 tests - -test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured -``` - -We can pass the name of any test function to run only that test: - -```text -$ cargo test one_hundred - Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs - Running target/debug/deps/lol-06a75b4a1f2515e9 - -running 1 test -test tests::one_hundred ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Doc-tests lol - -running 0 tests - -test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured -``` - -We can also pass part of a name, and `cargo test` will run all tests -that match: - -```text -$ cargo test add - Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs - Running target/debug/deps/lol-06a75b4a1f2515e9 - -running 2 tests -test tests::add_three_and_two ... ok -test tests::add_two_and_two ... ok - -test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured - - Doc-tests lol - -running 0 tests - -test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured -``` diff --git a/src/chXX-02-integration-testing.md b/src/chXX-02-integration-testing.md deleted file mode 100644 index 6c91f72acd..0000000000 --- a/src/chXX-02-integration-testing.md +++ /dev/null @@ -1,163 +0,0 @@ -# Integration testing - -In the last section, we talked about unit tests. But there's still that other -category: integration testing. In Rust, an integration test is a test that is -entirely external to your library. It uses it in the same way any other code -would. - -Cargo has support for integration tests through the `tests` directory. If you -make one, and put `.rs` files inside, Cargo will compile each of them as an -individual crate. Let's give it a try! First, make a `tests` directory at the -top level of your project, next to `src`. Then, make a new file, -`tests/integration_test.rs`, and put this inside: - -```rust,ignore -extern crate adder; - -#[test] -fn it_works() { - assert_eq!(4, adder::add_two(2)); -} -``` - -There's some small changes from our previous tests. We now have an `extern -crate adder` at the top. This is because each test in the `tests` directory is -an entirely separate crate, and so we need to import our library. This is also -why `tests` is a suitable place to write integration-style tests: they use the -library like any other consumer of it would. - -Let's run them: - -```bash -$ cargo test - Compiling adder v0.1.0 (file:///projects/tmp/adder) - Running target/adder-91b3e234d4ed382a - -running 1 test -test tests::it_works ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Running target/lib-c18e7d3494509e74 - -running 1 test -test it_works ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Doc-tests adder - -running 0 tests - -test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured -``` - -Now we have three sections: our previous test is also run, as well as our new -one. - -That's all there is to the `tests` directory. The `tests` module isn't needed -here, since the whole thing is focused on tests. - -## Submodules in integration tests - -As your integration tests grow, you may want to make more than one file in the -`tests` directory. As we mentioned before, that works well, given that Cargo -treats every file as its own crate. But there's one small trap that can happen. - -Imagine we wanted some common helper functions to be shared across our tests. -So we change our test to have a `common` module: - -```rust,ignore -extern crate adder; - -mod common; - -#[test] -fn it_works() { - common::helper(); - - assert_eq!(4, adder::add_two(2)); -} -``` - -And then, we create a `tests/common.rs` file to hold our common helpers: - -```rust -pub fn helper() { - // no implementation for now -} -``` - -Let's try running this: - -```bash -$ cargo test - Compiling adder v0.1.0 (file:///projects/tmp/adder) - Finished debug [unoptimized + debuginfo] target(s) in 0.25 secs - Running target/debug/deps/adder-ce99bcc2479f4607 - -running 1 test -test tests::internal ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Running target/debug/common-c3635c69f3aeef92 - -running 0 tests - -test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured - - Running target/debug/integration_tests-6d6e12b4680b0368 - -running 1 test -test it_works ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Doc-tests adder - -running 0 tests - -test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured -``` - -Wait a minute. Now we have four sections? - -```text - Running target/debug/common-c3635c69f3aeef92 -``` - -Because `common.rs` is in our `tests` directory, Cargo is also compiling it as -its own crate. Because `common.rs` is so simple, we didn't get an error, but -with more complex code, this might not work. So what can we do? - -The key is, always use the `common/mod.rs` form over the `common.rs` form when -making modules in integration tests. If we move `tests/common.rs` to -`tests/common/mod.rs`, we'll go back to our expected output: - -```bash -$ mkdir tests/common -$ mv tests/common.rs tests/common/mod.rs -$ cargo test - Compiling adder v0.1.0 (file:///projects/tmp/adder) - Finished debug [unoptimized + debuginfo] target(s) in 0.24 secs - Running target/debug/deps/adder-ce99bcc2479f4607 - -running 1 test -test tests::internal ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Running target/debug/integration_tests-6d6e12b4680b0368 - -running 1 test -test it_works ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Doc-tests adder - -running 0 tests - -test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured -``` From a68f2faf0859a180b1146b828bb8298533644b80 Mon Sep 17 00:00:00 2001 From: "Carol (Nichols || Goulding)" Date: Sat, 26 Nov 2016 14:02:19 -0500 Subject: [PATCH 12/18] Shorten section about integration test modules And add a section about integration testing binary crates --- src/ch11-03-test-organization.md | 135 ++++++------------------------- 1 file changed, 25 insertions(+), 110 deletions(-) diff --git a/src/ch11-03-test-organization.md b/src/ch11-03-test-organization.md index b9ef304a6b..3bc1578ed6 100644 --- a/src/ch11-03-test-organization.md +++ b/src/ch11-03-test-organization.md @@ -235,120 +235,35 @@ test it_adds_two ... ok test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured ``` -#### Submodules in integration tests +#### Submodules in Integration Tests As you add more integration tests, you may want to make more than one file in the `tests` directory in order to group the test functions by the functionality they're testing, for example. As we mentioned before, that will work fine, -given that Cargo treats every file as its own crate. But there's one small trap -that can happen if we try to extract a file that contains common helper -functions that we want to share across all of the integration test files. - -Let's try to extract a module called `common` from our -*tests/integration_test.rs* file into its own file by following the steps we -took to extract a module in Chapter 7. The `common` module contains a function -named `helper` that takes care of some setup that all tests in the *tests* -directory need to do. So we remove the `helper` function from the -*tests/integration_test.rs* file, leaving the `mod common` declaration: - -Filename: tests/integration_test.rs - -```rust,ignore -extern crate adder; - -mod common; - -#[test] -fn it_works() { - common::helper(); - - assert_eq!(4, adder::add_two(2)); -} -``` - -Then we create a `tests/common.rs` file to hold our common helpers: - -Filename: tests/common.rs - -```rust -pub fn helper() { - // test setup code goes here -} -``` - -Let's try running the tests: - -```text -$ cargo test - Compiling adder v0.1.0 (file:///projects/tmp/adder) - Finished debug [unoptimized + debuginfo] target(s) in 0.25 secs - Running target/debug/deps/adder-ce99bcc2479f4607 - -running 1 test -test tests::internal ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Running target/debug/common-c3635c69f3aeef92 - -running 0 tests - -test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured - - Running target/debug/integration_tests-6d6e12b4680b0368 - -running 1 test -test it_works ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Doc-tests adder - -running 0 tests - -test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured -``` - -Wait a minute. Now we have four sections: one is for our `common` module! - -```text -Running target/debug/common-c3635c69f3aeef92 -``` - -Because `common.rs` is in our `tests` directory, Cargo is also compiling it as -its own crate. `common.rs` is so simple, so we didn't get any errors, but -with more complex code, this might not work. So what can we do? - -The key is, always use the `common/mod.rs` form over the `common.rs` form when -making modules in integration tests. If we move `tests/common.rs` to -`tests/common/mod.rs`, we'll go back to our expected output: - -```text -$ mkdir tests/common -$ mv tests/common.rs tests/common/mod.rs -$ cargo test - Compiling adder v0.1.0 (file:///projects/tmp/adder) - Finished debug [unoptimized + debuginfo] target(s) in 0.24 secs - Running target/debug/deps/adder-ce99bcc2479f4607 - -running 1 test -test tests::internal ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Running target/debug/integration_tests-6d6e12b4680b0368 - -running 1 test -test it_works ... ok - -test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured - - Doc-tests adder - -running 0 tests - -test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured -``` +given that Cargo treats every file as its own crate. + +Eventually, you may have a set of helper functions that are common to all +integration tests, for example, functions that set up common scenarios. If you +extract these into a file in the *tests* directory, like *tests/common.rs* for +example, this file will be compiled into a separate crate just like the Rust +files in this directory that contain test functions are. There will be a +separate section in the test output for this file. Since this is probably not +what you want, it's recommended to instead use a *mod.rs* file in a +subdirectory, like *tests/common/mod.rs*, for helper functions. Files in +subdirectories of the *tests* directory do not get compiled as separate crates +or have sections in the test output. + +#### Integration Tests for Binary Crates + +If your project is a binary crate that only contains a *src/main.rs* and does +not have a *src/lib.rs*, it is not possible to create integration tests in the +*tests* directory and use `extern crate` to import the functions in +*src/main.rs*. This is one of the reasons Rust projects that provide a binary +have a straightforward *src/main.rs* that calls logic that lives in +*src/lib.rs*. With that structure, integration tests *can* test the library +crate by using `extern crate` to cover the important functionality, and if that +works, the small amount of code in *src/main.rs* will work as well and does not +need to be tested. ## Summary From 5e49128509796f9eb684d8cd450a5c1a6931155e Mon Sep 17 00:00:00 2001 From: "Carol (Nichols || Goulding)" Date: Sun, 27 Nov 2016 10:36:50 -0500 Subject: [PATCH 13/18] Fix error about what the eq and ne macros actually do Thank you @daschl!!! --- src/ch11-01-writing-tests.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/ch11-01-writing-tests.md b/src/ch11-01-writing-tests.md index bed1ff8ae0..c76644f839 100644 --- a/src/ch11-01-writing-tests.md +++ b/src/ch11-01-writing-tests.md @@ -165,13 +165,13 @@ is a custom message that you'd like to be added to the failure message. The macros expand to logic similar to this: ```rust,ignore -// assert_eq -if left_val == right_val { +// assert_eq! - panic if the values aren't equal +if left_val != right_val { panic!("your optional custom message") } -// assert_ne -if left_val != right_val { +// assert_ne! - panic if the values are equal +if left_val == right_val { panic!("your optional custom message") } ``` From 316db3e56d26a17aa9b5fc02fc38e443d5e4631c Mon Sep 17 00:00:00 2001 From: "Carol (Nichols || Goulding)" Date: Sun, 27 Nov 2016 10:54:16 -0500 Subject: [PATCH 14/18] Add a paragraph about needing PartialEq and Debug Also make the pseudocode a bit more accurate --- src/ch11-01-writing-tests.md | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/src/ch11-01-writing-tests.md b/src/ch11-01-writing-tests.md index c76644f839..0dc9bfc3e0 100644 --- a/src/ch11-01-writing-tests.md +++ b/src/ch11-01-writing-tests.md @@ -167,12 +167,22 @@ macros expand to logic similar to this: ```rust,ignore // assert_eq! - panic if the values aren't equal if left_val != right_val { - panic!("your optional custom message") + panic!( + "assertion failed: `(left == right)` (left: `{:?}`, right: `{:?}`): {}" + left_val, + right_val, + optional_custom_message + ) } // assert_ne! - panic if the values are equal if left_val == right_val { - panic!("your optional custom message") + panic!( + "assertion failed: `(left != right)` (left: `{:?}`, right: `{:?}`): {}" + left_val, + right_val, + optional_custom_message + ) } ``` @@ -201,6 +211,17 @@ The two arguments to `assert_eq!` are named "left" and "right" rather than "expected" and "actual"; the order of the value that comes from your code and the value hardcoded into your test isn't important. +Since these macros use the operators `==` and `!=` and print the values using +debug formatting, the values being compared must implement the `PartialEq` and +`Debug` traits. Types provided by Rust implement these traits, but for structs +and enums that you define, you'll need to add `PartialEq` in order to be able +to assert that values of those types are equal or not equal and `Debug` in +order to be able to print out the values in the case that the assertion fails. +Because both of these traits are derivable traits that we mentioned in Chapter +5, usually this is as straightforward as adding the `#[derive(PartialEq, +Debug)]` annotation to your struct or enum definition. See Appendix C for more +details about these and other derivable traits. + ## Test for failure with `should_panic` We can invert our test's failure with another attribute: `should_panic`. This From e3554b774b5e3071e8c5fde74564d618e310b9ab Mon Sep 17 00:00:00 2001 From: "Carol (Nichols || Goulding)" Date: Sun, 27 Nov 2016 11:10:23 -0500 Subject: [PATCH 15/18] Show test name matching works for module names too --- src/ch11-02-running-tests.md | 62 ++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/src/ch11-02-running-tests.md b/src/ch11-02-running-tests.md index 4800b26d71..be1208273b 100644 --- a/src/ch11-02-running-tests.md +++ b/src/ch11-02-running-tests.md @@ -112,6 +112,68 @@ test add_two_and_two ... ok test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured ``` +Module names become part of the test name, so module names can be used in a +similar way to run just the tests for a particular module. For example, if our +code was organized into a module named `adding` and a module named +`subtracting` with tests in each: + +```rust +mod adding { + #[test] + fn add_two_and_two() { + assert_eq!(4, 2 + 2); + } + + #[test] + fn add_three_and_two() { + assert_eq!(5, 3 + 2); + } + + #[test] + fn one_hundred() { + assert_eq!(102, 100 + 2); + } +} + +mod subtracting { + #[test] + fn subtract_three_and_two() { + assert_eq!(1, 3 - 2); + } +} +``` + +Running `cargo test` will run all of the tests, and the module names will +appear in the test names in the output: + +```text +$ cargo test + Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs + Running target/debug/deps/adder-d84f1c6cb24adeb4 + +running 4 tests +test adding::add_two_and_two ... ok +test adding::add_three_and_two ... ok +test subtracting::subtract_three_and_two ... ok +test adding::one_hundred ... ok +``` + +Running `cargo test adding` would run just the tests in that module and not any +of the tests in the subtracting module: + +```text +$ cargo test adding + Finished debug [unoptimized + debuginfo] target(s) in 0.0 secs + Running target/debug/deps/adder-d84f1c6cb24adeb4 + +running 3 tests +test adding::add_three_and_two ... ok +test adding::one_hundred ... ok +test adding::add_two_and_two ... ok + +test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured +``` + ### Ignore Some Tests Unless Specifically Requested Sometimes a few specific tests can be very time-consuming to execute, so during From d242dc44a719bfe370d037bd859830dde37625b3 Mon Sep 17 00:00:00 2001 From: "Carol (Nichols || Goulding)" Date: Sun, 27 Nov 2016 11:26:48 -0500 Subject: [PATCH 16/18] Adding some listing numbers --- src/ch11-01-writing-tests.md | 15 +++++++++++++-- src/ch11-02-running-tests.md | 12 ++++++++++-- src/ch11-03-test-organization.md | 24 ++++++++++++++++++------ 3 files changed, 41 insertions(+), 10 deletions(-) diff --git a/src/ch11-01-writing-tests.md b/src/ch11-01-writing-tests.md index 0dc9bfc3e0..fd8e3f35b0 100644 --- a/src/ch11-01-writing-tests.md +++ b/src/ch11-01-writing-tests.md @@ -229,7 +229,7 @@ is useful when we want to test that calling a particular function will cause an error. For example, let's test something that we know will panic from Chapter 8: attempting to create a slice using range syntax with byte indices that aren't on character boundaries. Add the `#[should_panic]` attribute before the -function like the `#[test]` attribute: +function like the `#[test]` attribute, as shown in Listing 11-1: Filename: src/lib.rs @@ -242,6 +242,10 @@ fn slice_not_on_char_boundaries() { } ``` + +Listing 11-1: A test expecting a `panic!` + + This test will succeed, since the code panics and we said that it should. If this code happened to run and did not cause a `panic!`, this test would fail. @@ -249,7 +253,8 @@ this code happened to run and did not cause a `panic!`, this test would fail. didn't fail for a different reason than the one you were expecting. To help with this, an optional `expected` parameter can be added to the `should_panic` attribute. The test harness will make sure that the failure message contains -the provided text. A safer version of the example above would be: +the provided text. A safer version of Listing 11-1 would be the following, in +Listing 11-2: Filename: src/lib.rs @@ -262,6 +267,12 @@ fn slice_not_on_char_boundaries() { } ``` + + + +Listing 11-2: A test expecting a `panic!` with a particular message + + Try on your own to see what happens when a `should_panic` test panics but doesn't match the expected message: cause a `panic!` that happens for a different reason in this test, or change the expected panic message to diff --git a/src/ch11-02-running-tests.md b/src/ch11-02-running-tests.md index be1208273b..abb5b6ddc0 100644 --- a/src/ch11-02-running-tests.md +++ b/src/ch11-02-running-tests.md @@ -48,7 +48,7 @@ working on code in a particular area, you might want to only run the tests having to do with that code. `cargo test` takes an argument that allows you to only run certain tests, specified by name. -Let's create three tests with the following names: +Let's create three tests with the following names as shown in Listing 11-3: Filename: src/lib.rs @@ -69,6 +69,10 @@ fn one_hundred() { } ``` + +Listing 11-3: Three tests with a variety of names + + Running with different arguments will run different subsets of the tests. No arguments, as we've already seen, runs all the tests: @@ -115,7 +119,7 @@ test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured Module names become part of the test name, so module names can be used in a similar way to run just the tests for a particular module. For example, if our code was organized into a module named `adding` and a module named -`subtracting` with tests in each: +`subtracting` with tests in each, as in Listing 11-4: ```rust mod adding { @@ -143,6 +147,10 @@ mod subtracting { } ``` + +Listing 11-4: Tests in two modules named `adding` and `subtracting` + + Running `cargo test` will run all of the tests, and the module names will appear in the test names in the output: diff --git a/src/ch11-03-test-organization.md b/src/ch11-03-test-organization.md index 3bc1578ed6..25bb8dd976 100644 --- a/src/ch11-03-test-organization.md +++ b/src/ch11-03-test-organization.md @@ -61,7 +61,7 @@ this is a common use of globs. Up until now in this chapter, we've been writing tests in our `adder` project that don't actually call any code we've written. Let's change that now! In *src/lib.rs*, place this `add_two` function and `tests` module that has a test -function to exercise the code: +function to exercise the code, as shown in Listing 11-5: Filename: src/lib.rs @@ -81,6 +81,10 @@ mod tests { } ``` + +Listing 11-5: Testing the function `add_two` in a child `tests` module + + Notice in addition to the test function, we also added `use add_two;` within the `tests` module. This brings the code we want to test into the scope of the inner `tests` module, just like we'd need to do for any inner module. If we run @@ -115,8 +119,8 @@ everything into the `test` module scope at once. There's controversy within the testing community about whether you should write unit tests for private functions or not. Regardless of which testing ideology you adhere to, Rust does allow you to test private functions due to the way -that the privacy rules work. Consider this code with the private function -`internal_adder`: +that the privacy rules work. Consider the code in Listing 11-6 with the private +function `internal_adder`: Filename: src/lib.rs @@ -140,6 +144,10 @@ mod tests { } ``` + +Listing 11-6: Testing a private function + + Because tests are just Rust code and the `tests` module is just another module, we can import and call `internal_adder` in a test just fine. If you don't think private functions should be tested, there's nothing in Rust that will compel @@ -159,9 +167,9 @@ Cargo has support for integration tests in the *tests* directory. If you make one and put Rust files inside, Cargo will compile each of the files as an individual crate. Let's give it a try! -First, make a *tests* directory at the top level of your project directory, next -to *src*. Then, make a new file, *tests/integration_test.rs*, and put this -inside: +First, make a *tests* directory at the top level of your project directory, +next to *src*. Then, make a new file, *tests/integration_test.rs*, and put the +code in Listing 11-7 inside: Filename: tests/integration_test.rs @@ -174,6 +182,10 @@ fn it_adds_two() { } ``` + +Listing 11-7: An integration test of a function in the `adder` crate + + We now have `extern crate adder` at the top, which we didn't need in the unit tests. Each test in the `tests` directory is an entirely separate crate, so we need to import our library into each of them. This is also why `tests` is a From b45eae6b86a75b0ba565710b39a65cc0f7a84ea4 Mon Sep 17 00:00:00 2001 From: "Carol (Nichols || Goulding)" Date: Mon, 28 Nov 2016 09:09:33 -0500 Subject: [PATCH 17/18] Change colon to period --- src/ch11-00-testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ch11-00-testing.md b/src/ch11-00-testing.md index 233093b7a5..1e4462d4e0 100644 --- a/src/ch11-00-testing.md +++ b/src/ch11-00-testing.md @@ -17,7 +17,7 @@ implement and compile that function, and Rust can do all the type checking and borrow checking that we've seen it's capable of doing. What Rust *can't* check for us is that we've implemented this function to return the argument plus two and not the argument plus 10 or the argument minus 50! That's where tests come -in: we can write tests that, for example, pass `3` to the `add_two` function +in. We can write tests that, for example, pass `3` to the `add_two` function and check that we get `5` back. We can run the tests whenever we make changes to our code to make sure we didn't change any existing behavior from what the tests specify it should be. From 093935fb9063108b495f2ceca890cd81f8594423 Mon Sep 17 00:00:00 2001 From: "Carol (Nichols || Goulding)" Date: Mon, 28 Nov 2016 11:39:34 -0500 Subject: [PATCH 18/18] More robust, not safer --- src/ch11-01-writing-tests.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ch11-01-writing-tests.md b/src/ch11-01-writing-tests.md index fd8e3f35b0..6da75b8346 100644 --- a/src/ch11-01-writing-tests.md +++ b/src/ch11-01-writing-tests.md @@ -253,8 +253,8 @@ this code happened to run and did not cause a `panic!`, this test would fail. didn't fail for a different reason than the one you were expecting. To help with this, an optional `expected` parameter can be added to the `should_panic` attribute. The test harness will make sure that the failure message contains -the provided text. A safer version of Listing 11-1 would be the following, in -Listing 11-2: +the provided text. A more robust version of Listing 11-1 would be the +following, in Listing 11-2: Filename: src/lib.rs