Introduce snapshot tests

[loiclec]

Pull Request

What does this PR do?

Introduce snapshot tests into milli, by using the insta crate. This implements the idea described by meilisearch/meilisearch#3360

See: insta.rs

Design

There is now a new file, snapshot_tests.rs, which is compiled only under #[cfg(test)]. It exposes the db_snap! macro, which is used to snapshot the content of a database.

When running cargo test, insta will check that the value of the current snapshot is the same as the previous one (on the file system). If they are the same, the test passes. If they are different, the test fails and you are asked to review the new snapshot to approve or reject it.

We don't want to save very large snapshots to the file system, because it will pollute the git repository and increase its size too much. Instead, we only save their md5 hashes under the name <snapshot_name>.hash.snap. There is a new environment variable called MILLI_TEST_FULL_SNAPS which can be set to true in order to also save the full content of the snapshot under the name <snapshot_name>.full.snap. However, snapshots with the extension .full.snap are never saved to the git repository.

Example

// In e.g. facets.rs
#[test]
fn my_test() {
    // create an index
    let index = TempIndex::new():
    index.add_documents(...);
    index.update_settings(|settings| ...);
    
    // then snapshot the content of one of its databases
    // the snapshot will be saved at the current folder under facets.rs/my_test/facet_id_string_docids.snap
    db_snap!(index, facet_id_string_docids);

    index.add_documents(...);   

    // we can also name the snapshot to ensure there is no conflict
    // this snapshot will be saved at facets.rs/my_test/updated/facet_id_string_docids.snap
    db_snap!(index, facet_id_string, docids, "updated");
    
    // and we can also use "inline" snapshots, which insert their content in the given string literal
    db_snap!(index, field_distributions, @"");
    // once the snapshot is approved, it will automatically get transformed to, e.g.:
    // db_snap!(index, field_distributions, @"
    // my_facet        21
    // other_field     3
    // ");
    
    // now let's add **many** documents
    index.add_documents(...);
    
    // because the snapshot is too big, its hash is saved instead
    // if the MILLI_TEST_FULL_SNAPS env variable is set to true, then the full snapshot will also be saved
    // at facets.rs/my_test/large/facet_id_string_docids.full.snap
    db_snap!(index, facet_id_string_docids, "large", @"5348bbc46b5384455b6a900666d2a502");
}

[Kerollmops]

It is pretty amazing the amount of lines of code that you removed by defining new helping methods on the TestIndex struct 🎉

Thank you very much for introducing insta.rs in milli, it looks amazing, I am impatient to learn more about it!

[irevoire]

That’s super noice, can’t wait to use it!

[loiclec]

Does anybody know why the tests are failing on Windows? 🤔

[Kerollmops]

Does anybody know why the tests are failing on Windows? 🤔

I have no clue. It looks like it is failing to infer types but ONLY on windows? Is it due to the new version of Rust? I reran it!

[Kerollmops]

Looks good to me, thank you very much!
bors merge

[bors]

Build succeeded:

• Run Rustfmt
• Tests on macos-latest
• Tests on ubuntu-18.04
• Tests on windows-latest