Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add memmem implementation, upgrade to Rust 2018, bump MSRV to Rust 1.41.1 #82

Merged
merged 4 commits into from
Apr 30, 2021
Merged

add memmem implementation, upgrade to Rust 2018, bump MSRV to Rust 1.41.1 #82

merged 4 commits into from
Apr 30, 2021

Commits on Apr 30, 2021

  1. edition: migrate to Rust 2018 

    This was long overdue, but I had been busy with other things.

This also bumps the MSRV to Rust 1.41, which is what is currently in
Debian stable.
    @BurntSushi
    BurntSushi committed Apr 30, 2021
    Configuration menu
    Copy the full SHA
    2871a08 View commit details
    Browse the repository at this point in the history

  2. organization: prepare for bringing in memmem 

    This basically bumps down all of the memchr tests/benchmarks into a
sub-module in order to make room for tests/benchmarks for other APIs
like memmem.
    @BurntSushi
    BurntSushi committed Apr 30, 2021
    Configuration menu
    Copy the full SHA
    78cc45d View commit details
    Browse the repository at this point in the history

  3. api: add memmem implementation, initially from bstr 

    This commit primarily adds vectorized substring search routines in
a new memmem sub-module. They were originally taken from bstr, but
heavily modified to incorporate a variant of the "generic SIMD"
algorithm[1]. The main highlights:

* We guarantee `O(m + n)` time complexity and constant space
  complexity.
* Two-Way is the primary implementation that can handle all cases.
* Vectorized variants handle a number of common cases.
* Vectorized code uses a heuristic informed by a frequency background
  distribution of bytes, originally devised inside the regex crate.
  This makes it more likely that searching will spend more time in the
  fast vector loops.

While adding memmem to this crate is perhaps a bit of a scope increase,
I think it fits well. It also puts a core primitive, substring
search, very low in the dependency DAG and therefore making it widely
available. For example, it is intended to use these new routines in the
regex, aho-corasick and bstr crates.

This commit does a number of other things, mainly as a result of
convenience. It drastically improves test coverage for substring search
(as compared to what bstr had), completely overhauls the benchmark suite
to make it more comprehensive and adds `cargo fuzz` support for all API
items in the crate.

Closes #58, Closes #72

[1] - http://0x80.pl/articles/simd-strfind.html#algorithm-1-generic-simd
    @BurntSushi
    BurntSushi committed Apr 30, 2021
    Configuration menu
    Copy the full SHA
    dbdf0a3 View commit details
    Browse the repository at this point in the history

  4. bench: add run outputs 

    This makes it easy to link to benchmarks when someone asks, but also
serves as a good way to archive benchmark data at defined points for
comparison later.

We also make a (feeble) attempt at putting a "pretty" version of a
subset of benchmarks in the README of each run directory.
    @BurntSushi
    BurntSushi committed Apr 30, 2021
    Configuration menu
    Copy the full SHA
    d7c3f23 View commit details
    Browse the repository at this point in the history