Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Execute Benchmarks for 0, 1, 10, 100, 1k, 10k Rows #299

Merged
merged 1 commit into from
Apr 23, 2016
Merged

Execute Benchmarks for 0, 1, 10, 100, 1k, 10k Rows #299

merged 1 commit into from
Apr 23, 2016

Conversation

killercup
Copy link
Member

This moves benchmark code into macros to easily abstract over the number of rows to benchmark.

Because you can't use macros in function name position, we have to write each benchmark name ourselves. (I tried moving the fns into a named mod, but there were some visibility issues and I gave up on that.)

The final output looks like this (the actual times are totally screwed since I'm doing other stuff—incl. watching @sgrif's stream—at the same time):

running 24 tests
test bench_medium_complex_query_selecting_10_000_rows       ... bench:  13,193,995 ns/iter (+/- 7,167,273)
test bench_medium_complex_query_selecting_10_000_rows_boxed ... bench:  13,001,051 ns/iter (+/- 4,904,166)
test bench_medium_complex_query_selecting__1_000_rows       ... bench:   1,113,618 ns/iter (+/- 560,993)
test bench_medium_complex_query_selecting__1_000_rows_boxed ... bench:   1,162,539 ns/iter (+/- 735,696)
test bench_medium_complex_query_selecting____100_rows       ... bench:     159,769 ns/iter (+/- 62,733)
test bench_medium_complex_query_selecting____100_rows_boxed ... bench:     170,054 ns/iter (+/- 107,642)
test bench_medium_complex_query_selecting_____10_rows       ... bench:      60,059 ns/iter (+/- 29,901)
test bench_medium_complex_query_selecting_____10_rows_boxed ... bench:      71,926 ns/iter (+/- 36,442)
test bench_medium_complex_query_selecting______0_rows       ... bench:       4,639 ns/iter (+/- 4,432)
test bench_medium_complex_query_selecting______0_rows_boxed ... bench:      13,534 ns/iter (+/- 13,648)
test bench_medium_complex_query_selecting______1_row        ... bench:      48,920 ns/iter (+/- 27,940)
test bench_medium_complex_query_selecting______1_row_boxed  ... bench:      56,033 ns/iter (+/- 21,060)
test bench_trivial_query_selecting_10_000_rows              ... bench:   6,637,456 ns/iter (+/- 10,654,968)
test bench_trivial_query_selecting_10_000_rows_boxed        ... bench:   6,489,666 ns/iter (+/- 9,187,709)
test bench_trivial_query_selecting__1_000_rows              ... bench:     771,187 ns/iter (+/- 4,666,322)
test bench_trivial_query_selecting__1_000_rows_boxed        ... bench:     652,489 ns/iter (+/- 1,172,967)
test bench_trivial_query_selecting____100_rows              ... bench:      71,229 ns/iter (+/- 87,046)
test bench_trivial_query_selecting____100_rows_boxed        ... bench:      61,548 ns/iter (+/- 81,419)
test bench_trivial_query_selecting_____10_rows              ... bench:       8,095 ns/iter (+/- 4,756)
test bench_trivial_query_selecting_____10_rows_boxed        ... bench:      10,789 ns/iter (+/- 11,514)
test bench_trivial_query_selecting______0_rows              ... bench:       1,429 ns/iter (+/- 1,198)
test bench_trivial_query_selecting______0_rows_boxed        ... bench:       6,341 ns/iter (+/- 15,519)
test bench_trivial_query_selecting______1_row               ... bench:       3,340 ns/iter (+/- 3,219)
test bench_trivial_query_selecting______1_row_boxed         ... bench:       5,742 ns/iter (+/- 2,939)

test result: ok. 0 passed; 0 failed; 0 ignored; 24 measured

This is pretty basic right now, as it's only the set of queries that were used before. Might be cool to extend this, and/or plot this to see asymptotical behaviour.

@sgrif
Copy link
Member

sgrif commented Apr 23, 2016

We really need to make concat_idents! useful. XD

sgrif
sgrif reviewed Apr 23, 2016
View reviewed changes
diesel_tests/tests/bench.rs
@@ -1,5 +1,8 @@
#![feature(custom_derive, plugin, custom_attribute, test)]
#![plugin(diesel_codegen, dotenv_macros)]

#![allow(non_snake_case)] // benchmark names contain several underscores to align numbers
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you just mention this in the commit message instead of a code comment? 😄

@killercup
Copy link
Member Author

Yep, this is actually rust-lang/rust#12249 from feb 2014 😄

@killercup killercup force-pushed the feature/benchmarks-factor-10 branch from e6b7356 to 0e0a650 Compare April 23, 2016 18:26
@mfpiccolo
Copy link
Contributor

Nice work @killercup. 😄

@killercup
Copy link
Member Author

killercup commented Apr 23, 2016 •
edited
Loading

(Comment addressed)

(Edit: Now with less typos.)

@killercup killercup force-pushed the feature/benchmarks-factor-10 branch from 0e0a650 to 73d95f8 Compare April 23, 2016 18:27
@killercup
Execute Benchmarks for 0, 1, 10, 100, 1k, 10k Rows
72ed447 
This moves benchmark code into macros to easily abstract over the number
of rows to benchmark. Because you can't use macros in function name
position, we have to write each benchmark name ourselves. (I tried
moving the fns into a named mod, but there were some visibility issues
and I gave up on that.)

For the numbers in the benchmark names to align, they are padded with
several underscores. For this to not print warning,
`#![allow(non_snake_case)]` was added.
@killercup killercup force-pushed the feature/benchmarks-factor-10 branch from 73d95f8 to 72ed447 Compare April 23, 2016 18:28
@sgrif
Copy link
Member

sgrif commented Apr 23, 2016

Thanks! ❤️

@sgrif sgrif merged commit 77ee50e into diesel-rs:master Apr 23, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@killercup @sgrif @mfpiccolo