[Perf Experiment] Change FxHash to GxHash #127040
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rustbot
added
A-query-system
This comment has been minimized.
This comment has been minimized.
lqd
reviewed
Jun 27, 2024
| @@ -0,0 +1 @@ | |||
| hola1 | |||
|
Changes to the size of AST and/or HIR nodes. cc @nnethercote Some changes occurred in compiler/rustc_sanitizers cc @rust-lang/project-exploit-mitigations, @rcvalle Some changes occurred in coverage instrumentation. cc @Zalathar Some changes occurred in src/librustdoc/clean/types.rs cc @camelid Some changes occurred in engine.rs, potentially modifying the public API of Some changes occurred in compiler/rustc_codegen_cranelift cc @bjorn3 These commits modify the If this was unintentional then you should revert the changes before this PR is merged. Some changes occurred in coverage instrumentation. cc @Zalathar Some changes occurred to MIR optimizations cc @rust-lang/wg-mir-opt HIR ty lowering was modified cc @fmease Some changes occurred to the CTFE / Miri engine cc @rust-lang/miri Some changes occurred in exhaustiveness checking cc @Nadrieril Some changes occurred in match lowering cc @Nadrieril The list of allowed third-party dependencies may have been modified! You must ensure that any new dependencies have compatible licenses before merging. Some changes occurred in match checking cc @Nadrieril Some changes occurred in cfg and check-cfg configuration cc @Urgau Some changes occurred in coverage instrumentation. cc @Zalathar Some changes occurred to the core trait solver cc @rust-lang/initiative-trait-system-refactor Some changes occurred in compiler/rustc_codegen_gcc Some changes occurred to the CTFE / Miri engine cc @rust-lang/miri |
Remove build script
This comment was marked as outdated.
This comment was marked as outdated.
|
Awaiting bors try build completion. @rustbot label: +S-waiting-on-perf |
rustbot
added
the
S-waiting-on-perf
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Jun 27, 2024
[Perf Experiment] Change FxHash to GxHash I've done some experiments comparing hashing libraries that can be found in crates.io, and it while FxHash is faster than the default hasher, Gxhash is still about 10 times faster than fxhash. This is just with the default implementation, the library permits customization using CPU-specific optimizations. But we can ask the maintainers for that if we were to verify that the optimization is big enough.
|
Gxhash seems to be throughput optimized, while rustc-hash is latency optimized. For basically all places where we use FxHasher, latency is much more important than throughput as the hash input is relatively small. |
|
Also gxhash intentionally invokes undefined behaviour for performance :p |
|
The job Click to see the possible cause of the failure (guessed by this bot) |
|
💔 Test failed - checks-actions |
bors
added
S-waiting-on-author
|
Yeah that seems to be doing OOB reads and then masking off the OOB parts. That is clear UB. (You need to use "masked load" intrinsics instead that avoid accessing the OOB parts - they exist unstably in intrinsics::simd, not sure if Intel has them as vendor intrinsics.)
|
|
Just FYI, several people have spent a lot of time trying to improve on
On what inputs? As @bjorn3 said, rustc's hashing is dominated by tiny inputs, e.g. individual pointers or integers, and fxhash is optimized for those cases, to the point where it's something one one or two instructions per input. There's very little room for improvement. |
|
☔ The latest upstream changes (presumably #127111) made this pull request unmergeable. Please resolve the merge conflicts. |
|
I've done some more benchmarks, and it seems that, for small benchmarks (3 bytes, 10b, 30b and 100b) So, I'll test it this afternoon, and let's see if the math is mathing |
|
Some of these numbers looks suspicious, tbh. Not even regarding the differences between hashing algorithms themselves, but more the absolute values - I'm pretty sure you can't hash 3 bytes (much less 100 bytes) in 0.001ns. It might have been optimized out by the compiler entirely, or it might have hit the limits of your timer precision. It might be better to repeat the hashing e.g. 1000x and then measure that whole duration, rather than measuring just the hash of 3 bytes on its own. But trying e.g. FNV instead of FxHash could be interesting, I'm not sure if such a trivial algo has been tried yet. |
|
|
FNV hashes one byte at a time. FxHash can do a whole integer/word at a time. As I said above, this is very well trodden territory. |
|
Closing this since it hasn't had any movement in a month and it seems to just be a perf experiment, feel free to reopen if you are going to work on it again. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I've done some experiments comparing hashing libraries that can be found in crates.io, and it while FxHash is faster than the default hasher, Gxhash is still about 10 times faster than fxhash.
This is just with the default implementation, the library permits customization using CPU-specific optimizations. But we can ask the maintainers for that if we were to verify that the optimization is big enough.