Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

storage: avoid per-kv allocations during consistency checks #29419

Merged
merged 1 commit into from
Aug 31, 2018
Merged

storage: avoid per-kv allocations during consistency checks #29419

merged 1 commit into from
Aug 31, 2018

Commits on Aug 31, 2018

  1. storage: avoid per-kv allocations during consistency checks 

    I noticed in an `alloc_objects` heap profile on a 3-node cluster
restoring tpcc that more than 46% of all allocations were made in
`computeChecksumPostApply`. Specifically, these allocations were all
made in `Replica.sha512`. 28% of allocations were due to protobuf
marshaling of `hlc.LegacyTimestamp`. The other 18% was in `encoding/binary.Write`.

This removes both of these sources of per-key allocations. The first
allocation was avoided by sharing a byte buffer across protobuf marshals.
The second was avoided by removing the call to `binary.Write` (see
golang/go#27403). I confirmed that this is no
longer an issue by looking at heap profiles from before and after in a test
that performed a consistency check.

I plan to follow up on golang/go#27403 and
search for any other offenders in our codebase. I already see a few.

Release note: None
    @nvanbenschoten
    nvanbenschoten committed Aug 31, 2018
    Configuration menu
    Copy the full SHA
    a858aeb View commit details
    Browse the repository at this point in the history