Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

store/cachekv, x/bank/types: algorithmically fix pathologically slow code #8719

Merged
merged 4 commits into from
Feb 27, 2021
Merged

store/cachekv, x/bank/types: algorithmically fix pathologically slow code #8719

merged 4 commits into from
Feb 27, 2021

Commits on Feb 27, 2021

  1. store/cachekv, x/bank/types: algorithmically fix pathologically slow … 

    …code

After continuously profiling InitGensis with 100K accounts, it showed
pathologically slow code, that was the result of a couple of patterns:
* Unconditional and not always necessary map lookups
* O(n^2) sdk.AccAddressFromBech32 retrievals when the code is expensive,
during a quicksort

The remedy involved 4 parts:
* O(n) sdk.AccAddressFromBech32 invocations, down from O(n^2) in the quicksort
* Only doing map lookups when the domain key check has passed
* Using a black magic compiler technique of the map clearing idiom
* Zero allocation []byte<->string conversion

With 100K accounts, this brings InitGenesis down to ~6min, instead of
20+min, it reduces the sort code from ~7sec down to 50ms.

Also some simple benchmark reflect the change:
```shell
name                    old time/op    new time/op    delta
SanitizeBalances500-8     19.3ms ±10%     1.5ms ± 5%  -92.46%  (p=0.000 n=20+20)
SanitizeBalances1000-8    41.9ms ± 8%     3.0ms ±12%  -92.92%  (p=0.000 n=20+20)

name                    old alloc/op   new alloc/op   delta
SanitizeBalances500-8     9.05MB ± 6%    0.56MB ± 0%  -93.76%  (p=0.000 n=20+18)
SanitizeBalances1000-8    20.2MB ± 3%     1.1MB ± 0%  -94.37%  (p=0.000 n=20+19)

name                    old allocs/op  new allocs/op  delta
SanitizeBalances500-8      72.4k ± 6%      4.5k ± 0%  -93.76%  (p=0.000 n=20+20)
SanitizeBalances1000-8      162k ± 3%        9k ± 0%  -94.40%  (p=0.000 n=20+20)
```

The CPU profiles show the radical change as per
#7766 (comment)

Later on, we shall do more profiling and fixes but for now this brings
down the run-time for InitGenesis.

Fixes #7766
    @odeke-em
    odeke-em committed Feb 27, 2021
    Configuration menu
    Copy the full SHA
    c19be27 View commit details
    Browse the repository at this point in the history

  2. Update x/bank/types/balance.go

    Alessio Treglia authored Feb 27, 2021
    Configuration menu
    Copy the full SHA
    c78329b View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into algorithmically-fix-slow-InitGenesis

    @odeke-em
    odeke-em authored Feb 27, 2021
    Configuration menu
    Copy the full SHA
    76331ec View commit details
    Browse the repository at this point in the history

  4. CHANGELOG: add entry

    @odeke-em
    odeke-em committed Feb 27, 2021
    Configuration menu
    Copy the full SHA
    7828624 View commit details
    Browse the repository at this point in the history