-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf: Add a map coins struct to speedup bank genesis #15764
Conversation
How about store the coins in a slice first, then add them in one call? We can use sth like hash map as an implementation detail of Add? |
Can you add a test or benchmark that validates your claims and prevents regressions? |
@08d2 tests are added for MapCoins, in particular every AddCoins test is applied there. Also covered by the Bank InitGenesis tests. Benchmarked in larger context, but I can add one here sure. (The asymptotics are clear though) @yihuang I'm not sure how to do that without making sdk.Coins an enum type over Map / []sdk.Coin. Unless you mean making a bulk add operation, but that would require more heap allocations in the bank InitGenesis context! |
Its a pretty significant speedup:
I tried making benchmarks with more non-intersecting coins, and they wouldn't finish in reasonable timeframes with sdk.Coins due to the N^2 behavior. (Worked fine with mapcoins) The main remaining mapcoins bottleneck is heap allocations for int adds, which could also be removed, once there is a mutative int add operation |
I see, makes sense. |
Would you be opposed to adding this one API in this PR since it's a very hot path? |
I'm happy to add that in! Alright if I do so in a separate PR though? Currently doing other things to speedup Genesis work / would prefer this get through on its own :) |
Sure thing 👍 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are a couple small cleanup opportunities here, but no big changes needed.
// map coins is a map representation of sdk.Coins | ||
// intended solely for use in bulk additions. | ||
// All serialization and iteration should be done after conversion to sdk.Coins. | ||
type MapCoins map[string]Int |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While this is clearly backed by a map, the name MapCoins
doesn't indicate the intent of the type. Maybe BulkCoins
or BulkAddCoins
would be clearer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, happy to switch to anything
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Merge queue setting changed
Merge queue setting changed
@ValarDragon can you run |
Description
This PR introduces a map coins struct, and uses it in the bank keepers genesis.
The point of this PR is to significantly speedup bulk add operations. Today every add requires a full heap copy of the operands being added together. (And forces addition to therefore be linear in both sizes)
So if were doing N additions, each containing O(1) independent coins, this takes O(N^2) time right now. This is quite expensive for some operations, such as InitGenesis. It causes about 1% of the time delay of an Osmosis InitGenesis (which is quite expensive)
This adds up quite fast! After this PR, the time-delay for AddCoins in bank genesis is negligible. (InitBalances is still time delay, understandably. However that was always less than the contribution than these AddCoins)
Author Checklist
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.
I have...
!
to the type prefix if API or client breaking changeCHANGELOG.md
Reviewers Checklist
All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.
I have...
!
in the type prefix if API or client breaking change