vm: store rkyv-serialized contracts in rocksdb, not borsh #10785
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nagisa
force-pushed
the
replaces-compiled-contracts-with-rkyv
branch
from
3e2da90
to
f645ef6
Compare
Borsh is not an appropriate serialization format for a couple of reasons: 1. It is pretty conservative and defensive about the data it handles -- for instance it will conservatively grow the vector it is deserializing which makes an otherwise O(n) operation an O(n^2) one. This makes sense as a tradeoff with untrusted data where an allocation of $TOOMANYBYTES would crash the program. For us this is not a relevant property and causes a significant slowdown at the vector sizes we're dealing with. 2. The data is really just a one big vector of bytes that we have read out from the database. Borsh APIs kinda force this vector to be reallocated no matter what, while ideally we'd want just a pointer/slice to said bytes. rkyv serves these needs pretty well out of the box, although we can also consider some less fancy ways for (de)serialization if we move to the filesystem proper.
nagisa
force-pushed
the
replaces-compiled-contracts-with-rkyv
branch
from
f645ef6
to
9816df3
Compare
|
#10791 is a better approach. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Borsh is not an appropriate serialization format for a couple of reasons:
rkyv serves these needs pretty well out of the box, although we can also consider some less fancy ways for (de)serialization if we move to the filesystem proper.