-
Notifications
You must be signed in to change notification settings - Fork 133
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Finality votes cache #116
Finality votes cache #116
Conversation
Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>
…ech/parity-bridges-common into remove-in-memory-storage
Why is it on top of #105 ? Is there a specific thing required from that PR? Maybe split it into two then or ping us to review #105 :) Reviewing "on top" PRs is really super hard (read: impossible) and after we squash-merge #105 there's going to be a mess with commits which also makes it a bit harder to grasp your intentions (by looking at individual commits). |
|
It should be ready now. This PR only adds commits >= finality cache. One last thing that I wanted to mention is that now PoA headers are mostly referenced by |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm!
/// Finality votes for given block. | ||
#[derive(RuntimeDebug, Decode, Encode)] | ||
#[cfg_attr(test, derive(Clone, Default, PartialEq))] | ||
pub struct FinalityVotes<Submitter> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Worth adding a custom Default
implementation to simplify creation.
* removeInMemoryStorage + extract Kovan stuff to runtime * removed comment from the future * remove redundant conversions * remove redundant `u8 as usize` * remove redundant `u8 as usize` * Update modules/ethereum/src/mock.rs Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com> * use hex-literal in kovan config * cargo fmt --all * extracted insert_header * cargo fmt --all * finality cache * cargo fmt --all * cargo fmt --all * impl Default for FinalityVotes Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com> Co-authored-by: Hernando Castano <castano.ha@gmail.com>
* removeInMemoryStorage + extract Kovan stuff to runtime * removed comment from the future * remove redundant conversions * remove redundant `u8 as usize` * remove redundant `u8 as usize` * Update modules/ethereum/src/mock.rs Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com> * use hex-literal in kovan config * cargo fmt --all * extracted insert_header * cargo fmt --all * finality cache * cargo fmt --all * cargo fmt --all * impl Default for FinalityVotes Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com> Co-authored-by: Hernando Castano <castano.ha@gmail.com>
attempt to solve (1) from #78 (comment)
on top of #105
This PR adds persistent finality votes cache. It doesn't change computational complexity of headers finalization algorithm, but instead significantly decreases current implementation overhead. I.e. if number of unfinalized headers is
N
, then we still need to cycle overN
headers, but instead of makingN
storage reads, we only need to perform limited number of reads (though at least one of these reads may read larger number of bytes than regular header - see below).We still need to introduce 'spanned finalization' (mentioned in #78 (comment)), but I think for now (I mean for initial start) the cache should be enough. On the same 1926 unfinalized Kovan headers from #78, I'm now getting
time(import(header#1926)) / time(import(header#1)) ~ 6.5
instead of previous~46.5
.The cache now works as follows:
N
is inserted, we're building structFinalityVotes
, that includes info about all unfinalized ancestors of this header (note that it isn't header itself, but some shortened version);N % FinalityVotesCachingInterval == 0
then we're inserting this structure into finality cache;N+1
is inserted, we will only need to read headerN
and this struct;N+2
we'll read two headers (N+1
,N
) and the same struct;N + FinalityVotesCachingInterval
we'll insert another entry into finality cache;N + FinalityVotesCachingInterval + 1
we'll again need to read only one header (N + FinalityVotesCachingInterval
and that struct).The cache itself may be improved later, e.g. by not inserting entries if finalization is regular (i.e. we do not have large delays) or by making bridge guess
FinalityVotesCachingInterval
on its own (i.e. we know number of validators and we know we need votes of 2/3 validators => so having2/3*len(validators) + regular_finalization_delay
unfinalized blocks may be a good criteria for enabling cache and initial caching interval).The cached struct has approximate size of
NUM_VOTED_VALIDATORS*28 + NUM_UNFINALIZED_HEADERS*81
. E.g. if 5 validators have authored 1000 unfinalized blocks, then the size would be ~80K. This is rather large storage entry, but: (1) it is much better than reading 1000 headers and (2) I'm not sure if we'll be able to get rid of this struct even in 'spanned finalization', because we'll need to cache computation results somewhere betweencontinue_finalization()
calls.Performance comparison on initial Kovan headers: before this PR vs after this PR (native execution, debug build).