-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[R4R][Feature]: Improve trie prefetch #952
[R4R][Feature]: Improve trie prefetch #952
Commits on Jul 7, 2022
-
trie prefetcher for From/To address in advance
We found that trie prefetch could be not fast enough, especially trie prefetch of the outer big state trie tree. Instead of do trie prefetch until a transaction is finalized, we could do trie prefetch in advance. Try to prefetch the trie node of the From/To accounts, since their root hash are most likely to be changed.
Configuration menu - View commit details
-
Copy full SHA for 8fa25d7 - Browse repository at this point
Copy the full SHA 8fa25d7View commit details -
Parallel TriePrefetch for large trie update.
Currently, we create a subfetch for each account address to do trie prefetch. If the address has very large state change, trie prefetch could be not fast enough, e.g. a contract modified lots of KV pair or a large number of account's root hash is changed in a block. With this commit, there will be children subfetcher created to do trie prefetch in parallell if the parent subfetch's workload exceed the threshold.
Configuration menu - View commit details
-
Copy full SHA for 70f7d9e - Browse repository at this point
Copy the full SHA 70f7d9eView commit details -
some improvemnts of parallel trie prefetch implementation
1.childrenLock is removed, since it is not necessary APIs of triePrefetcher is not thread safe, they should be used sequentially. A prefetch will be interrupted by trie() or clos(), so we only need mark it as interrupted and check before call scheduleParallel to avoid the concurrent access to paraChildren 2.rename subfetcher.children to subfetcher.paraChildren 3.use subfetcher.pendingSize to replace totalSize & processedIndex 4.randomly select the start child to avoid always feed the first one 5.increase threshold and capacity to avoid create too many child routine
Configuration menu - View commit details
-
Copy full SHA for c12564b - Browse repository at this point
Copy the full SHA c12564bView commit details -
** nil check refine ** create a separate routine for From/To prefetch, avoid blocking the cirtical path
Configuration menu - View commit details
-
Copy full SHA for c21562d - Browse repository at this point
Copy the full SHA c21562dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1f292f4 - Browse repository at this point
Copy the full SHA 1f292f4View commit details -
Configuration menu - View commit details
-
Copy full SHA for ced98db - Browse repository at this point
Copy the full SHA ced98dbView commit details
Commits on Jul 8, 2022
-
some changes to triePrefetcher
** remove the abortLoop, move the subfetcher abort operation into mainLoop since we want to make subfetcher's create & schedule & abort within a loop to avoid concurrent access locks. ** no wait subfetcher's term signal in abort() it could speed up the close by closing subfetcher concurrently. we send stop signnal to all subfetchers in burst and wait their term signal later.
Configuration menu - View commit details
-
Copy full SHA for 95abbf8 - Browse repository at this point
Copy the full SHA 95abbf8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 065c0b8 - Browse repository at this point
Copy the full SHA 065c0b8View commit details -
Configuration menu - View commit details
-
Copy full SHA for 93cfb3d - Browse repository at this point
Copy the full SHA 93cfb3dView commit details
Commits on Jul 12, 2022
-
update parallel trie prefetcher configuration
tested with different combination of parallelTriePrefetchThreshold & parallelTriePrefetchCapacity, found the most efficient configure could be: parallelTriePrefetchThreshold = 10 parallelTriePrefetchCapacity = 20
Configuration menu - View commit details
-
Copy full SHA for 1c2d378 - Browse repository at this point
Copy the full SHA 1c2d378View commit details
Commits on Jul 14, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 84d281f - Browse repository at this point
Copy the full SHA 84d281fView commit details