Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[R4R][Feature]: Improve trie prefetch #952

Merged
merged 11 commits into from
Jul 15, 2022

Commits on Jul 7, 2022

  1. trie prefetcher for From/To address in advance

    We found that trie prefetch could be not fast enough, especially trie prefetch of
    the outer big state trie tree.
    Instead of do trie prefetch until a transaction is finalized, we could do trie prefetch
    in advance. Try to prefetch the trie node of the From/To accounts, since their root hash
    are most likely to be changed.
    setunapo committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    8fa25d7 View commit details
    Browse the repository at this point in the history
  2. Parallel TriePrefetch for large trie update.

    Currently, we create a subfetch for each account address to do trie prefetch. If the address
    has very large state change, trie prefetch could be not fast enough, e.g. a contract modified
    lots of KV pair or a large number of account's root hash is changed in a block.
    
    With this commit, there will be children subfetcher created to do trie prefetch in parallell if
    the parent subfetch's workload exceed the threshold.
    setunapo committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    70f7d9e View commit details
    Browse the repository at this point in the history
  3. some improvemnts of parallel trie prefetch implementation

    1.childrenLock is removed, since it is not necessary
      APIs of triePrefetcher is not thread safe, they should be used sequentially.
      A prefetch will be interrupted by trie() or clos(), so we only need mark it as
      interrupted and check before call scheduleParallel to avoid the concurrent access to paraChildren
    2.rename subfetcher.children to subfetcher.paraChildren
    3.use subfetcher.pendingSize to replace totalSize & processedIndex
    4.randomly select the start child to avoid always feed the first one
    5.increase threshold and capacity to avoid create too many child routine
    setunapo committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    c12564b View commit details
    Browse the repository at this point in the history
  4. fix review comments

    ** nil check refine
    ** create a separate routine for From/To prefetch, avoid blocking the cirtical path
    setunapo committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    c21562d View commit details
    Browse the repository at this point in the history
  5. remove the interrupt member

    setunapo committed Jul 7, 2022
    Configuration menu
    Copy the full SHA
    1f292f4 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    ced98db View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2022

  1. some changes to triePrefetcher

    ** remove the abortLoop, move the subfetcher abort operation into mainLoop
       since we want to make subfetcher's create & schedule & abort within a loop to
       avoid concurrent access locks.
    
    ** no wait subfetcher's term signal in abort()
       it could speed up the close by closing subfetcher concurrently.
       we send stop signnal to all subfetchers in burst and wait their term signal later.
    setunapo committed Jul 8, 2022
    Configuration menu
    Copy the full SHA
    95abbf8 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    065c0b8 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    93cfb3d View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2022

  1. update parallel trie prefetcher configuration

    tested with different combination of parallelTriePrefetchThreshold & parallelTriePrefetchCapacity,
    found the most efficient configure could be:
      parallelTriePrefetchThreshold = 10
      parallelTriePrefetchCapacity  = 20
    setunapo committed Jul 12, 2022
    Configuration menu
    Copy the full SHA
    1c2d378 View commit details
    Browse the repository at this point in the history

Commits on Jul 14, 2022

  1. Configuration menu
    Copy the full SHA
    84d281f View commit details
    Browse the repository at this point in the history