Skip to content

Commit

Permalink
Problem: memiavl restore not fast enough (backport crypto-org-chain#1241
Browse files Browse the repository at this point in the history
)

* Problem: memiavl restore not fast enough

Solution:
- increase channel buffer size to improve parallisation

it reduce restore time by 44% when testing with testnet, thanks @yzang2019 for sharing the finding.

* Update CHANGELOG.md

Signed-off-by: yihuang <huang@crypto.com>

* fix unit test

---------

Signed-off-by: yihuang <huang@crypto.com>
  • Loading branch information
yihuang committed Nov 15, 2023
1 parent a11de9d commit 01ca39d
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 1 deletion.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

## Unreleased

- [#1241](https://github.com/crypto-org-chain/cronos/pull/1241) Improve parallelization of memiavl restoration.

### State Machine Breaking

- [cronos#695](https://github.com/crypto-org-chain/cronos/pull/695) Implement ADR-007, generic events format with indexed params.
Expand Down
4 changes: 3 additions & 1 deletion memiavl/import.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ import (
sdkerrors "github.com/cosmos/cosmos-sdk/types/errors"
)

const NodeChannelBuffer = 2048

// Import restore memiavl db from state-sync snapshot stream
func Import(
dir string, height uint64, format uint32, protoReader protoio.Reader,
Expand Down Expand Up @@ -116,7 +118,7 @@ type TreeImporter struct {
}

func NewTreeImporter(dir string, version int64) *TreeImporter {
nodesChan := make(chan *iavl.ExportNode)
nodesChan := make(chan *iavl.ExportNode, NodeChannelBuffer)
quitChan := make(chan error)
go func() {
defer close(quitChan)
Expand Down

0 comments on commit 01ca39d

Please sign in to comment.