Skip to content
This repository has been archived by the owner on Nov 6, 2020. It is now read-only.

ParityDB integration with Parity CLI #8388

Closed
wants to merge 3 commits into from

Conversation

sorpaas
Copy link
Collaborator

@sorpaas sorpaas commented Apr 12, 2018

cc #7865

This adds an experimental ParityDB integration with the CLI. If you want to test this, please use the kvdb-paritydb-cli branch (https://github.com/sorpaas/parity/tree/kvdb-paritydb-cli), which contains the merged change of #8371. Use feature flag paritydb, for example cargo run --features paritydb -- --base-path ~/.paritydb-test --config dev. Please remember to use a different base path as it will mess up with existing databases. With that branch, I'm at least able to use the dev chain with ParityDB, send transactions and mine blocks. Haven't done more testing yet.

Some current limitations of ParityDB:

  • Key length needs to be a fixed value (unlike rocksdb which can be a variable). So in this implementation I hard-set it to 64 bytes, postfix keys if they're less than this length.
  • No prefix-iteration functionality in ParityDB. So the current implementation uses a filter under the full iterator. Fortunately we're not using that functionality a lot.
  • No column functionality in ParityDB. So the current implementation just opens a separate DB for each column.

And several TODO tasks before we can consider this to be merge-able:

@sorpaas sorpaas added A3-inprogress ⏳ Pull request is in progress. No review needed at this stage. M4-core ⛓ Core client code / Rust. labels Apr 12, 2018
@sorpaas sorpaas added this to the 1.11 milestone Apr 12, 2018
@rphmeier
Copy link
Contributor

rphmeier commented Apr 12, 2018

Is the goal of this PR to replace rocksdb entirely with ParityDB?
I'm not that well-informed on the feature, but I thought ParityDB was meant to be used to store account RLP by address to be used for fast reads of state data, and nothing else.

As a an arbitrary key-value store, I'd imagine that rocksdb is superior.

@sorpaas
Copy link
Collaborator Author

sorpaas commented Apr 12, 2018

@rphmeier I would be glad to have some more clarifications on that as well, so I don't end up writing non-useful code. 😆

But anyway, for this PR, yes. One of the goal was to have a feature flag that replaces rocksdb with ParityDB. But most of the effort was in kvdb_paritydb which tries to create a KeyValueDB interface for ParityDB, so that part might still be useful for account storage.

@rphmeier
Copy link
Contributor

rphmeier commented Apr 12, 2018

I guess the main thing is that we want to both persist the actual trie nodes and blocks somewhere (a normal key-value store is good for that) but also enable efficient lookup of state objects like account RLP without having to undergo the log(n) cost of lookups in the trie. writes would go both into the KeyValueDB (full trie update) and the ParityDB (address-account pairs directly). I'm not sure how it works with pruning or reorgs.

@sorpaas
Copy link
Collaborator Author

sorpaas commented Apr 12, 2018

@rphmeier Okay I think I get what you mean. I think that still requires more refactoring before we add ParityDB dependency. So I'm going to close this PR for now. The code here is still useful as it is just a KVDB interface implementation, but I think it'll be better to get the refactoring done first.

@sorpaas sorpaas closed this Apr 12, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A3-inprogress ⏳ Pull request is in progress. No review needed at this stage. M4-core ⛓ Core client code / Rust.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants