Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chainHead: Add support for storage closest merkle descendant #14818 #1153

Merged
merged 21 commits into from
Sep 18, 2023
Merged

chainHead: Add support for storage closest merkle descendant #14818 #1153

merged 21 commits into from
Sep 18, 2023

Conversation

lexnv
Copy link
Contributor

@lexnv lexnv commented Aug 25, 2023 •
edited
Loading

This PR adds support for fetching the closest merkle value of some key.

Builds on top of

Migrates paritytech/substrate#14818 to the monorepo.
Closes: paritytech/substrate#14550
Closes: #1506

// @paritytech/subxt-team

@lexnv
git: Ignore target folder for main repo
d9728fb 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
Cargo: Update trie to github lexnv/expose_merkle_value
289732a 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
client: Expose merkle value on storageProvider trait
e51afac 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
primitives/backend: Expose merkle value on Backend trait
bc39b11 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
primitives/trie: Low level functions for reading the merkle value
d23ad73 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
primitives/trie: Implement closest merkle value on TrieBackendEssence..
fe12896 
and TrieBackend

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
client: Implement closest merkle value up until StorageProvider
cb7fd4d 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/storage: Query the storage for closest merkle value
766de69 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/tests: Check merkle values propagate to the chainHead_storage
d23e1c1 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv lexnv added A0-needs_burnin Pull request needs to be tested on a live validator node before merge. DevOps is notified via matrix R0-silent Changes should not be mentioned in any release notes labels Aug 25, 2023
@lexnv lexnv self-assigned this Aug 25, 2023
@paritytech-ci paritytech-ci requested review from a team August 28, 2023 09:45
@lexnv
Merge remote-tracking branch 'origin/master' into lexnv/chainhed_trie…
6493a10 
…_db_merkle

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
primitives: Adjust merkle value api
3607785 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
Update interface with merkle value changes
62f677a 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/storage: Adjust the returned values
b6dd92e 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
Merge remote-tracking branch 'origin/master' into lexnv/chainhed_trie…
445f225 
…_db_merkle
@lexnv
bench: Use trie github link
1e52670 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead/tests: Adjust testing
520c651 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead: Support merkle value types
ba184a0 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv
chainHead: Fix build
3294a1d 
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
This was referenced Sep 12, 2023
Merged
Merged
lexnv added a commit that referenced this pull request Sep 13, 2023
@lexnv @bkchr
Update trie-db version to 0.28.0 (#1522)
0bebc8a 
This PR updates:
- trie-db from 0.27.1 to 0.28.0
- trie-bench from 0.37.0 to 0.38.0 (deb-dependency)


While at it, also adapts the recorder to take into account the newly
added `TrieAccess::InlineValue`.

Needed by:
- #1153

@paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
@lexnv
Merge remote-tracking branch 'origin/master' into lexnv/chainhed_trie…
3e7a60d 
…_db_merkle

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
@lexnv lexnv requested review from jsdw and skunert September 13, 2023 16:11
taqtiqa-mark pushed a commit to subversive-upstream/polkadot-sdk-cumulus that referenced this pull request Sep 14, 2023
@lexnv @bkchr
Update trie-db version to 0.28.0 (#1522)
8c1c568 
This PR updates:
- trie-db from 0.27.1 to 0.28.0
- trie-bench from 0.37.0 to 0.38.0 (deb-dependency)


While at it, also adapts the recorder to take into account the newly
added `TrieAccess::InlineValue`.

Needed by:
- paritytech/polkadot-sdk#1153

@paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
skunert
skunert approved these changes Sep 15, 2023
View reviewed changes
Copy link
Contributor

@skunert skunert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks nice!

substrate/client/rpc-spec-v2/src/chain_head/tests.rs Outdated Show resolved Hide resolved
skunert

This comment was marked as duplicate.

Sign in to view

@lexnv @skunert
Update substrate/client/rpc-spec-v2/src/chain_head/tests.rs
cd0c0c1 
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
tadeohepperle
tadeohepperle approved these changes Sep 15, 2023
View reviewed changes
Copy link
Contributor

@tadeohepperle tadeohepperle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know too much about how substrate handles storage, but since the changes got merged into sp-trie and this PR mostly forwards to functions from sp-trie I think this should be all good. Good work!

substrate/client/db/src/lib.rs
key: &[u8],
) -> Result<Option<MerkleValue<B::Hash>>, Self::Error> {
self.state.closest_merkle_value(key)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: What does it mean here, if an Error is returned, vs. an Option is returned? Under what circumstances can errors happen?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, the Option tells us if there is descendant merkle value below the provided key, since the key may not be in the database and may not have any descendants.

On the other hand, the error is a way to propagate that something went wrong during this operation. Mainly on the trie-db side:

  • unable to fetch bytes from the database (invalid state root / incomplete data)
  • corrupted data (failing to decode the node)

Here would be the main place from which those errors originate

@lexnv lexnv merged commit 5d34664 into master Sep 18, 2023
9 checks passed
@lexnv lexnv deleted the lexnv/chainhed_trie_db_merkle branch September 18, 2023 10:54
@kiltbot kiltbot mentioned this pull request Oct 19, 2023
Closed
bgallois pushed a commit to duniter/duniter-polkadot-sdk that referenced this pull request Mar 25, 2024
@lexnv @bkchr
Update trie-db version to 0.28.0 (paritytech#1522)
7ffb644 
This PR updates:
- trie-db from 0.27.1 to 0.28.0
- trie-bench from 0.37.0 to 0.38.0 (deb-dependency)


While at it, also adapts the recorder to take into account the newly
added `TrieAccess::InlineValue`.

Needed by:
- paritytech#1153

@paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
bgallois pushed a commit to duniter/duniter-polkadot-sdk that referenced this pull request Mar 25, 2024
@lexnv @skunert
chainHead: Add support for storage closest merkle descendant #14818 (p…
a5a37f8 
…aritytech#1153)

This PR adds support for fetching the closest merkle value of some key.


Builds on top of
- paritytech/trie#199

Migrates paritytech/substrate#14818 to the
monorepo.
Closes: paritytech/substrate#14550
Closes: paritytech#1506

// @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 8, 2024
@boundless-forest @serban300
Move calculation relayer reward into `MessageDeliveryAndDispatchPayme…
658c379 
…nt` (paritytech#1153)

* Refactor logic

* Thanks svyatonik help, it compile

* Fix failed unit test

* Remove compile warning

* Rename

* Return result in pay_relayers_rewards

* Fix runtime compile issue

* Use MessageNonce

* Fix review issue

* Missing u64 replacement

* Revert return type changes

* Fix merge issue

* Remove useless clone
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 8, 2024
@boundless-forest @serban300
Move calculation relayer reward into `MessageDeliveryAndDispatchPayme…
27e5748 
…nt` (paritytech#1153)

* Refactor logic

* Thanks svyatonik help, it compile

* Fix failed unit test

* Remove compile warning

* Rename

* Return result in pay_relayers_rewards

* Fix runtime compile issue

* Use MessageNonce

* Fix review issue

* Missing u64 replacement

* Revert return type changes

* Fix merge issue

* Remove useless clone
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 8, 2024
@boundless-forest @serban300
Move calculation relayer reward into `MessageDeliveryAndDispatchPayme…
6e379ac 
…nt` (paritytech#1153)

* Refactor logic

* Thanks svyatonik help, it compile

* Fix failed unit test

* Remove compile warning

* Rename

* Return result in pay_relayers_rewards

* Fix runtime compile issue

* Use MessageNonce

* Fix review issue

* Missing u64 replacement

* Revert return type changes

* Fix merge issue

* Remove useless clone
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 8, 2024
@boundless-forest @serban300
Move calculation relayer reward into `MessageDeliveryAndDispatchPayme…
a07a34a 
…nt` (paritytech#1153)

* Refactor logic

* Thanks svyatonik help, it compile

* Fix failed unit test

* Remove compile warning

* Rename

* Return result in pay_relayers_rewards

* Fix runtime compile issue

* Use MessageNonce

* Fix review issue

* Missing u64 replacement

* Revert return type changes

* Fix merge issue

* Remove useless clone
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 9, 2024
@boundless-forest @serban300
Move calculation relayer reward into `MessageDeliveryAndDispatchPayme…
a26b5dc 
…nt` (paritytech#1153)

* Refactor logic

* Thanks svyatonik help, it compile

* Fix failed unit test

* Remove compile warning

* Rename

* Return result in pay_relayers_rewards

* Fix runtime compile issue

* Use MessageNonce

* Fix review issue

* Missing u64 replacement

* Revert return type changes

* Fix merge issue

* Remove useless clone
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 9, 2024
@boundless-forest @serban300
Move calculation relayer reward into `MessageDeliveryAndDispatchPayme…
a03d3bb 
…nt` (paritytech#1153)

* Refactor logic

* Thanks svyatonik help, it compile

* Fix failed unit test

* Remove compile warning

* Rename

* Return result in pay_relayers_rewards

* Fix runtime compile issue

* Use MessageNonce

* Fix review issue

* Missing u64 replacement

* Revert return type changes

* Fix merge issue

* Remove useless clone
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 9, 2024
@boundless-forest @serban300
Move calculation relayer reward into `MessageDeliveryAndDispatchPayme…
18bb9bb 
…nt` (paritytech#1153)

* Refactor logic

* Thanks svyatonik help, it compile

* Fix failed unit test

* Remove compile warning

* Rename

* Return result in pay_relayers_rewards

* Fix runtime compile issue

* Use MessageNonce

* Fix review issue

* Missing u64 replacement

* Revert return type changes

* Fix merge issue

* Remove useless clone
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 9, 2024
@boundless-forest @serban300
Move calculation relayer reward into `MessageDeliveryAndDispatchPayme…
3ad40ce 
…nt` (paritytech#1153)

* Refactor logic

* Thanks svyatonik help, it compile

* Fix failed unit test

* Remove compile warning

* Rename

* Return result in pay_relayers_rewards

* Fix runtime compile issue

* Use MessageNonce

* Fix review issue

* Missing u64 replacement

* Revert return type changes

* Fix merge issue

* Remove useless clone
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 9, 2024
@boundless-forest @serban300
Move calculation relayer reward into `MessageDeliveryAndDispatchPayme…
16722d9 
…nt` (paritytech#1153)

* Refactor logic

* Thanks svyatonik help, it compile

* Fix failed unit test

* Remove compile warning

* Rename

* Return result in pay_relayers_rewards

* Fix runtime compile issue

* Use MessageNonce

* Fix review issue

* Missing u64 replacement

* Revert return type changes

* Fix merge issue

* Remove useless clone
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 9, 2024
@boundless-forest @serban300
Move calculation relayer reward into `MessageDeliveryAndDispatchPayme…
9dcfac4 
…nt` (paritytech#1153)

* Refactor logic

* Thanks svyatonik help, it compile

* Fix failed unit test

* Remove compile warning

* Rename

* Return result in pay_relayers_rewards

* Fix runtime compile issue

* Use MessageNonce

* Fix review issue

* Missing u64 replacement

* Revert return type changes

* Fix merge issue

* Remove useless clone
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 10, 2024
@boundless-forest @serban300
Move calculation relayer reward into `MessageDeliveryAndDispatchPayme…
a86b1f7 
…nt` (paritytech#1153)

* Refactor logic

* Thanks svyatonik help, it compile

* Fix failed unit test

* Remove compile warning

* Rename

* Return result in pay_relayers_rewards

* Fix runtime compile issue

* Use MessageNonce

* Fix review issue

* Missing u64 replacement

* Revert return type changes

* Fix merge issue

* Remove useless clone
serban300 pushed a commit to serban300/polkadot-sdk that referenced this pull request Apr 10, 2024
@boundless-forest @serban300
Move calculation relayer reward into `MessageDeliveryAndDispatchPayme…
e2fddc6 
…nt` (paritytech#1153)

* Refactor logic

* Thanks svyatonik help, it compile

* Fix failed unit test

* Remove compile warning

* Rename

* Return result in pay_relayers_rewards

* Fix runtime compile issue

* Use MessageNonce

* Fix review issue

* Missing u64 replacement

* Revert return type changes

* Fix merge issue

* Remove useless clone
bkchr pushed a commit that referenced this pull request Apr 10, 2024
@boundless-forest @bkchr
Move calculation relayer reward into `MessageDeliveryAndDispatchPayme…
f209251 
…nt` (#1153)

* Refactor logic

* Thanks svyatonik help, it compile

* Fix failed unit test

* Remove compile warning

* Rename

* Return result in pay_relayers_rewards

* Fix runtime compile issue

* Use MessageNonce

* Fix review issue

* Missing u64 replacement

* Revert return type changes

* Fix merge issue

* Remove useless clone
@ukint-vs ukint-vs mentioned this pull request Apr 17, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@skunert skunert skunert approved these changes

@tadeohepperle tadeohepperle tadeohepperle approved these changes

@jsdw jsdw Awaiting requested review from jsdw

Assignees

@lexnv lexnv

Labels
A0-needs_burnin Pull request needs to be tested on a live validator node before merge. DevOps is notified via matrix R0-silent Changes should not be mentioned in any release notes
Projects
None yet
Milestone
No milestone
3 participants
@lexnv @skunert @tadeohepperle