-
Notifications
You must be signed in to change notification settings - Fork 700
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create a Basic Proving Trie for the Runtime #3881
Changes from 9 commits
d50d5e4
7f4422e
bdc0c84
da385ab
6ab4bfb
1cfb29f
3080c7b
39b0fca
599f576
a1c8886
6bde2a3
ead7951
0cac048
acfe734
45f4287
efbe1c9
cfa62ce
885fd94
18a5c2e
5e3e518
f35cacc
e6c3759
89679c3
6e853ba
00a7aa9
ffbd656
0217d0f
f20038a
f934d8d
c92bd6a
266a89b
3034aab
1ceab64
c426a65
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,178 @@ | ||
// This file is part of Substrate. | ||
|
||
// Copyright (C) Parity Technologies (UK) Ltd. | ||
// SPDX-License-Identifier: Apache-2.0 | ||
|
||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
//! Types for a simple merkle trie used for checking and generating proofs. | ||
|
||
use crate::{Decode, DispatchError, Encode}; | ||
|
||
use sp_std::vec::Vec; | ||
use sp_trie::{ | ||
trie_types::{TrieDBBuilder, TrieDBMutBuilderV0}, | ||
LayoutV0, MemoryDB, Recorder, Trie, TrieMut, EMPTY_PREFIX, | ||
}; | ||
|
||
/// A trait for creating a merkle trie for checking and generating merkle proofs. | ||
pub trait ProvingTrie<Hashing, Hash, Key, Value> | ||
shawntabrizi marked this conversation as resolved.
Show resolved
Hide resolved
shawntabrizi marked this conversation as resolved.
Show resolved
Hide resolved
shawntabrizi marked this conversation as resolved.
Show resolved
Hide resolved
|
||
where | ||
Self: Sized, | ||
{ | ||
/// Create a new instance of a `ProvingTrie` using an iterator of key/value pairs. | ||
fn generate_for<I>(items: I) -> Result<Self, DispatchError> | ||
where | ||
I: IntoIterator<Item = (Key, Value)>; | ||
/// Access the underlying trie root. | ||
fn root(&self) -> &Hash; | ||
/// Check a proof contained within the current memory-db. Returns `None` if the | ||
/// nodes within the current `MemoryDB` are insufficient to query the item. | ||
fn query(&self, key: Key) -> Option<Value>; | ||
/// Create the full verification data needed to prove a key and its value in the trie. Returns | ||
/// `None` if the nodes nodes within the current `MemoryDB` are insufficient to create a proof. | ||
shawntabrizi marked this conversation as resolved.
Show resolved
Hide resolved
|
||
fn create_proof(&self, key: Key) -> Option<Vec<Vec<u8>>>; | ||
/// Create a new instance of `ProvingTrie` from raw nodes. Nodes can be generated using the | ||
/// `create_proof` function. | ||
fn from_nodes(root: Hash, nodes: &[Vec<u8>]) -> Self; | ||
} | ||
|
||
/// A basic trie implementation for checking and generating proofs for a key / value pair. | ||
pub struct BasicProvingTrie<Hashing, Hash, Key, Value> | ||
where | ||
Hashing: sp_core::Hasher<Out = Hash>, | ||
{ | ||
db: MemoryDB<Hashing>, | ||
root: Hash, | ||
_phantom: core::marker::PhantomData<(Key, Value)>, | ||
} | ||
|
||
impl<Hashing, Hash, Key, Value> ProvingTrie<Hashing, Hash, Key, Value> | ||
for BasicProvingTrie<Hashing, Hash, Key, Value> | ||
where | ||
Hashing: sp_core::Hasher<Out = Hash>, | ||
Hash: Default + Send + Sync, | ||
Key: Encode, | ||
Value: Encode + Decode, | ||
{ | ||
fn generate_for<I>(items: I) -> Result<Self, DispatchError> | ||
where | ||
I: IntoIterator<Item = (Key, Value)>, | ||
{ | ||
let mut db = MemoryDB::default(); | ||
let mut root = Default::default(); | ||
|
||
{ | ||
let mut trie = TrieDBMutBuilderV0::new(&mut db, &mut root).build(); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cheme why do we use a specific version here? Should this not be V1? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In general the version does not matter much for proof as V0 can be use on a V1 trie and V1 can be use on a V0, and since it is a proof we mainly care for access node (both version should have same access patterns). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. So you want me to make this a parameter we pass in? or make it V1, or leave it untouched? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. either V1 only (would not be compatible in some case) or pass it as a parameter. |
||
for (key, value) in items.into_iter() { | ||
key.using_encoded(|k| value.using_encoded(|v| trie.insert(k, v))) | ||
.map_err(|_| "failed to insert into trie")?; | ||
} | ||
} | ||
|
||
Ok(Self { db, root, _phantom: Default::default() }) | ||
} | ||
|
||
fn root(&self) -> &Hash { | ||
&self.root | ||
} | ||
|
||
fn query(&self, key: Key) -> Option<Value> { | ||
let trie = TrieDBBuilder::new(&self.db, &self.root).build(); | ||
key.using_encoded(|s| trie.get(s)) | ||
.ok()? | ||
.and_then(|raw| Value::decode(&mut &*raw).ok()) | ||
} | ||
|
||
fn create_proof(&self, key: Key) -> Option<Vec<Vec<u8>>> { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe better passing Vec and allow call multiple query here. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Updated name and also added the multi-value API |
||
let mut recorder = Recorder::<LayoutV0<Hashing>>::new(); | ||
shawntabrizi marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
{ | ||
let trie = | ||
TrieDBBuilder::new(&self.db, &self.root).with_recorder(&mut recorder).build(); | ||
|
||
key.using_encoded(|k| { | ||
trie.get(k).ok()?.and_then(|raw| Value::decode(&mut &*raw).ok()) | ||
})?; | ||
} | ||
|
||
Some(recorder.drain().into_iter().map(|r| r.data).collect()) | ||
} | ||
|
||
fn from_nodes(root: Hash, nodes: &[Vec<u8>]) -> Self { | ||
cheme marked this conversation as resolved.
Show resolved
Hide resolved
|
||
use sp_trie::HashDBT; | ||
|
||
let mut memory_db = MemoryDB::default(); | ||
for node in nodes { | ||
HashDBT::insert(&mut memory_db, EMPTY_PREFIX, &node[..]); | ||
} | ||
|
||
Self { db: memory_db, root, _phantom: Default::default() } | ||
} | ||
} | ||
|
||
#[cfg(test)] | ||
mod tests { | ||
use super::*; | ||
use crate::traits::BlakeTwo256; | ||
use sp_core::H256; | ||
use sp_std::{collections::btree_map::BTreeMap, str::FromStr}; | ||
|
||
// A trie which simulates a trie of accounts (u32) and balances (u128). | ||
type BalanceTrie = BasicProvingTrie<BlakeTwo256, H256, u32, u128>; | ||
|
||
// The expected root hash for an empty trie. | ||
fn empty_root() -> H256 { | ||
H256::from_str("0x03170a2e7597b7b7e3d84c05391d139a62b157e78786d8c082f29dcf4c111314") | ||
.unwrap() | ||
} | ||
|
||
#[test] | ||
fn empty_trie_works() { | ||
let empty_trie = BalanceTrie::generate_for(Vec::new()).unwrap(); | ||
assert_eq!(*empty_trie.root(), empty_root()); | ||
} | ||
|
||
#[test] | ||
fn basic_end_to_end() { | ||
// Create a map of users and their balances. | ||
let mut map = BTreeMap::<u32, u128>::new(); | ||
for i in 0..10u32 { | ||
map.insert(i, i.into()); | ||
} | ||
|
||
// Put items into the trie. | ||
let balance_trie = BalanceTrie::generate_for(map).unwrap(); | ||
|
||
// Root is changed. | ||
let root = *balance_trie.root(); | ||
assert!(root != empty_root()); | ||
|
||
// Assert valid key is queryable. | ||
assert_eq!(balance_trie.query(6u32), Some(6u128)); | ||
assert_eq!(balance_trie.query(9u32), Some(9u128)); | ||
// Invalid key returns none. | ||
assert_eq!(balance_trie.query(69u32), None); | ||
|
||
// Create a proof for a valid key. | ||
let proof = balance_trie.create_proof(6u32).unwrap(); | ||
// Can't create proof for invalid key. | ||
assert_eq!(balance_trie.create_proof(69u32), None); | ||
|
||
// Create a new proving trie from the proof. | ||
let new_balance_trie = BalanceTrie::from_nodes(root, &proof); | ||
|
||
// Assert valid key is queryable. | ||
assert_eq!(new_balance_trie.query(6u32), Some(6u128)); | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not too sure if it should be part of sp-runtime crate.
Thing is the choice of using this trie should not be a default for any runtime (I think it is not the best tree for most use case, and I would encourage more binary trie, anyway it is very use case specific).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we have a binary trie available in Substrate?
Does this trie work along side child tries?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the proof is to be run on child trie, there is no choice but to use this trie indeed.
(if using child trie root to prove, please be careful the chain will not be switching from trie V0 to trie V1 (should not matter for crowdloan as iirc the values are <= 32 byte), as a migration may change child trie root.
About this trie version, the simpliest would probably be to just pass the version in parameter.
When I read the PR I did not have this use case in mind but it is true that if what we are proving is from substrate state, then it could make sense to have this in runtime.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can I name this
ProvingTrieBase16
, and that would be better?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not too sure anymore what was my point 🤦
Probably at first reading I was thinking it should be an external crate, then realizing it is being tightly linked with substrate state could actually be fine.
ProvingTrieBase16
may not be a good idea as long as substrate does have a single trie backend.Maybe it could simple be a module in sp-trie, but guess I am ok with things being this way.