Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inner hash of value with new nodes #142

Merged
merged 209 commits into from
Oct 19, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
209 commits
Select commit Hold shift + click to select a range
46984e1
refacto from complex_proof
cheme May 18, 2020
17eea1a
iterbuild prefix fix
cheme May 18, 2020
dfd1c28
compact proof parameterized
cheme May 18, 2020
8bcd01e
restore clippy change
cheme May 18, 2020
5755325
bench update from #81
cheme May 18, 2020
c2e8168
Merge branch 'master' into test-layouts
cheme Mar 2, 2021
32f6b29
comment
cheme Mar 10, 2021
1e6a968
fix bench check
cheme Mar 10, 2021
e1dedee
hash db with inner hashed representation.
cheme Mar 10, 2021
fcc57c7
Looks like a bad approach: storing hash under certain condition and
cheme Mar 10, 2021
c13d87d
meta for hashdb and memory db, conflict with layout def with no
cheme Mar 15, 2021
ceedfe6
issues with default impl
cheme Mar 15, 2021
704cf89
self ref trait issue
cheme Mar 15, 2021
34a414d
fix tests
cheme Mar 15, 2021
760456d
No params for NoMeta
cheme Mar 15, 2021
b08ad17
getting meta for triedbmut from encoded node and storing it
cheme Mar 16, 2021
8a9cacf
plugged for trie-db
cheme Mar 16, 2021
bb8748d
test (all borked)
cheme Mar 16, 2021
e6c88ab
memory db null node always use default meta.
cheme Mar 16, 2021
f30da8f
Fix or remove test in regards to current implementation state.
cheme Mar 16, 2021
75a33f8
specific inner hashing on test
cheme Mar 16, 2021
cfae842
Remove unimplemented tests
cheme Mar 16, 2021
0d45913
iter_build implementing
cheme Mar 16, 2021
75b55c5
restore iter_build related failing tests.
cheme Mar 16, 2021
65c53c1
missing one
cheme Mar 16, 2021
3621bcc
fix algo
cheme Mar 16, 2021
41e7ae1
change type of treshold from layout
cheme Mar 16, 2021
0ed4c13
instantiable layout for context specific behavior.
cheme Mar 16, 2021
7419d6a
draft version: TODO add meta persistence in triedbmut.
cheme Mar 16, 2021
4071592
Adding meta in node, need to allow change (at this point it is unique
cheme Mar 17, 2021
fd86446
preparing migration test
cheme Mar 17, 2021
5fa70c2
Range in memory db is useless for current use case.
cheme Mar 17, 2021
b82e93d
init tries
cheme Mar 17, 2021
99b2cab
test scenario
cheme Mar 23, 2021
c3ad34c
fix layout logic: encoding is terrible, but for testing.
cheme Mar 23, 2021
396d838
Remove MetaInput (not really usefull).
cheme Mar 23, 2021
e3de799
meta non optional
cheme Mar 23, 2021
7be3c3e
inner layout for meta.
cheme Mar 23, 2021
d43f2bf
Revert "inner layout for meta."
cheme Mar 23, 2021
85f5376
Put Layout as primary type in triedbmut
cheme Mar 23, 2021
d8bba83
bring back MetaInput but only at trie level.
cheme Mar 24, 2021
67cf79f
better api
cheme Mar 24, 2021
d6cfbdc
Remove set_inner_hash_value.
cheme Mar 24, 2021
2356b7f
specific inline build
cheme Mar 24, 2021
76f34c0
inline usage
cheme Mar 24, 2021
d7416a9
removing parent reference (callback on child append would be better),
cheme Mar 24, 2021
0aed852
Correct meta for new leafs.
cheme Mar 24, 2021
bae46a6
set_value_callback
cheme Mar 24, 2021
c5eb982
Major change, use cleaner enum in triedbmut.
cheme Mar 24, 2021
25f448a
branch do not change on modified value.
cheme Mar 24, 2021
f6031ef
plug branch for insert.
cheme Mar 25, 2021
6ddc827
plug branch for remove
cheme Mar 25, 2021
aafa1c4
allow lazy init of branch size
cheme Mar 25, 2021
ae774b9
store old old children list
cheme Mar 25, 2021
c1e480e
need to remove range: it get confused with version and is not strictly
cheme Mar 25, 2021
427db48
Scenario test passing (needs more test and fix api, but proves usable).
cheme Mar 25, 2021
175eefc
test init for proof similar to substrate.
cheme May 4, 2021
fe21838
use no extension (easier to debug)
cheme May 4, 2021
1d13ea4
code to register branch access.
cheme May 5, 2021
822a5ea
revert accessed value flag: proof will need to register them.
cheme May 5, 2021
888135d
pass meta in trie_codec
cheme May 5, 2021
df84db7
fix extract
cheme May 5, 2021
d2f502e
small fixes.
cheme May 6, 2021
09310e2
Replace by hash, next have codec with variant for hash as value.
cheme May 6, 2021
2c1372b
Get info of hashed value from meta
cheme May 6, 2021
8fca892
started, but require variant for reencode with right size.
cheme May 6, 2021
fab48c9
incomplete values for trie-db crate
cheme May 6, 2021
d25e80f
Correct size written and run ok.
cheme May 6, 2021
ab9213a
checkpoint: test completed.
cheme May 7, 2021
8b86e13
TODO just this trait change
cheme May 7, 2021
63b6c96
Use a callback approach instead, because we do not know if value
cheme May 7, 2021
5557752
actual test without compact
cheme May 7, 2021
8488eb6
call back for test
cheme May 7, 2021
1bb585f
other access callback, currently ignore inline val
cheme May 7, 2021
f964266
Iterator for key only (do not indicate value access to hash-db).
cheme May 7, 2021
3631ecd
Rename Value function, and remove it from hashdb trait.
cheme May 10, 2021
f418ed7
using meta in codec mostly require latest change with metas in nodes.
cheme May 10, 2021
cb5d6f3
Merge branch 'hashed-value' into hashed-value-proof
cheme May 10, 2021
b711964
Merge branch 'hashed-value-proof' into hashed-value-data-access
cheme May 10, 2021
18a6c5e
Merge branch 'tmp' into hashed-value-proof
cheme May 10, 2021
9fcb5f8
remove encoded callback
cheme May 10, 2021
798401b
remove decoded children (we use `doceded_callback` instead).
cheme May 10, 2021
e0284e0
finish iter_build refact
cheme May 10, 2021
6f206af
remove a few TODOs
cheme May 10, 2021
637eddb
remove value range function since we got range from codec during
cheme May 10, 2021
5767c26
remove some todos (call back are only here to force update, on encoded
cheme May 10, 2021
10f3924
Merge branch 'hashed-value-proof' into hashed-value-data-access
cheme May 11, 2021
18ecd54
Read some meta from encoded state.
cheme May 14, 2021
5dfea95
read state meta in extract value.
cheme May 14, 2021
75e2c3e
pass meta around nodes.
cheme May 14, 2021
b38b207
quick fix
cheme May 14, 2021
cbe279c
encode flag
cheme May 14, 2021
41f5e1e
pass associated type as parameter when flagging.
cheme May 14, 2021
47f0be3
flag db mut.
cheme May 17, 2021
594c900
passing parent meta for new values meta.
cheme May 17, 2021
6c2baf3
do_value_hash with size condition
cheme May 18, 2021
333dd0b
Revert "do_value_hash with size condition"
cheme May 18, 2021
b7d3f01
do test in encode instead
cheme May 18, 2021
3686fac
actually hashing in codec was redundant with MetaHasher storage.
cheme May 18, 2021
b3a6725
Access value plan.
cheme May 19, 2021
0474660
decode on correct hash.
cheme May 19, 2021
ab6d8bb
alternate fix
cheme May 19, 2021
7fe9c0c
read from root meta
cheme May 20, 2021
438de85
write back meta in root (allows set meta with layout init).
cheme May 20, 2021
6c98de5
switch off parent for global, lot of dead code to remove.
cheme May 20, 2021
65a5dda
remove parent meta passing
cheme May 20, 2021
b371e45
remove flag function and fix ability to set state meta on empty node.
cheme May 20, 2021
3ee0212
Rename MetaInput to GlobalMeta (when stable could also Meta -> NodeMeta
cheme May 21, 2021
88f2c9c
read layout from root in trie_codec.
cheme May 21, 2021
618c726
recorder with meta to enable trie_codec tests.
cheme May 21, 2021
fc1868f
Simplify iter_build
cheme May 21, 2021
29f9b9e
trie root implementation
cheme May 21, 2021
247b1f9
iter build set root meta
cheme May 21, 2021
d101d0f
clean some unused params
cheme May 21, 2021
31acde1
Revert change to triedbmut node changes management.
cheme May 21, 2021
0b365f7
global layout for empty on clearing trie
cheme May 21, 2021
e9aec59
global meta eaccessor
cheme May 21, 2021
e516875
set global meta
cheme May 21, 2021
0957101
missing branch value stream
cheme May 24, 2021
c356f76
regression on extension test
cheme May 25, 2021
d8d46c8
debugging
cheme May 25, 2021
f5b5050
fix prefix for extension implementation
cheme May 25, 2021
2b1fc06
support displaying trie meta in debug
cheme May 26, 2021
4d7dabf
out of date TODO
cheme May 26, 2021
24f1143
reexport triedbkey iterator
cheme May 27, 2021
da84037
Merge branch 'master' into hashed-value-simple4
cheme May 28, 2021
3d1299c
Merge branch 'master' into hashed-value-simple4
cheme May 29, 2021
0f04db6
Add start of node encoding for value (including size).
cheme Jun 8, 2021
22910ad
NoMeta clonable and default.
cheme Jun 9, 2021
74ab198
unbound generate verify proof to () meta.
cheme Jun 11, 2021
3ba095c
fix bench
cheme Jun 11, 2021
45eb1c1
start removals
cheme Jun 14, 2021
3935eb2
update api
cheme Jun 14, 2021
60340d3
set_state_meta api
cheme Jun 14, 2021
b4b0d19
proof recorder needs global meta.
cheme Jun 14, 2021
4cd9ec2
rename layout_meta to global_meta
cheme Jun 15, 2021
9ded335
Remove some tests layouts.
cheme Jun 15, 2021
54c61d7
before switching test case.
cheme Jun 15, 2021
f574a14
Get trie implementation from substrate.
cheme Jun 18, 2021
3e4b7b0
Remove need for storing meta, and state meta (all from codec).
cheme Jun 18, 2021
bcf4141
unused value out of meta
cheme Jun 18, 2021
58a57f6
clean hashdb and memory db from meta
cheme Jun 18, 2021
8248d39
remove most of meta hasher
cheme Jun 18, 2021
51e96b6
remove metahasher from trie root
cheme Jun 18, 2021
c7c3218
remove all metahasher
cheme Jun 18, 2021
1a2ebb1
move TrieMeta definition.
cheme Jun 18, 2021
e90c6dc
Remove Meta trait.
cheme Jun 18, 2021
c43ba9b
Remove useless meta function.
cheme Jun 18, 2021
978b330
offset from codec.
cheme Jun 18, 2021
4231ac6
Remove old match codec code.
cheme Jun 18, 2021
c7e0aeb
remove meta from stream
cheme Jun 18, 2021
e90f62f
import vec for no_std
cheme Jun 21, 2021
41b9756
fix
cheme Jun 21, 2021
511e98b
Properly test no std for trie db
cheme Jun 21, 2021
9bb23e3
update doc, split referencet trie, update fuzzer.
cheme Jun 21, 2021
b5d6ca6
remove inline info from generate.rs
cheme Jun 21, 2021
e351dbd
Remove a few more outdated comment and methods.
cheme Jun 21, 2021
d6439b5
ommitted value are never encoded to hashed value (removing from verify).
cheme Jun 21, 2021
9233e01
Remove original size from hashed value.
cheme Jun 21, 2021
cec2aa2
additional plan position is still needed.
cheme Jun 21, 2021
618beec
Merge branch 'master' into hashed-value-simple4
cheme Jul 22, 2021
70f908d
Merge branch 'master' into hashed-value-simple4
cheme Aug 17, 2021
9f5519d
fix doc
cheme Aug 17, 2021
c20725e
Merge branch 'hashed-value-simple4' of github.com:cheme/trie into has…
cheme Aug 17, 2021
20cac45
minor doc and naming changes.
cheme Aug 17, 2021
b2e3290
force node update when meta did change, consider false positive a non
cheme Sep 1, 2021
b995d5b
missing check
cheme Sep 1, 2021
63e1651
Unsafe suspend (only for testing, could revert).
cheme Sep 2, 2021
95d8d3c
start change
cheme Sep 2, 2021
4a86761
progress
cheme Sep 2, 2021
ebba02f
likely ok implementation, need to rework compact proofs and possibly
cheme Sep 2, 2021
af538cc
iter_build support
cheme Sep 2, 2021
b40a0f6
actually process some hashed nodes.
cheme Sep 2, 2021
074f1ce
Exception in test for small thresholds.
cheme Sep 2, 2021
abb9548
Compact proof new format.
cheme Sep 3, 2021
36a5772
incorrect removal
cheme Sep 3, 2021
2f075fa
delete of attached value (TODO test the prefix value?).
cheme Sep 3, 2021
9c7082d
no need to add the value nodes with proof mods.
cheme Sep 3, 2021
55bfb6e
correct implementation for proof compact with value (never attach a
cheme Sep 3, 2021
9f39809
Merge branch 'master' into hashed-value-simple4-new
cheme Sep 3, 2021
d4e0edb
fix bench
cheme Sep 3, 2021
0e0dc68
parameterized Value of triedbmut
cheme Sep 4, 2021
8cfbde8
simplify unchanged
cheme Sep 4, 2021
8434b7f
remove remaining todos.
cheme Sep 6, 2021
91d8174
remove unchanged function and fix impl
cheme Sep 6, 2021
0ba3a9c
rename enum ValueMgmt
cheme Sep 6, 2021
5518c3e
rename Alt reference
cheme Sep 6, 2021
cdbd7fc
max_inline_value
cheme Sep 6, 2021
432c4d6
renamings
cheme Sep 6, 2021
a8467fe
review fixes.
cheme Sep 6, 2021
97caad5
Use new HashedValue variant for triedb_mut nodes.
cheme Sep 6, 2021
159e964
Value prefix must be full key because it may change with trie structure
cheme Sep 21, 2021
910c792
revert to static based layout
cheme Oct 4, 2021
3d57a06
Switching escape header to single character.
cheme Oct 4, 2021
24c9ae9
Switch to using Option<Value>
cheme Oct 4, 2021
33f2b9c
remove unused children node
cheme Oct 4, 2021
b7238ef
Remove hybrid compatibility in compact proof with value (no in standard
cheme Oct 5, 2021
99e83d3
fat db key iter
cheme Oct 5, 2021
61ae49b
test
cheme Oct 5, 2021
40b8a45
Remove NoValue variant from trie_root.
cheme Oct 6, 2021
3e668b0
enum for the encode callback
cheme Oct 6, 2021
7837ace
incorrect asumption on triedbmut lookup fixed.
cheme Oct 7, 2021
3caaba0
review suggestions
cheme Oct 7, 2021
28db1ad
Revert to boolean for omit_value in proof generate.
cheme Oct 15, 2021
729700e
review suggestion, rename Value to inline and HashedValue to ValueNode
cheme Oct 18, 2021
632af89
also rename ValueNode to Node.
cheme Oct 18, 2021
ec4e955
update crates version, prepare publish.
cheme Oct 18, 2021
89b344d
trie-bench versioning
cheme Oct 18, 2021
a63b21d
Changelog dates
cheme Oct 19, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion hash-db/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Copyright 2017, 2018 Parity Technologies
// Copyright 2017, 2021 Parity Technologies
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
Expand Down
3 changes: 3 additions & 0 deletions memory-db/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ The format is based on [Keep a Changelog].

## [Unreleased]

## [0.28.0] - 2021-10-19
- Change in api bound. [#142](https://github.com/paritytech/trie/pull/142)

## [0.27.0] - 2021-07-02
- Update `parity-util-mem` to 0.10. [#137](https://github.com/paritytech/trie/pull/137)

Expand Down
2 changes: 1 addition & 1 deletion memory-db/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "memory-db"
version = "0.27.0"
version = "0.28.0"
authors = ["Parity Technologies <admin@parity.io>"]
description = "In-memory implementation of hash-db, useful for tests"
repository = "https://github.com/paritytech/trie"
Expand Down
2 changes: 1 addition & 1 deletion memory-db/benches/bench.rs
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ fn instantiation(b: &mut Criterion) {
fn compare_to_null_embedded_in_struct(b: &mut Criterion) {
struct X {
a_hash: <KeccakHasher as Hasher>::Out,
};
}
let x = X {
a_hash: KeccakHasher::hash(&[0u8][..]),
};
Expand Down
20 changes: 10 additions & 10 deletions memory-db/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -282,10 +282,10 @@ pub fn legacy_prefixed_key<H: KeyHasher>(key: &H::Out, prefix: Prefix) -> Vec<u8
prefixed_key
}

impl<'a, H, KF, T, M> Default for MemoryDB<H, KF, T, M>
impl<H, KF, T, M> Default for MemoryDB<H, KF, T, M>
where
H: KeyHasher,
T: From<&'a [u8]>,
T: for<'a> From<&'a [u8]>,
KF: KeyFunction<H>,
M: MemTracker<T> + Default,
{
Expand Down Expand Up @@ -337,15 +337,15 @@ where
}
}

impl<'a, H, KF, T, M> MemoryDB<H, KF, T, M>
impl<H, KF, T, M> MemoryDB<H, KF, T, M>
where
H: KeyHasher,
T: From<&'a [u8]>,
T: for<'a> From<&'a [u8]>,
KF: KeyFunction<H>,
M: MemTracker<T> + Default,
{
/// Create a new `MemoryDB` from a given null key/data
pub fn from_null_node(null_key: &'a [u8], null_node_data: T) -> Self {
pub fn from_null_node(null_key: &[u8], null_node_data: T) -> Self {
MemoryDB {
data: HashMap::default(),
hashed_null_node: H::hash(null_key),
Expand All @@ -356,7 +356,7 @@ where
}

/// Create a new instance of `Self`.
pub fn new(data: &'a [u8]) -> Self {
pub fn new(data: &[u8]) -> Self {
Self::from_null_node(data, data.into())
}

Expand Down Expand Up @@ -544,8 +544,8 @@ where
impl<H, KF, T, M> HashDB<H, T> for MemoryDB<H, KF, T, M>
where
H: KeyHasher,
T: Default + PartialEq<T> + for<'a> From<&'a [u8]> + Clone + Send + Sync,
KF: Send + Sync + KeyFunction<H>,
T: Default + PartialEq<T> + AsRef<[u8]> + for<'a> From<&'a [u8]> + Clone + Send + Sync,
KF: KeyFunction<H> + Send + Sync,
M: MemTracker<T> + Send + Sync,
{
fn get(&self, key: &H::Out, prefix: Prefix) -> Option<T> {
Expand Down Expand Up @@ -628,7 +628,7 @@ where
impl<H, KF, T, M> HashDBRef<H, T> for MemoryDB<H, KF, T, M>
where
H: KeyHasher,
T: Default + PartialEq<T> + for<'a> From<&'a [u8]> + Clone + Send + Sync,
T: Default + PartialEq<T> + AsRef<[u8]> + for<'a> From<&'a [u8]> + Clone + Send + Sync,
KF: KeyFunction<H> + Send + Sync,
M: MemTracker<T> + Send + Sync,
{
Expand All @@ -651,7 +651,7 @@ where
impl<H, KF, T, M> AsHashDB<H, T> for MemoryDB<H, KF, T, M>
where
H: KeyHasher,
T: Default + PartialEq<T> + for<'a> From<&'a[u8]> + Clone + Send + Sync,
T: Default + PartialEq<T> + AsRef<[u8]> + for<'a> From<&'a[u8]> + Clone + Send + Sync,
KF: KeyFunction<H> + Send + Sync,
M: MemTracker<T> + Send + Sync,
{
Expand Down
3 changes: 3 additions & 0 deletions test-support/reference-trie/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ The format is based on [Keep a Changelog].

## [Unreleased]

## [0.24.0] - 2021-10-19
- Additional test codec substrate like. [#142](https://github.com/paritytech/trie/pull/142)

## [0.23.0] - 2021-01-27
- Update parity-scale-codec to 2.0.

Expand Down
6 changes: 3 additions & 3 deletions test-support/reference-trie/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "reference-trie"
version = "0.23.0"
version = "0.24.0"
authors = ["Parity Technologies <admin@parity.io>"]
description = "Simple reference trie format"
repository = "https://github.com/paritytech/trie/"
Expand All @@ -11,8 +11,8 @@ edition = "2018"
hash-db = { path = "../../hash-db" , version = "0.15.2"}
hash256-std-hasher = { path = "../../hash256-std-hasher", version = "0.15.2" }
keccak-hasher = { path = "../keccak-hasher", version = "0.15.3" }
trie-db = { path = "../../trie-db", default-features = false, version = "0.22.0" }
trie-root = { path = "../../trie-root", default-features = false, version = "0.16.0" }
trie-db = { path = "../../trie-db", default-features = false, version = "0.23.0" }
trie-root = { path = "../../trie-root", default-features = false, version = "0.17.0" }
parity-scale-codec = { version = "2", features = ["derive"] }

[dev-dependencies]
Expand Down
Loading