Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core/state: introduce stateupdate structure #29530

Merged
merged 3 commits into from
Jun 3, 2024

Conversation

rjl493456442
Copy link
Member

@rjl493456442 rjl493456442 commented Apr 15, 2024

This pull request rewrites the statedb by introducing a state update struct.

A state update struct will be constructed once the statedb.Commit is invoked. It contains all the state
changes caused by previous state transition, e.g.

  • original state root and post-execute state root
  • deleted accounts along with their original value
  • updated accounts along with their original value
  • dirty contract codes
  • dirty trie nodes (state hasher)

Instead of committing the state change into state snapshot and trie database blindly, the state update
will be returned and then can be consumed by different state database implementation in different
ways. It gives more flexibilities to design database abstraction for different purposes.

What's more, the statedb has been simplified significantly, especially in state change construction.

@rjl493456442 rjl493456442 force-pushed the state-update branch 3 times, most recently from 8e190d8 to ea9d92e Compare April 18, 2024 07:05
@rjl493456442 rjl493456442 force-pushed the state-update branch 4 times, most recently from 005bb86 to 60746bb Compare April 29, 2024 07:50
@rjl493456442 rjl493456442 force-pushed the state-update branch 6 times, most recently from 354717c to bd6b20a Compare May 13, 2024 01:54
@rjl493456442 rjl493456442 force-pushed the state-update branch 2 times, most recently from 2fe48c1 to 2089fdb Compare May 14, 2024 03:04
core/state/state_object.go Outdated Show resolved Hide resolved
@rjl493456442
Copy link
Member Author

This pull request has been deployed full sync for a few days, since the genesis. It can sync more than 16m blocks with no error.

// boundary; however post the byzantium fork, the commit will only be performed
// at the end of block, this set essentially tracks all the modifications
// made within the block.
needCommit Storage
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that the previous 3 items are all XXXStorage, I'd recommend using uncommittedStorage for this one.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though, it's a bit weird, because it contains the "origin" values too, which also somewhat clash with originStorage. Does it make sense to have the origin in 2 different variables? Can't we just use this field as a map[hash]struct{} to track which slots need commit, but otherwise rely on originStorage for the actual value content?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately the tracked values in origin, pending and uncommitted have different meanings:

  • origin: the value before the entire state transition of block
  • pending: the latest value after mutation (e.g. after applying each transaction)
  • uncommittedStorage: the value before applying the "uncommitted" mutations.

Before the byzantium, the value being tracked in uncommittedStorage is the original value before the transaction; Post the byzantium, the value being tracked in uncommittedStorage is the original value before the block.

The reason for maintaining this value is, we want to make sure only dirty slots will be committed.

@@ -193,6 +191,7 @@ func (s *stateObject) GetCommittedState(key common.Hash) common.Hash {
// have been handles via pendingStorage above.
// 2) we don't have new values, and can deliver empty response back
if _, destructed := s.db.stateObjectsDestruct[s.address]; destructed {
s.originStorage[key] = common.Hash{} // track the empty slot as origin value
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any particular reason for starting to track this here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, i think it would be better to track the "empty" value as origin, reasons:

Here, https://github.com/rjl493456442/go-ethereum/blob/state-update/core/state/state_object.go#L428 we blindly assume the original value is tracked in the s.originStorage map. It would still be correct if we don't track empty value, but it feels pretty weird.

Besides, it aligns with the behavior in https://github.com/rjl493456442/go-ethereum/blob/state-update/core/state/state_object.go#L233, as the slot read from database could also be empty.

What's more, tracking the empty value here could potentially be useful for differentiating "existence" and "emptiness" in the future? e.g. we could error out if a slot is never accessed (not existent in the originStorage map) but later be modified?

}
s.pendingStorage = make(Storage) // reset pending map
s.needCommit = make(Storage) // empties the commit markers
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want to use some map clearing method here to retain the memory and not reallocate for the next tx (pre byzantium).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maps.Delete is still in golang/exp package. Probably wait a bit until it's fully shipped in standard maps package?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

data: types.SlimAccountRLP(s.data),
}
if s.origin == nil {
op.origin = nil // the account was not present
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a noop, right? Can't we just have a single if branch to set origin if s.origin != nil?

return blob
}
)
for key, slot := range s.pendingStorage {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pls rename slot to value. Slot is really the key itself.

// Overwrite the clean value of storage slots
s.originStorage[key] = slot
}
s.pendingStorage = make(Storage)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use a map clearing method here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maps.Delete is still in golang/exp package. Probably wait a bit until it's fully shipped in standard maps package?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't this work?

https://pkg.go.dev/builtin#clear

Copy link
Member

@karalabe karalabe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the PR is a good cleanup, I've tried to check the tiny details, but it's just insane thinking through all the implications. I'm going to rely on fuzzers, tests and syncs at this point to hope everything's fine.

@karalabe karalabe added this to the 1.14.4 milestone Jun 3, 2024
@karalabe karalabe merged commit d38b88a into ethereum:master Jun 3, 2024
3 checks passed
jorgemmsilva pushed a commit to iotaledger/go-ethereum that referenced this pull request Jun 17, 2024
* core/state: introduce stateUpate structure

* core/state: remove outdated function description

* core/state: address comments
stwiname pushed a commit to subquery/data-node-go-ethereum that referenced this pull request Sep 9, 2024
* core/state: introduce stateUpate structure

* core/state: remove outdated function description

* core/state: address comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants