feat: build a real pending block

[MicaiahReid]

This change expands eth_getBlockByNumber so that Ganache actually returns a pending block when you call it and pass the "pending" tag. shhhhh, don't tell anyone but in the past we'd just return the latest block.

So, if you retrieve the pending block, Ganache will now return a block that is after the latest block and which contains the transactions that are still waiting in the transaction pool to be mined. It's almost like we can see into the

Sadly our clairvoyance does have its limitations, so don't expect transactions that haven't made it into the transaction pool yet to be in the pending block.

Fixes #772, #3481, #3529

[MicaiahReid]

For the reviewers - I have a few notes on design decisions.

When I first started this PR about a year ago, David and I went back and forth on how to handle some race conditions.

• What do we do if there are constantly transactions coming into the pool that alter the results of the pending block? Do we fix the pending block, or return one that we know to be wrong?
• What do we do if a block is being mined as the user requests the pending block? Do we wait to mine the new block, then make a pending block off of that one? Or do we just make a pending block off of the current latest block?

Like most things in life, my answer is to take the easy way. When a pending block is requested, we clone the state of the vm and executable transaction pool as is, right now, and make a pending block off of it.

What do we do if there are constantly transactions coming into the pool that alter the results of the pending block? Do we fix the pending block, or return one that we know to be wrong?

We return a pending block that is already outdated and incorrect. There will always be the possibility of the pending block being incorrect, so I don't think it's worth spending too much energy to make sure it's accurate in this race condition. In the real world, requesting a pending block is just one node's opinion on the next block, based off of the transactions in their pool. So it's even less likely that a pending block is correct in the real world.

What do we do if a block is being mined as the user requests the pending block? Do we wait to mine the new block, then make a pending block off of that one? Or do we just make a pending block off of the current latest block?

In this case we still make a pending block based off of the latest block at the time the request is made. If mining has already started as that request is made, because creating the pending block requires mining all of the same transactions that will be included in the already started block, the pending block will finish second to the already started block. We considered pausing the real miner to make sure the pending block returns first, but it seems silly to slow down real mining for this.

[jeffsmale90]

Partial review, I got pulled down a rabbit hole :)

[jeffsmale90]

I don't know if there's other cases where this is going to be problematic, but at least for eth_getProof, we can pass in pending block and throw an internal error: State trie does not contain state root. Note Geth does support this.

On

ganache/src/chains/ethereum/ethereum/src/api.ts

Line 1121 in 83d195b

await stateManagerCopy.setStateRoot(

we call StateManager.setStateRoot with the state root of the target block. If we pass in pending, this will be a 32 byte 00 buffer.

curl -s http://localhost:8545 -X POST -H "ContentType: application/json" -d '{"method":"eth_getProof","params":["0x1895a79379d99a344983745bf3737d058bbce3ac",["0x0","0x1"],"pending"],"id":1,"jsonrpc":"2.0"}'

{"id":1,"jsonrpc":"2.0","error":{"message":"State trie does not contain state root","stack":"Error: State trie does not contain state root\n    at DefaultStateManager.setStateRoot (/Users/jeffsmale/g/src/chains/ethereum/ethereum/node_modules/@ethereumjs/vm/src/state/stateManager.ts:442:15)\n    at async EthereumApi.eth_getProof (/Users/jeffsmale/g/src/chains/ethereum/ethereum/src/api.ts:1124:5)","code":-32700}}

[jeffsmale90]

I'm curious what the real world benefit of storing the _serialized value here is. I'm concerned that, because the block is mutable, this could result in unexpected results.

[MicaiahReid]

We serialize the block when we mine it, so this prevents us from having to do it again any time a block's raw data is needed. We need this raw data when fetching a raw block in the block manager.

All of the fields used in serialization are private and aren't currently modified, but I suppose it's true they could be. This seems worthy of adding to a discussion to weigh the risk/reward.

[jeffsmale90]

I really like how you've extended the Block class, but I feel that overriding the toJSON behaviour is insufficient - can we rather override the properties stateRoot and hash, so that we can do things like:

const block: PendingBlock = ...;
assert.strictEqual(block.hash(), AN_EMPTY_HASH);
assert.strictEqual(block.stateRoot, A_NULL_STATEROOT);

[MicaiahReid]

We are overriding the toJSON function because we still need the actual stateRoot and hash values of the pending block for use cases that traverse the trie. For example, if we were to override the stateRoot, this function in your eth_getProof RPC method wouldn't work:

 await stateManagerCopy.setStateRoot(
      targetBlock.header.stateRoot.toBuffer()
    );

However, geth returns empty data for the toJSON representation of the pending block, hence this "half-measure" override of the data.