The prestateTracer under diffMode does not return all the storage diffs.

[lum7na]

System information

Geth version: 1.14.12
OS & Version: Linux

Expected behavior

Hi! I'm trying to debug the transaction using debug_traceTransaction. I use the built-in prestateTracer and enable diffMode. I notice in the post section of the returned result, an address that should have been modified was ignored.

The following modifications occurred after executing the transaction setRational(-34342247582603709930262480515100532418408549587488298634819624648327228197968,54)

0x0000000000000000000000000000000000000000000000000000000000000000 
 -> 0xb412fa0861bb1532171e2f7e6d88df75e8bb79d4ba10674f506b1d13a2d973b0
0x0000000000000000000000000000000000000000000000000000000000000001
 -> 0x0000000000000000000000000000000000000000000000000000000000000036

The debug_traceTransaction RPC call returned the correct result. However, if I continue to execute the transaction setRational(0,-3686082785242131774829242764878224605745458421934561058325782450253198843211). The state modification should be:

0x0000000000000000000000000000000000000000000000000000000000000000
 -> 0x0000000000000000000000000000000000000000000000000000000000000000
0x0000000000000000000000000000000000000000000000000000000000000001 
 -> 0xf7d9c0469c411e6b7d7863011ce7f3f258685e4ce99f5467f28c3598787722b5

However, debug_traceTransaction only returned one modification in the post section: {'0x0000000000000000000000000000000000000000000000000000000000000001': '0xf7d9c0469c411e6b7d7863011ce7f3f258685e4ce99f5467f28c3598787722b5'}. It ignored the modification of 0x0000000000000000000000000000000000000000000000000000000000000000.

pragma  abicoder v2;

contract TestRational {
  struct Rational { int256 numerator; int256 denominator;  }

  Rational r;

  function getRational() public returns(Rational memory) {
    return r;
  }

  function setRational(int256 num, int256 denom) public {
    require(denom != 0, hex"44656e6f6d696e61746f722063616e6e6f74206265207a65726f");
    r.numerator = num;
    assembly  { { if 0x10000000001 { } } }
    bytes32 v_69452 = keccak256(abi.encodePacked(msg.sender));
    r.denominator = denom;
  }

  constructor () {
    r.numerator = -1;
    r.denominator = 2;
  }

}

trace = w3.manager.request_blocking('debug_traceTransaction', [tx_hash, {'tracer':'prestateTracer','tracerConfig': {'diffMode': True}}])

Result:

AttributeDict({'post': AttributeDict({'0x0000000000000000000000000000000000000000': AttributeDict({'balance': '0x355dacaa5c6e'}), '0x5c9776deac45c8b416239604cd8850d5dc81d3fb': AttributeDict({'storage': AttributeDict({'0x0000000000000000000000000000000000000000000000000000000000000001': '0xf7d9c0469c411e6b7d7863011ce7f3f258685e4ce99f5467f28c3598787722b5'})}), '0xbe931d46286d4e27217d57d1a4672b4d536e46fd': AttributeDict({'balance': '0xfffffffffffffffffffffffffffffffffffffffffffffffffffe84c17977809d', 'nonce': 3})}), 'pre': AttributeDict({'0x0000000000000000000000000000000000000000': AttributeDict({'balance': '0x2f9aa6ccc4bb'}), '0x5c9776deac45c8b416239604cd8850d5dc81d3fb': AttributeDict({'balance': '0x0', 'code': '0x60806040526004361015610013575b610166565b61001d5f3561003c565b8063d492613c146100375763d61f5f420361000e57610131565b6100a8565b60e01c90565b60405190565b5f80fd5b5f80fd5b90565b61005c81610050565b0361006357565b5f80fd5b9050359061007482610053565b565b919060408382031261009e578061009261009b925f8601610067565b93602001610067565b90565b61004c565b5f0190565b346100d7576100c16100bb366004610076565b90610312565b6100c9610042565b806100d3816100a3565b0390f35b610048565b5f9103126100e657565b61004c565b6100f490610050565b9052565b9060208061011a936101105f8201515f8601906100eb565b01519101906100eb565b565b919061012f905f604085019401906100f8565b565b34610161576101413660046100dc565b61015d61014c610458565b610154610042565b9182918261011c565b0390f35b610048565b5f80fd5b90565b90565b61018461017f6101899261016a565b61016d565b610050565b90565b60209181520190565b5f7f44656e6f6d696e61746f722063616e6e6f74206265207a65726f000000000000910152565b6101c9601a60209261018c565b6101d281610195565b0190565b6101eb9060208101905f8183039101526101bc565b90565b156101f557565b6101fd610042565b62461bcd60e51b815280610213600482016101d6565b0390fd5b5f1b90565b906102285f1991610217565b9181191691161790565b61024661024161024b92610050565b61016d565b610050565b90565b90565b9061026661026161026d92610232565b61024e565b825461021c565b9055565b60018060a01b031690565b61028590610271565b90565b60601b90565b61029790610288565b90565b6102a39061028e565b90565b6102b26102b79161027c565b61029a565b9052565b6102c7816014936102a6565b0190565b601f801991011690565b634e487b7160e01b5f52604160045260245ffd5b906102f3906102cb565b810190811067ffffffffffffffff82111761030d57604052565b6102d5565b906501000000000161034661037e9361033e846103376103315f610170565b91610050565b14156101ee565b5f8001610251565b610380575b6103756103663361035a610042565b918291602083016102bb565b602082018103825203906102e9565b60015f01610251565b565b61034b565b90610398610391610042565b92836102e9565b565b6103a46040610385565b90565b5f90565b6103b361039a565b90602080836103c06103a7565b8152016103cb6103a7565b81525050565b6103d96103ab565b90565b5f1c90565b90565b6103f06103f5916103dc565b6103e1565b90565b61040290546103e4565b90565b9061040f90610050565b9052565b9061044a610441600161042461039a565b9461043b6104335f83016103f8565b5f8801610405565b016103f8565b60208401610405565b565b61045590610413565b90565b6104606103d1565b5061046a5f61044c565b9056fea26469706673582212202158c4b7c478cf652bcc8e34d94c08914dd90ea48eaa634be1a206ada09f359764736f6c634300081b0033', 'nonce': 1, 'storage': AttributeDict({'0x0000000000000000000000000000000000000000000000000000000000000000': '0xb412fa0861bb1532171e2f7e6d88df75e8bb79d4ba10674f506b1d13a2d973b0', '0x0000000000000000000000000000000000000000000000000000000000000001': '0x0000000000000000000000000000000000000000000000000000000000000036'})}), '0xbe931d46286d4e27217d57d1a4672b4d536e46fd': AttributeDict({'balance': '0xfffffffffffffffffffffffffffffffffffffffffffffffffffe9c276179cd9e', 'nonce': 2})})})

[lightclient]

Seems empty values are explicitly removed:

go-ethereum/eth/tracers/native/prestate.go

Lines 234 to 237 in e3d61e6

	// don't include the empty slot
	if val == (common.Hash{}) {
	delete(t.pre[addr].Storage, key)
	}

would ask @s1na, @jsvisa for more context.

Looks to always have been this way: #25422

[jsvisa]

Yeah, the cleared post storage were not included:

go-ethereum/eth/tracers/native/prestate.go

Lines 245 to 247 in e3d61e6

	if newVal != (common.Hash{}) {
	postAccount.Storage[key] = newVal
	}

[lightclient]

Honestly I also stumbled across this a few weeks ago. What was the reasoning for not just including a simple diff w/o removing zero values ? Is it okay to remove these clauses?

[jsvisa]

cleared slots will not show up in post storage

@lightclient I think this is proposed by @s1na in #25422 (comment)

But here, we can't distinguish the cleared and modified to zero differ, so the modified to zero treated as the clear case.

Is it okay to remove these clauses

As the other clients also follow geth's implementation, so I think let's keep as it is right now. We should add more instructions of the diffMode in geth's builtin tracers https://geth.ethereum.org/docs/developers/evm-tracing/built-in-tracers#prestate-tracer-config

[jsvisa]

@lightclient after some digging, I think we should remove the if clause, because in post's state, if a slot is cleared that is also a modification, and a modification should be included in the diffMode, else user will think this slot is touched but without modification.

[s1na]

I think it is correct as it stands. As a general rule:

• If it exists in pre but not post (what this issue is about): that thing has been deleted during execution.
• If it exists only in post but not pre: it has been created during execution.
• If it exists in both: it has been modified.

In the case above a storage slot is cleared. You can tell this has happened because the value for that slot is returned in pre but no mention of it in post.

You have to take both of these maps together to build the whole picture.

[jsvisa]

I see, I got @lum7na 's point, in his mind, he just set the value from x to y(here is zero), so he would require a slot diff of x->y, but when the y is zero, then in EVM this was treated as a slot clear logical, so we omit the slot in the post map.

After all, I agree with @s1na, let me add more documents in the built-in tracers, sorry for the inconvenience.

[jsvisa]

pre can include fields that were not modified, if some other fields for the same account were modified.

In the meanwhile, I was confused with this one in https://geth.ethereum.org/docs/developers/evm-tracing/built-in-tracers#prestate-tracer-config

I think those fields were code, balance, nonce, let's see a example as below:

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;

import "forge-std/Script.sol";
import {console} from "forge-std/console.sol";

contract TestRational {
    constructor() {
        r.numerator = -1;
        r.denominator = 2;
    }

    struct Rational {
        int256 numerator;
        int256 denominator;
    }

    Rational r;

    function getRational() public view returns (Rational memory) {
        return r;
    }

    function setRational(int256 num, int256 denom) public {
        require(denom != 0, hex"44656e6f6d696e61746f722063616e6e6f74206265207a65726f");
        r.numerator = num;
        assembly {
            {
                if 0x10000000001 {}
            }
        }
        bytes32 v_69452 = keccak256(abi.encodePacked(msg.sender));
        r.denominator = denom;
    }

    function setRational2(int256 num, int256 denom) public payable {
        require(denom != 0, hex"44656e6f6d696e61746f722063616e6e6f74206265207a65726f");
        if (r.numerator == num) {
            return;
        }
        r.numerator = num;
    }
}

contract Issue30774 is Script {
    function run() external {
        vm.startBroadcast();

        TestRational testRational = new TestRational();
        testRational.setRational(1, 1);
        testRational.setRational2{value: 1}(1, 1);
        vm.stopBroadcast();

        console.log("Rational");
    }
}

In the second setRational2 we sent 1wei to the contract, and the diff result as below:

{
  post: {
    0x0000000000000000000000000000000000000000: {
      balance: "0x233e16af82a"
    },
    0x040528d526a5dfc315bd990230f5aa285d168173: {
      balance: "0x1"
    },
    0xf077b491b355e64048ce21e3a6fc4751eeea77fa: {
      balance: "0x1a0555aa0149ed972",
      nonce: 18
    }
  },
  pre: {
    0x0000000000000000000000000000000000000000: {
      balance: "0x231420db60c"
    },
    0x040528d526a5dfc315bd990230f5aa285d168173: {
      balance: "0x0",
      code: "0x608060405260043610610033575f3560e01c80638702b6ed14610037578063d492613c1461004c578063d61f5f421461006b575b5f80fd5b61004a6100453660046101a9565b6100bd565b005b348015610057575f80fd5b5061004a6100663660046101a9565b610123565b348015610076575f80fd5b506040805180820182525f80825260209182018190528251808401909352548252600154908201526040805182518152602092830151928101929092520160405180910390f35b805f036101115760405162461bcd60e51b815260206004820152601a60248201527f44656e6f6d696e61746f722063616e6e6f74206265207a65726f00000000000060448201526064015b60405180910390fd5b5f5482900361011e575050565b505f55565b805f036101725760405162461bcd60e51b815260206004820152601a60248201527f44656e6f6d696e61746f722063616e6e6f74206265207a65726f0000000000006044820152606401610108565b5f8281556040516bffffffffffffffffffffffff193360601b16602082015260340160408051601f19818403019052525060015550565b5f80604083850312156101ba575f80fd5b5050803592602090910135915056fea2646970667358221220230f73022bcc1a72523beb8ef0ed6be1de384e721e28a402073c7761e1eb272e64736f6c63430008190033",
      nonce: 1
    },
    0xf077b491b355e64048ce21e3a6fc4751eeea77fa: {
      balance: "0x1a0555aa806de09d7",
      nonce: 17
    }

The code and nonce were not modified inside this transaction, but were included in the pre, I think we should only include the modified fields, remove the unmodified fields from the pre object as well. Or as considered of back compatibility, we can add more documents. WDYT @s1na

[s1na]

The code and nonce were not modified inside this transaction, but were included in the pre, I think we should only include the modified fields, remove the unmodified fields from the pre object as well. Or as considered of back compatibility, we can add more documents. WDYT @s1na

Hmm I see your point. Generally the rule I stated above applies to the account level and the storage slot level. It doesn't work for individual account fields. Since account balance/nonce etc are not created and destroyed in the same way.

Thinking about it showing only the fields that are modified is probably cleaner. I'm not sure why we went with showing the full account in pre. However I'm hesitant about changing it now specially because it doesn't "unlock" something, it doesn't add something rather we remove something that might not be necessary. I don't hold this opinion strongly though and can be convinced otherwise.

[s1na]

@lum7na Can you please confirm if this is still an issue for you? I will try to summarize why you're getting the result you are getting.

The state transition that happens in your tx is as follows:

0x0000000000000000000000000000000000000000000000000000000000000000
 -> 0x0000000000000000000000000000000000000000000000000000000000000000
0x0000000000000000000000000000000000000000000000000000000000000001 
 -> 0xf7d9c0469c411e6b7d7863011ce7f3f258685e4ce99f5467f28c3598787722b5

You are surprised that the state diff returned by the prestateTracer only includes 0x0000000000000000000000000000000000000000000000000000000000000000 in the pre but its post state is not available in post.

The reason is that this is considered a deletion of the storage slot. Deletions are represented in the state diff in this way where they only appear in pre and not in post. So you have to compare these 2 objects to arrive at the diff.

Unless it is an issue I will close this ticket. @jsvisa has made some updates to the website docs to clarify this aspect.

[lum7na]

Hello! I have resolved my issue. I just find the behavior here a bit strange, sometimes modifying to 0 does not mean deletion.

[s1na]

Thanks for confirming.