Nodes with SPREAD pruning return incorrect historic values

[sdesmond46]

Before filing a new issue, please provide the following information.

I'm running:

• Which Aion version?: 0.3.2
• Which operating system?: Linux
• Which distributor and version?: Ubuntu 16.04
• How installed?: via binaries
• Are you fully synchronized?: yes
• Did you try to restart the node?: yes

Your issue description goes here below. Try to include actual vs. expected behavior and steps to reproduce the issue.

Many of the web3 APIs take a block number parameter which allows you to get a value at a specific block number, or "latest" to specify the most recent block. When running a node using leveldb and pruning strategy of FULL, this works properly. When changing this node to have a pruning strategy of SPREAD, it does not. The documentation says that the two modes should return the same answer (the SPREAD node would be slower), but that doesn't seem to be the case here.

Steps to Reproduce

• Run a kernel on the mainnet, json-rpc enabled, using level DB, and state-storage method spread
• Here's what the db section of the config should look like

  <db>
        <!--Sets the physical location on disk where data will be stored.-->
        <path>database</path>
        <!--Boolean value. Enable/disable database integrity check run at startup.-->
        <check_integrity>false</check_integrity>
        <!--Data pruning behavior for the state database. Options: FULL, TOP, SPREAD.-->
        <!--FULL: the state is not pruned-->
        <!--TOP: the state is kept only for the top K blocks; limits sync to branching only within the stored blocks-->
        <!--SPREAD: the state is kept for the top K blocks and at regular block intervals-->
        <state-storage>SPREAD</state-storage>
        <!--Database implementation used to store data; supported options: leveldb, h2, rocksdb.-->
        <!--Caution: changing implementation requires re-syncing from genesis!-->
        <vendor>leveldb</vendor>
        <!--Boolean value. Enable/disable database compression to trade storage space for execution time.-->
        <enable_db_compression>false</enable_db_compression>
    </db>

• Wait for this to get synced
• Run curl http://localhost:8545 -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","method":"eth_getBalance","params":["0xa00338aeb0fb1c8a47c215a0a9daece90bedc406dc31e905788a4f3d83a33943", "1532944"],"id":42}'

EXPECTED
{"jsonrpc":"2.0","result":"0x97a17c7b4d747cb","id":42}

ACTUAL
{"jsonrpc":"2.0","result":"0x0","id":42}

You can run the exact same curl command, but instead of passing in 1532944 as the block param, you pass "latest" and you'll get the correct current balance, so the database and kernel seem to be behaving properly otherwise. (curl http://localhost:8545 -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","method":"eth_getBalance","params":["0xa00338aeb0fb1c8a47c215a0a9daece90bedc406dc31e905788a4f3d83a33943", "latest"],"id":42}').

You can perform the same test on mastery using account: 0xa0fdc164e71e9649be63bab1d1d854eeb51871b79852325822eb39d40d5c2002 which had a balance of 0x4e8c1281b53f3b11 at block 1381275 (curl http://localhost:8545 -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","method":"eth_getBalance","params":["0xa0fdc164e71e9649be63bab1d1d854eeb51871b79852325822eb39d40d5c2002", 1255945],"id":42}'. Using a SPREAD node, 0x0 is returned.

Terminal output with SPREAD kernel on port 8545 and FULL kernel on 8546

Samms-MacBook:smoke-test sdesmond$ curl http://localhost:8545 -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","method":"eth_getBalance","params":["0xa0fdc164e71e9649be63bab1d1d854eeb51871b79852325822eb39d40d5c2002", 1255945],"id":42}' -s | jq
{
  "result": "0x0",
  "id": 42,
  "jsonrpc": "2.0"
}
Samms-MacBook:smoke-test sdesmond$ curl http://localhost:8546 -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","method":"eth_getBalance","params":["0xa0fdc164e71e9649be63bab1d1d854eeb51871b79852325822eb39d40d5c2002", 1255945],"id":42}' -s | jq
{
  "result": "0x4e8c1281b53f3b11",
  "id": 42,
  "jsonrpc": "2.0"
}

[AlexandraRoatis]

Yes, this is by design. Pruning the state by definition will lead to not being able to query historic information. The difference between spread and top is in the ability to easily rebuild the missing data at runtime when branching. To query historic data a full node is required. The full state can be rebuilt offline with the --state FULL CLI command.

[sdesmond46]

Yes, this is by design. Pruning the state by definition will lead to not being able to query historic information. The difference between spread and top is in the ability to easily rebuild the missing data at runtime when branching. To query historic data a full node is required. The full state can be rebuilt offline with the --state FULL CLI command.

Oh interesting, I misunderstood how rebuilding the state information worked. My assumption was that the previous state would be computed on the fly when an API call comes in asking for that information. But from how long it's taking to migrate my kernel from STREAD to FULL, I think that would require too much time for that to be feasibly.

I think at the very least, API calls asking for historic information should return an error instead of returning incorrect information with no indication that it was incorrect.

[AionJayT]

@sdesmond46

I think at the very least, API calls asking for historic information should return an error instead of returning incorrect information with no indication that it was incorrect.

That's a good call. We should add some info for this scenario.