Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update of the skipped tests lists #231

Merged
merged 2 commits into from
Nov 18, 2017
Merged

Update of the skipped tests lists #231

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2a64566
Updated Byzantium release note in README, removed Node 8 test require…
holgerd77 Nov 10, 2017
96e9f57
Removed now passing tests or tests not existing any more from the BRO…
holgerd77 Nov 10, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 13 additions & 9 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,9 @@ Implements Ethereum's VM in JS

#### Note on Byzantium Support:

The ``master`` branch of this repository has now been updated with the latest
[Byzantium changes](https://github.com/ethereumjs/ethereumjs-vm/pull/161) please install
directly from GitHub if you want to try out the latest ``VM`` version and report
issues on our [Gitter channel](https://gitter.im/ethereum/ethereumjs-lib).
This repository has now been updated with the latest
[Byzantium changes](https://github.com/ethereumjs/ethereumjs-vm/pull/161)
(``2.3.x`` [releases](https://github.com/ethereumjs/ethereumjs-vm/releases)).

For a ``Spurious Dragon``/``EIP 150`` compatible version of this library install the
latest of the ``2.2.x`` series (see [Changelog](./CHANGELOG.md)).
Expand Down Expand Up @@ -41,7 +40,8 @@ Also more examples can be found here
- [old blog post](https://wanderer.github.io/ethereum/nodejs/code/2014/08/12/running-contracts-with-vm/)

# BROWSER
To build for standalone use in the browser, install `browserify` and check [run-transactions-simple exsample](https://github.com/ethereumjs/ethereumjs-vm/tree/master/examples/run-transactions-simple). This will give you a global variable `EthVM` to use. The generated file will be at `./examples/run-transactions-simple/build.js`.

To build for standalone use in the browser, install `browserify` and check [run-transactions-simple example](https://github.com/ethereumjs/ethereumjs-vm/tree/master/examples/run-transactions-simple). This will give you a global variable `EthVM` to use. The generated file will be at `./examples/run-transactions-simple/build.js`.

# API
- [`new VM([opts])`](#new-vmstatetrie-blockchain)
Expand Down Expand Up @@ -206,8 +206,6 @@ The opFns for `CREATE`, `CALL`, and `CALLCODE` call back up to `runCall`.

### Running Tests

_Note: Requires at least Node.js `8.0.0` installed to run the tests, this is because `ethereumjs-testing` uses `async/await` and other ES2015 language features_

Tests can be found in the ``tests`` directory, with ``FORK_CONFIG`` set in ``tests/tester.js``. There are test runners for [State tests](http://www.ethdocs.org/en/latest/contracts-and-transactions/ethereum-tests/state_tests/index.html) and [Blockchain tests](http://www.ethdocs.org/en/latest/contracts-and-transactions/ethereum-tests/blockchain_tests/index.html). VM tests are disabled since Frontier gas costs are not supported any more. Tests are then executed by the [ethereumjs-testing](https://github.com/ethereumjs/ethereumjs-testing) utility library using the official client-independent [Ethereum tests](https://github.com/ethereum/tests).

For a wider picture about how to use tests to implement EIPs you can have a look at this [reddit post](https://www.reddit.com/r/ethereum/comments/6kc5g3/ethereumjs_team_is_seeking_contributors/)
Expand Down Expand Up @@ -259,23 +257,27 @@ can be found in ``tests/tester.js``. By default tests from all skip lists are om

You can change this behaviour with:

`node tests/tester -s --skip=BROKEN,PERMANENT`˘
`node tests/tester -s --skip=BROKEN,PERMANENT`

to skip only the ``BROKEN`` and ``PERMANENT`` tests and include the ``SLOW`` tests.
There are also the keywords ``NONE`` or ``ALL`` for convenience.

It is also possible to only run the tests from the skip lists:

`node tests/tester -s runSkipped=SLOW`
`node tests/tester -s --runSkipped=SLOW`

### Debugging

#### Local Debugging

Blockchain tests support `--debug` to verify the postState:

`node ./tests/tester -b --debug --test='ZeroValue_SELFDESTRUCT_ToOneStorageKey_OOGRevert_d0g0v0_EIP158'`

All/most State tests are replicated as Blockchain tests in a ``GeneralStateTests`` [sub directory](https://github.com/ethereum/tests/tree/develop/BlockchainTests/GeneralStateTests) in the Ethereum tests repo, so for debugging single test cases the Blockchain test version of the State test can be used.

#### Debugging Tools

For comparing ``EVM`` traces [here](https://gist.github.com/cdetrio/41172f374ae32047a6c9e97fa9d09ad0) are some instructions for setting up ``pyethereum`` to generate corresponding traces for state tests.

Compare TAP output from blockchain/state tests and produces concise diff of the differences between them (example):
Expand All @@ -286,6 +288,8 @@ curl https://gist.githubusercontent.com/jwasinger/e7004e82426ff0a7137a88d273f118
python utils/diffTestOutput.py output-wip-byzantium.txt output-master.txt
```

An extremely rich and powerful toolbox is the [evmlab](https://github.com/holiman/evmlab) from ``holiman``, both for debugging and creating new test cases or example data.


# LICENSE
[MPL-2.0](https://www.mozilla.org/MPL/2.0/)
42 changes: 6 additions & 36 deletions tests/tester.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,52 +6,21 @@ const FORK_CONFIG = argv.fork || 'Byzantium'
// tests which should be fixed
const skipBroken = [
'CreateHashCollision', // impossible hash collision on generating address
'TransactionMakeAccountBalanceOverflow',
'RecursiveCreateContracts',
'sha3_bigSize',
'createJS_ExampleContract', // creates an account that already exsists
'mload32bitBound_return',
'mload32bitBound_return2',
'QuadraticComplexitySolidity_CallDataCopy', // tests hash collisoin, sending from a contract
'uncleBlockAtBlock3AfterBlock3',
'ForkUncle', // correct behaviour unspecified (?)
'UncleFromSideChain', // same as ForkUncle, the TD is the same for two diffent branches so its not clear which one should be the finally chain
'bcSimpleTransitionTest', // HF stuff
'CALL_Bounds', // nodejs crash
'CALLCODE_Bounds', // nodejs crash
'CREATE_Bounds', // nodejs crash
'CreateCollisionToEmpty', // temporary till fixed (2017-09-21)
'TransactionCollisionToEmptyButCode', // temporary till fixed (2017-09-21)
'TransactionCollisionToEmptyButNonce', // temporary till fixed (2017-09-21)
'randomStatetest642', // temporary till fixed (2017-09-25)
'DELEGATECALL_Bounds', // nodejs crash
'RevertDepthCreateAddressCollision', // test case is wrong
'zeroSigTransactionInvChainID', // metropolis test
'randomStatetest643',
'static_CreateHashCollision', // impossible hash collision on generating address
'static_TransactionMakeAccountBalanceOverflow',
'static_RecursiveCreateContracts',
'static_sha3_bigSize',
'static_createJS_ExampleContract', // creates an account that already exsists
'static_mload32bitBound_return',
'static_mload32bitBound_return2',
'static_QuadraticComplexitySolidity_CallDataCopy', // tests hash collisoin, sending from a contract
'static_uncleBlockAtBlock3AfterBlock3',
'static_ForkUncle', // correct behaviour unspecified (?)
'static_UncleFromSideChain', // same as ForkUncle, the TD is the same for two diffent branches so its not clear which one should be the finally chain
'static_bcSimpleTransitionTest', // HF stuff
'static_CALL_Bounds', // nodejs crash
'static_CALLCODE_Bounds', // nodejs crash
'static_CREATE_Bounds', // nodejs crash
'static_DELEGATECALL_Bounds', // nodejs crash
'static_RevertDepthCreateAddressCollision', // test case is wrong
'static_zeroSigTransactionInvChainID', // metropolis test
'zeroSigTransactionInvChainID' // metropolis test
'randomStatetest642', // BROKEN, rustbn.js error
'randomStatetest643' // BROKEN, breaks tests run (leave at the end), rustbn.js error
]
// tests skipped due to system specifics / design considerations
const skipPermanent = [
'SuicidesMixingCoinbase', // sucides to the coinbase, since we run a blockLevel we create coinbase account.
'static_SuicidesMixingCoinbase' // sucides to the coinbase, since we run a blockLevel we create coinbase account.
'static_SuicidesMixingCoinbase', // sucides to the coinbase, since we run a blockLevel we create coinbase account.
'ForkUncle', // Only BlockchainTest, correct behaviour unspecified (?)
'UncleFromSideChain' // Only BlockchainTest, same as ForkUncle, the TD is the same for two diffent branches so its not clear which one should be the finally chain
]
// tests running slow (run from time to time)
const skipSlow = [
Expand Down Expand Up @@ -121,6 +90,7 @@ const skipVM = [
'ABAcalls0',
'ABAcallsSuicide0',
'ABAcallsSuicide1',
'sha3_bigSize',
'CallRecursiveBomb0',
'CallToNameRegistrator0',
'CallToPrecompiledContract',
Expand Down