Improve chain and fork support

[holgerd77]

Summary
This PR replaces the old common library with the new ethereumjs-common library and introduces two new VM options chain and hardfork.

This will allow for the following:

• In-parallel development on future HF features
• Modular HF dev with separated PRs, no need for one big switch-over PR
• Parallel support of different HFs (necessary for client) at least from Byzantium onwards
• Easier release management on HF updates

Note on API design
The API extension for the moment has kept simple with either statically setting chain and hardfork or by default falling back to a mainnet byzantium setting - also for the sake of backwards compatibility. In the future this can be extended to allow the option to directly derive these settings from the blockchain object passed and directly derive the HF from the block number of the processed block, e.g. to run the VM in a client context.

Tests
Tests can now be run with the fork option, e.g.:

node tests/tester.js --fork='Constantinople' -s --test='stackOverflow'

The hardfork parameter is then passed on to the VM (command from the above is currently breaking though, since we are not yet running with the tests having Constantinople states included, so this can only be tested with a custom state test json file).

In the future it might be an idea to add the Constantinople test runs in parallel to Circle (Travis is too slow and Circle is not breaking the build if tests are not passing). Like this we have the possibility to merge if Byzantium tests remain passing and at the same time have the test output for Constantinople for checking progress for the specific PR.

EDIT:
This is now open for review. I have tested every code change/parameter call updates not being touched by state or BC tests (e.g. updated parameter calls in payOmmersAndMiner()) at least once by manually triggering the respective code parts (we really really need some additional tests to test the external API, but thats another construction site 😄 ).

[coveralls]

Coverage decreased (-0.2%) to 85.59% when pulling 0b3c5d1 on improve-chain-and-fork-support into d92e7b5 on master.

[holgerd77]

@cdetrio @jwasinger Would be really nice to have a review on this, even if it's just a short look + opinion. I would otherwise merge early next week, this is blocking further development.

[jwasinger]

IMO this should just be a static map from hard fork -> costs. It doesn't seem to make sense to instantiate it as an object and then attach it to the current runState since it is external to the call state and static for the lifetime of a transaction.

[holgerd77]

This is deeply rooted in the new ethereumjs-common library and there are various reasons for why this is designed to be created as an object, please have a closer look into the library if you want to understand what it does and the reasons for the design choices.

[holgerd77]

One main thing it does is to encapsulate the hardfork choice switch, otherwise you would have to do in the using library (here: the VM) every time you want to access an parameter value, like:

if (vm.hardfork === 'byzantium')  {
  // read param from byzantium.js file
} if (vm.hardfork === 'constantinople') {
  // ...
}

With instantiating the Common library with network and the hardfork, this is further done internally and the library makes the correct choice with:

c.param('pow', 'minerReward')

[jwasinger]

It is much easier to have the costs structured like this:

const ethereumjsCommon = {
  'Byzantium': {
    ...
  },
  'Constantinople': {
    'blockReward': ...,
     .... 
  },
  ...
}

At the very least, if this object could be instantiated once when it is imported and not attached to the runState, I think it would be cleaner and more intuitive.

[jwasinger]

One main thing it does is to encapsulate the hardfork choice switch, otherwise you would have to do in the using library (here: the VM) every time you want to access an parameter value, like:

if (vm.hardfork === 'byzantium')  {
  // read param from byzantium.js file
} if (vm.hardfork === 'constantinople') {
  // ...
}

Or you have all HF parameters in one file with the JSON structure that I pasted up above. Then you don't need to do any if statement like this.

But... I digress. It would be great to have the common parameters as a global const variable. Not attached to runState. Then I think this can be merged.

[holgerd77]

@jwasinger Could you move on with this, I would like to move on to Constantinople work and would need this as a basis. I really can't abstain attaching this to the run state, otherwise I would have to re-create the whole commons library and would loose much of the benefits from there, and this was in development for weeks.

The commons library has been reviewed thoroughly by Vinay and the API constructors used here are already in production on the latest block library release, so I would like to introduce an analogue constructor instantiation in other libraries so that things stay consistent.

I actually thought about this more to be a you-guys-should-at-least-perceive-once-whats-happening here review and not as a tear-everything-apart review, hehe. 😄 Nevertheless you are of course free to bring on further criticism if you stumble on things not thought through thoroughly.

[axic]

If this has hardfork and suppertedHardforks as an argument, I'd assume it returns nil if hardfork is not part of the supported ones. Should this code here check if the call failed or returned nil meaning an unsupported fork was chosen?

[axic]

Answer is here:

This will e.g. throw an error when a param is requested for an unsupported hardfork and like this prevents unpredicted behaviour.

Though perhaps a comment here may be good.

[holgerd77]

Will make this more clear in the constructor documentation.

[holgerd77]

@axic Added the comment you suggested to the JSDoc and README. Do you also want to have a broader look on this? I would otherwise merge within 24h as announced above.

[holgerd77]

Just rebased this.

[holgerd77]

@axic @jwasinger @cdetrio @hugo-dc If there is no further discussion around this I would merge this next Monday or Tuesday, I want to start on the Constantinople work and I need this as a basis.

Not sure if you guys have already seen the proposed timeline from today's Core Devs Meeting, where Constantinople EIPs will probably be finalized.

I think I will flesh out issues on the VM for the different EIPs (which after this PR is merged can then be handled separately without breaking the current Byzantium VM) and then do a call for contribution on Reddit. I think that people from the community join to help is the best chance that this is done at least somewhat timely.

Would love to have this ready within the proposed client development time (4 weeks, sigh), so that people can use this in the dev tools. Not sure if this is realistic, I will have to step down close to zero during that exact time and contributions from new hires will also realistically not yet be done within that timeframe.

We'll see. 😄

[holgerd77]

At least at the beginning of testnet (September 10th) should be targeted as a second goal for final VM release date.

[holgerd77]

Made a comment ethereum/pm#50 (comment) on EthereumJS VM Constantinople status and a time estimation over on the Core Dev Meeting issue page. Can't join myself, if someone of you is (probably) joining would be nice if he could take up on this.

[holgerd77]

Have rebased this with the latest state manager changes, will merge once tests have run through.