Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement bitwise shifting #251

Merged
merged 6 commits into from
Jul 24, 2018
Merged

Implement bitwise shifting #251

merged 6 commits into from
Jul 24, 2018

Conversation

axic
Copy link
Member

@axic axic commented Jan 26, 2018

@axic
Copy link
Member Author

axic commented Jan 26, 2018

Fails because of the stBadOpcodes test.

@axic
Copy link
Member Author

axic commented Jan 26, 2018

@holgerd77 @cdetrio @jwasinger this would be enabled for constantinople only. How to do it?

@holgerd77
Copy link
Member

@axic Do you want to have this already implemented for testing purposes or anything? Or can't we just leave the PR open until Constantinople is more in sight?

@axic
Copy link
Member Author

axic commented Jan 26, 2018

No need to merge yet until there are some state tests for it and it is fully implemented. Not sure even at that point how can we merge it since there's only one version supported: the last.

But if you want you can implement the SHR part and review the rest based on the EIP.

@holgerd77
Copy link
Member

Just rebased this.

@axic
Copy link
Member Author

axic commented Jul 20, 2018

Using #304 features here now.

@axic
Copy link
Member Author

axic commented Jul 20, 2018

@holgerd77 do you want to check if the API is used correctly?

@holgerd77
Copy link
Member

This isn't prepared to work with block numbers yet (also since in a simulated environment (Truffle,...) block numbers are not correctly corresponding to the desired HF behaviour), this is something for later down the road once this becomes relevant for the client.

For now this compares against the hardfork set in the common class, so do something like:

if (runState._common.gteHardfork('constantinople') {
   // Do the constantinople stuff here
}

I generally had the idea to refactor the opFns.js file and directory structure a bit for this with the following structure in mind, replacing the current lib/opFns.js file:

  • lib/opFns/byzantium.js Includes (for now) all the current opcodes for Byzantium, might be split further in the future
  • lib/opFns/constantinople.js New Constantinople opcodes
  • lib/util.js all the generic functions from lib/opFns.js

With this we would have a cleaner structure and separation and could add the opcodes for the HF selected in runCode.js with something like:

if (runState._common.gteHardfork('constantinople') {
  opFns = Object.assign({}, require('lib/opFns/constantinople.js'))
}

(First shot, not thoroughly thought through)
and the do the checks there centrally.

What do you think?

@holgerd77
Copy link
Member

Regarding the test setup we can also now running at least the state tests for both hardforks in parallel (especially with the now fast-running Circle tests) and keep the Byzantium tests activated to be mandatory for passing CI until all Constantinople tests pass.

So we don't have to change the hardfork setting in the tester file directly, but we can introduce another package.json script + Circle task, maybe something generic to be future-ready like:

{
  "testState": "npm run build:dist && node ./tests/tester -s --dist --fork='byzantium'",
  "testStateDev": "npm run build:dist && node ./tests/tester -s --dist --fork='constantinople"",
}

...plus a new Circle job test_state_dev. And in the protected branch setting we let the dev tests run but don't mark them as mandatory. Like this we always have an overview on the increase of Constantinople tests passing while making sure to not break the current Byzantium build.

@axic
Copy link
Member Author

axic commented Jul 23, 2018

I generally had the idea to refactor the opFns.js file and directory structure a bit for this with the following structure in mind, replacing the current lib/opFns.js file:

I think that idea makes sense from an optimisation perspective, but right now I would think that adding single conditions achieve the goal, in terms of both readability and speed of implementing the new features.

@axic
Copy link
Member Author

axic commented Jul 23, 2018

This isn't prepared to work with block numbers yet (also since in a simulated environment (Truffle,...) block numbers are not correctly corresponding to the desired HF behaviour), this is something for later down the road once this becomes relevant for the client.

In that case I'd think a helper on runState could do the trick: runState.hardforkEnabled('...'). That helps the block number logic to be hidden in a higher level.

@holgerd77
Copy link
Member

@axic I think there should be currently no needs for something to be hidden if we use the API methods I describe above, like

if (runState._common.gteHardfork('constantinople') {
   // Do the constantinople stuff here
}

Or am I missing your point?

@axic
Copy link
Member Author

axic commented Jul 23, 2018

The block number question could be solved (and decided) by the caller in that case.

@holgerd77
Copy link
Member

Atm I don't really see a "block number question"? Or do you already want to start integrating dynamic fork switch support depending on block number? At the moment I would envision this for just statically instantiate the VM with Byzantium or Constantinople, and then everything is set. Or am I targeting too low?

@axic
Copy link
Member Author

axic commented Jul 23, 2018

Didn't know gteHardfork existed (it is not documented). Looks good for this usecase.

@axic axic force-pushed the bitwise branch 2 times, most recently from 24751c6 to 76da77d Compare July 23, 2018 19:31
@coveralls
Copy link

coveralls commented Jul 23, 2018

Coverage Status

Coverage decreased (-0.8%) to 84.814% when pulling d843cd2 on bitwise into 8d5f330 on master.

@axic
Copy link
Member Author

axic commented Jul 23, 2018

Since SAR was not implemented and wasn't returning a stack item, the stack check code in runCode kicked in, but it was never tested properly and had missing instantiation of VMError:

 file: sar01 test: sar01
/home/circleci/project/ethereumjs-vm/dist/exceptions.js:15
  this.error = error;
             ^

TypeError: Cannot set property 'error' of undefined
    at VmError (/home/circleci/project/ethereumjs-vm/dist/exceptions.js:15:14)
    at runOp (/home/circleci/project/ethereumjs-vm/dist/runCode.js:221:21)

@holgerd77
Copy link
Member

@axic
Copy link
Member Author

axic commented Jul 23, 2018

This passes the constantinople tests now, woohoo!

@axic axic changed the title [WIP] Implement bitwise shifting Implement bitwise shifting Jul 23, 2018
@axic axic requested a review from holgerd77 July 23, 2018 20:02
@axic
Copy link
Member Author

axic commented Jul 23, 2018

Reason for coverage decrease: coveralls is only running on test_state_byzantium.

@holgerd77
Copy link
Member

Congrats! 👍

Just double checked that all the relevant tests are passing, these should be sar_*, shl_* and shr_* I suppose.

We should really change the test summary behavior so that skipped tests are not displayed as passed ones - which currently is the case. Will see if I can make a PR here, shouldn't be that much of a change.

@axic
Copy link
Member Author

axic commented Jul 24, 2018

We should really change the test summary behavior so that skipped tests are not displayed as passed ones - which currently is the case. Will see if I can make a PR here, shouldn't be that much of a change.

Should that be part of this PR?

@holgerd77
Copy link
Member

Ah, just checked this manually, changing t.comment(`No ${options.forkConfig} post state defined, skip test`) in GeneralStateTestRunner.js to t.skip(...) doesn't make things better, since this is then counted as passed.

I was actually wrong on this, skipped tests are atm not counted or reported at all. This is an open (for a long time) issue on tape: tape-testing/tape#197

So we would have to manually add a counter or something, seems not worth it for the moment, this should better be added on tape itself (hopefully sometime).

@axic
Copy link
Member Author

axic commented Jul 24, 2018

So can this be merged?

holgerd77
holgerd77 previously approved these changes Jul 24, 2018
Copy link
Member

@holgerd77 holgerd77 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

if (a.gten(256)) {
return new BN(0)
}
return b.shln(a.toNumber()).iand(utils.MAX_INTEGER)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/ethereum/EIPs/blob/master/EIPS/eip-145.md#0x1b-shl-shift-left

I am not super-fluent on bit operations, but I think I have found all conditions from the EIP in the implementation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(always a bit shaky on this signed/unsigned stuff)

if (a.gten(256)) {
return new BN(0)
}
return b.shrn(a.toNumber())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

} else {
return c
}
},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/ethereum/EIPs/blob/master/EIPS/eip-145.md#0x1d-sar-arithmetic-shift-right

I also think I can roughly follow what is going on here and think all EIP conditions are covered and can also place all - this to me appearing a bit esoteric - mask stuff to the places in the EIP. Would have to do an extra hour of bit operation reading though for a better understanding.

Since tests are passing I will approve.

Copy link
Member Author

@axic axic Jul 24, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here the point is to fill the top with set bits, that is achieved by ORing the mask. The mask is then created by taking an all bits set input (MAX_INTEGER) and making sure the bottom part where actual data remainded is 0.

@holgerd77
Copy link
Member

Ah, just saw your question, did the review before. If you rebase, I can do a quick re-approval and we can merge if you want.

@axic
Copy link
Member Author

axic commented Jul 24, 2018

@holgerd77 Rebased.

Copy link
Member

@holgerd77 holgerd77 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renewed approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants