EIP-1011: Hybrid Casper FFG / Ethash

[5chdn]

http://eips.ethereum.org/EIPS/eip-1011

[rphmeier]

I can take it (edit: busy with Polkadot stuff; can mentor implementation)

[5chdn]

I'm on it to figure out the specs.

[vbuterin]

Would be happy to provide any assistance or support.

We will try to prioritize improving documentation asap. In the mean time, there's the paper and the python code:

http://github.com/karlfloersch/docker-pyeth-dev
https://github.com/karlfloersch/pyethereum
https://github.com/karlfloersch/pyethereum/tree/develop/ethereum/hybrid_casper
http://github.com/ethereum/casper

I recommend starting with the fork choice rule (priority 1: highest justified epoch #, priority 2: max total diff) and the validator logic.

[5chdn]

cc @pipermerriam

[5chdn]

http://notes.eth.sg/MYEwhswJwMzAtADgCwEYBM9kAYBGJ4wBTETKdGZdXAVmRvUQDYg=?view

https://arxiv.org/abs/1710.09437

[rphmeier]

The paper is pretty good documentation for a start.

Initial engineering concerns:

• BlockChain currently holds the whole fork-choice decision, based only on total difficulty. Parts of this will need to be deferred to the Engine.
• Engine needs to get a persistent data store to keep track of any votes targeting non-finalized checkpoints to avoid violating slashing conditions. This brings complication due to snapshot restoration replacing the DB in-place.

[vbuterin]

does a smart contract keep track of votes and justified checkpoints?

Yes. The Casper contract basically keeps track of everything.

How are light clients meant to track finality?

Probably via a special-purpose protocol, like randomly selecting 50 validator indices and verifying that valid votes for enough of those validators exist.

Engine needs to get a persistent data store to keep track of any votes

targeting non-finalized checkpoints to avoid violating slashing conditions. You can simplify this by simply tracking the highest epoch you're used as a target and the highest epoch you've used as a source; that way there's only 2 variables that you need to figure out how to persist.

…

On Mon, Jan 1, 2018 at 3:02 PM, Robert Habermeier ***@***.***> wrote: The paper is pretty good documentation for a start, but I haven't dug into how votes are broadcast/accumulated in practice -- does a smart contract keep track of votes and justified checkpoints? or does a full node have to track all justified checkpoints internally? How are light clients meant to track finality? Engineering concerns: - BlockChain currently holds the whole fork-choice decision, based only on total difficulty. Parts of this will need to be deferred to the Engine. - Engine needs to get a persistent data store to keep track of any votes targeting non-finalized checkpoints to avoid violating slashing conditions. This brings complication due to snapshot restoration replacing the DB in-place. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#7162 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACIKbqSvvKBFORkhxZwjyaawxj1H2CFgks5tGTnhgaJpZM4QupmR> .

[rphmeier]

Probably via a special-purpose protocol, like randomly selecting 50
validator indices and verifying that valid votes for enough of those
validators exist.

@vbuterin
This does imply that light clients would require lots of old state to be available. the way we architecture Parity right now, we model dynasty changes as a signal (log event) which is applied automatically upon finality. State proofs of a new validator set are kept in a compressed form by full nodes to ensure general availability. For engines like Aura or Tendermint this is fairly easy because finality is determined outside of the chain's state.

In CasperFFG this will definitely be more difficult and it seems that light clients will have to download lots of state data progressively to follow the fork-choice rule correctly. It would probably be simpler to give light clients a batch of proofs for all finalized dynasty changes (i.e. a warp sync) than to have it try and follow the header chain from genesis.

[rphmeier]

As far as implementing the fork-choice rule, I think this will be fairly easy to do (will provide the skeleton for that in an upcoming PR) through the introduction of arbitrary engine metadata being allowed to be stored in the blockchain-db.

trait Metadata {
    // set and get engine-specific metadata
    fn set_key(&mut self, key: &[u8], val: &[u8]);
    fn get(&self, key: &[u8]) -> Option<Vec<u8>>;

    // set a reversion limit: the block indicated by this hash will never be reverted
    fn set_reversion_limit(&mut self, block_hash: H256);

    // epoch transition stuff can be merged into here also.
}

The consensus engine gets two new functions:

• fn update_metadata(header: &Header, ancestry: &Ancestry, metadata: &mut Metadata)
• fn fork_choice(prospective: &Header, current_best: &Header, metadata: &Metadata)

The fork_choice function always called after the update_metadata function, and never if the prospective block would reorganize past a set reversion limit.

Here are some example fork choice implementations:

    // Ethash
    fn total_diff_fork_choice(prospective, current_best, metadata_store) -> H256 {
	let new_total_diff = metadata_store.get_total_difficulty(prospective);
	let current_best_diff = metadata_store.get_total_difficulty(current_best);

	if current_best_diff > new_total_diff {
		current_best.hash()
	} else {
		prospective.hash()
	}
    }

    // a casperFFG fork choice
    fn casperFFG_fork_choice(prospective, current_best, metadata_store) -> H256 { 
	let left_justified_height = get_justified_height(prospective);
	let right_justified_height = get_justified_height(current_best);

	// Note that there exists at most one justified checkpoint at a height `n`.
	// the last justified checkpoint must have been cast as such according to the best block
	match left_justified_height.cmp(right_justified_height) {
		Ordering::Equal => total_diff_fork_choice_rule(prospective, current_best, metadata_store)
		Ordering::Less => current_best.hash(),
		Ordering::Greater => prospective.hash(),
	}
    }

The respective update_metadata functions would set the new total difficulty and justified/finalized epochs based on chain or state data.

[rphmeier]

@vbuterin when you say "highest justified epoch #", should that be the highest witnessed at any contract-state or only at the contract state of either chain head in the fork choice?

[vbuterin]

The "justified epoch #" of a block is the last-justified-epoch in the contract state in the post-state of that block. So you would look for the block which returns the highest LJE if you try to call the getter for that variable using the post-state of the block.

…

[5chdn]

Copy-pasty from Gitter:

Danny Ryan @djrtwo Apr 01 06:14
Hey client implementers. We’re working on some casper contract updates and rebooting the casper testnet soon. We’d like to get more (all) clients involved this time around. We’ve updated the implementation guide and an EIP is coming soon. We are working on porting the validator service to web3.py to be able to plug into different backends, so the client/forkchoice/gas changes are the top priority. DM me and let me know where you’re at in terms of implementation, etc, and I’ll keep you updated in the coming weeks on testnet next steps.

Note: if you have already begun implementation, there have been a couple of tweaks to the gas/vote-transaction rules recently. I can help you identify the changes. Just let me know.

Danny Ryan @djrtwo Apr 02 06:35
Last site got DoS’d because someone posted the link to reddit. Moved it to the casper github repo
https://github.com/ethereum/casper/blob/master/IMPLEMENTATION.md

[5chdn]

https://github.com/ethereum/EIPs/pull/987/files

[sorpaas]

I would like to help out on this issue.

[5chdn]

ethereum/EIPs#1011

[sorpaas]

Discussed with @ascjones. I'll take over the Casper ABI work and try to see whether we can use the fork choice and metadata framework to make an implementation to connect to the Casper testnet.

[djrtwo]

Just fyi, vyper is soon moving to fixedMxN rather than decimal. Casper contract will move to a subtype of this when vyper releases the update.

Relevant issue here vyperlang/vyper#566