[WIP] Introduce consensus code for Whisk (SSLE) with Curdleproofs #3205

nalinbhardwaj · 2023-01-12T14:45:26Z

Ethresearch post: https://ethresear.ch/t/whisk-a-practical-shuffle-based-ssle-protocol-for-ethereum/11763

Rebased version of #2800

@asn-d6

@asn-d6: As discussed, I fixed a few bugs along the way but likely also introduced some bugs :)

This PR changes the Whisk tracker format to be of the form `(r * pubkey, r * BLS_GT_GENERATOR)` instead of `(r * k * BLS_G1_GENERATOR, r * BLS_G1_GENERATOR)`. This allows for non-interactive tracker registrations from validator pubkeys, removing ~50 lines the code. It also significantly reduces the amount of state overhead. This PR also removes permutation commitments, though those can be easily readded if deemed necessary.

@asn-d6

@asn-d6: Readded a consistency check for `IsValidWhiskOpeningProof` (involving `pubkey` instead of `k_commitment`).

This needs its own ethresearch post, and some additional analysis to see if we can do the shuffle ZKP in the allowed timeframe. This reverts commit 8517aca.

Ditch the Feistel logic and instead have each shuffler pick the row they shuffle using their RANDAO reveal.

This commit further simplifies 0faef30 by removing the entire squareshuffle. The latest version of https://eprint.iacr.org/2022/560 proposes that each shuffler picks random indices from the entire candidate set instead of organizing validators into a square.

dapplion · 2023-03-05T00:48:17Z

tests/core/pyspec/eth2spec/merge/mainnet.py

+
+
+assert FINALIZED_ROOT_INDEX == get_generalized_index(BeaconState, 'finalized_checkpoint', 'root')
+assert NEXT_SYNC_COMMITTEE_INDEX == get_generalized_index(BeaconState, 'next_sync_committee')


I don't think you want to commit this files. @hwwhww aren't this gitignored or what may have happened? Same for tests/core/pyspec/eth2spec/merge/minimal.py

Right, it should be ignored. The spec name stub "merge" was very outdated. I guess @nalinbhardwaj didn't clean the old stub folder before pulling from the dev branch.

dapplion · 2023-03-05T00:48:56Z

tests/core/pyspec/eth2spec/test/context.py

 from eth2spec.utils import bls

 from .exceptions import SkippedTest
 from .helpers.constants import (
-    PHASE0, ALTAIR, BELLATRIX, CAPELLA, EIP4844,
+    PHASE0, ALTAIR, BELLATRIX, CAPELLA, EIP4844, WHISK, SHARDING,


Suggested change

PHASE0, ALTAIR, BELLATRIX, CAPELLA, EIP4844, WHISK, SHARDING,

PHASE0, ALTAIR, BELLATRIX, CAPELLA, EIP4844, WHISK,

dapplion · 2023-03-05T00:49:29Z

tests/core/pyspec/eth2spec/test/helpers/constants.py


 # The forks that pytest can run with.
 ALL_PHASES = (
    # Formal forks
    PHASE0, ALTAIR, BELLATRIX, CAPELLA,
    # Experimental patches
-    EIP4844,
+    EIP4844, CUSTODY_GAME, WHISK,


Suggested change

EIP4844, CUSTODY_GAME, WHISK,

EIP4844, WHISK,

dapplion · 2023-03-05T00:52:06Z

configs/mainnet.yaml

@@ -81,7 +77,6 @@ MIN_PER_EPOCH_CHURN_LIMIT: 4
 # 2**16 (= 65,536)
 CHURN_LIMIT_QUOTIENT: 65536

-


Unnecessary polluting diff styling configs/mainnet.yaml and configs/minimal.yaml, should be reverted for a cleaner PR.

dapplion · 2023-03-19T13:39:10Z

Some notes regarding implementation:

Instead of extending the validator class, add two new lists in the state. That reduces the additional hashing load from +4 hashes / val record to +2 hashes / val record. It also prevents adding 1 extra depth to the validator type and break light-client uses. It also simplifies implementation, as clients are used to different states but the Validator type has remained constant since phase0 and is heavily used.
Add whisk_opening_proof in the block body, not in the BeaconBlock type. I see no reason to not add the property in the body, and similarly to comment above, adding new properties outside the body adds implementation complexity for no clear gain.
Use strict snake case, Lodestar does not expect capital letters after _ otherwise it would break tooling.

See suggested diff here nalinbhardwaj/consensus-specs@whisk-curdleproofs...dapplion:consensus-specs:whisk-curdleproofs-review

If you want @nalinbhardwaj I can help rebase this PR into current structure, as now features are in a different directory than when this PR was created.

asn-d6 · 2023-03-24T07:53:29Z

Some notes regarding implementation:

* Instead of extending the validator class, add two new lists in the state. That reduces the additional hashing load from +4 hashes / val record to +2 hashes / val record. It also prevents adding 1 extra depth to the validator type and break light-client uses. It also simplifies implementation, as clients are used to different states but the Validator type has remained constant since phase0 and is heavily used.

* Add `whisk_opening_proof` in the block body, not in the BeaconBlock type. I see no reason to not add the property in the body, and similarly to comment above, adding new properties outside the body adds implementation complexity for no clear gain.

* Use strict snake case, Lodestar does not expect capital letters after `_` otherwise it would break tooling.

See suggested diff here nalinbhardwaj/consensus-specs@whisk-curdleproofs...dapplion:consensus-specs:whisk-curdleproofs-review

If you want @nalinbhardwaj I can help rebase this PR into current structure, as now features are in a different directory than when this PR was created.

Hey @dapplion . Thanks for reviewing the code. That was invaluable.

I think all your suggestions are great with regards to better utilizing the state and the beacon blocks.

I don't have a strong opinion about the naming, but I kinda liked having capital letters to signify the group elements (instead of field elements). Instead of changing the spec, isn't this something that can be changed on the lodestar side?

With regards to incorporating your diff: Would you like to rebase this PR, and then do a PR on top of it, so that we are also able to review your changes? I think most of your changes are great, but perhaps I'd like to comment on a line or two.

dapplion · 2023-03-27T12:08:14Z

@asn-d6 Opened PR against this branch, so it's in @nalinbhardwaj repo

Whisk review implementor notes nalinbhardwaj/consensus-specs#1

Feel free to rebase first if you want and I can rebase my diff too

hwwhww · 2023-04-21T07:17:35Z

Hey, @nalinbhardwaj @asn-d6

Could you move the feature specs to specs/_features/whisk?

I suppose it needs more proper reviews before merging, but I can at least add a CI job to filter out the basic bugs. :)

hwwhww · 2023-06-08T13:34:09Z

Closing via #3342

Thank you @nalinbhardwaj @asn-d6 @JustinDrake @dapplion!

asn-d6 and others added 21 commits January 12, 2023 06:09

Introduce consensus code for Whisk

389f260

polish, simplify, clean up (~100 fewer lines)

7aa41a8

@asn-d6: As discussed, I fixed a few bugs along the way but likely also introduced some bugs :)

minor cleanups and fixes

f12140a

simplify is_k_commitment_unique

32e2462

Update beacon-chain.md

c89685b

Update beacon-chain.md

a32b66e

Initialize k in get_validator_from_deposit()

9043503

minor cleanups

e4c0201

Update beacon-chain.md

5ad92b1

A couple of fixes to the no-registration simplification

801757c

@asn-d6: Readded a consistency check for `IsValidWhiskOpeningProof` (involving `pubkey` instead of `k_commitment`).

remove unused helpers

fa67a63

use Mary's suggested tracker

2a00a36

Update beacon-chain.md

bfd2976

Revert G_t element optimization

48f11f5

This needs its own ethresearch post, and some additional analysis to see if we can do the shuffle ZKP in the allowed timeframe. This reverts commit 8517aca.

Implement new shuffling strategy

0faef30

Ditch the Feistel logic and instead have each shuffler pick the row they shuffle using their RANDAO reveal.

Curdleproofs edits

6bb0b33

working whisk eth2spec

5cce9dd

working whisk dummy test

5e7aeca

add more boilerplate set up code

cab7b90

rebase constants

3c41e9f

asn-d6 mentioned this pull request Jan 17, 2023

[WIP] Introduce consensus code for Whisk (SSLE) #2800

Closed

dapplion reviewed Mar 5, 2023

View reviewed changes

dapplion mentioned this pull request Mar 5, 2023

Add formater in CI #3281

Open

This was referenced Mar 25, 2023

Add whisk dev-friendly API asn-d6/curdleproofs#5

Merged

Implement whisk PoC dapplion/lighthouse#1

Draft

dapplion mentioned this pull request Mar 27, 2023

Whisk review implementor notes nalinbhardwaj/consensus-specs#1

Open

dapplion mentioned this pull request Apr 30, 2023

Whisk (SSLE) with Curdleproofs - rebased #3342

Merged

hwwhww closed this Jun 8, 2023

mratsim mentioned this pull request Nov 27, 2024

Torus-acceleration for multiexponentiation on GT mratsim/constantine#485

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] Introduce consensus code for Whisk (SSLE) with Curdleproofs #3205

[WIP] Introduce consensus code for Whisk (SSLE) with Curdleproofs #3205

nalinbhardwaj commented Jan 12, 2023

dapplion Mar 5, 2023 •

edited

Loading

hwwhww Mar 7, 2023

dapplion Mar 5, 2023

dapplion Mar 5, 2023

dapplion Mar 5, 2023

dapplion commented Mar 19, 2023 •

edited

Loading

asn-d6 commented Mar 24, 2023

dapplion commented Mar 27, 2023 •

edited

Loading

hwwhww commented Apr 21, 2023

hwwhww commented Jun 8, 2023



		assert FINALIZED_ROOT_INDEX == get_generalized_index(BeaconState, 'finalized_checkpoint', 'root')
		assert NEXT_SYNC_COMMITTEE_INDEX == get_generalized_index(BeaconState, 'next_sync_committee')

	PHASE0, ALTAIR, BELLATRIX, CAPELLA, EIP4844, WHISK, SHARDING,
	PHASE0, ALTAIR, BELLATRIX, CAPELLA, EIP4844, WHISK,

		@@ -81,7 +77,6 @@ MIN_PER_EPOCH_CHURN_LIMIT: 4
		# 2**16 (= 65,536)
		CHURN_LIMIT_QUOTIENT: 65536

[WIP] Introduce consensus code for Whisk (SSLE) with Curdleproofs #3205

[WIP] Introduce consensus code for Whisk (SSLE) with Curdleproofs #3205

Conversation

nalinbhardwaj commented Jan 12, 2023

dapplion Mar 5, 2023 • edited Loading

Choose a reason for hiding this comment

hwwhww Mar 7, 2023

Choose a reason for hiding this comment

dapplion Mar 5, 2023

Choose a reason for hiding this comment

dapplion Mar 5, 2023

Choose a reason for hiding this comment

dapplion Mar 5, 2023

Choose a reason for hiding this comment

dapplion commented Mar 19, 2023 • edited Loading

asn-d6 commented Mar 24, 2023

dapplion commented Mar 27, 2023 • edited Loading

hwwhww commented Apr 21, 2023

hwwhww commented Jun 8, 2023

dapplion Mar 5, 2023 •

edited

Loading

dapplion commented Mar 19, 2023 •

edited

Loading

dapplion commented Mar 27, 2023 •

edited

Loading