Merge pull request #2915 from asn-d6/consensus-4844-proofs-optimization

Optimizing EIP-4844 block validation (using KZG proofs)
ethereum · Jun 26, 2022 · d4a2bdc · d4a2bdc
2 parents 245ac6c + a3339c7
commit d4a2bdc
Show file tree

Hide file tree

Showing 4 changed files with 209 additions and 45 deletions.
diff --git a/specs/eip4844/beacon-chain.md b/specs/eip4844/beacon-chain.md
@@ -12,17 +12,13 @@
 - [Custom types](#custom-types)
 - [Constants](#constants)
   - [Domain types](#domain-types)
-- [Preset](#preset)
-  - [Trusted setup](#trusted-setup)
 - [Configuration](#configuration)
 - [Containers](#containers)
   - [Extended containers](#extended-containers)
     - [`BeaconBlockBody`](#beaconblockbody)
 - [Helper functions](#helper-functions)
-  - [KZG core](#kzg-core)
-    - [`blob_to_kzg`](#blob_to_kzg)
-    - [`kzg_to_versioned_hash`](#kzg_to_versioned_hash)
   - [Misc](#misc)
+    - [`kzg_to_versioned_hash`](#kzg_to_versioned_hash)
     - [`tx_peek_blob_versioned_hashes`](#tx_peek_blob_versioned_hashes)
     - [`verify_kzgs_against_transactions`](#verify_kzgs_against_transactions)
 - [Beacon chain state transition function](#beacon-chain-state-transition-function)
@@ -41,37 +37,22 @@ This upgrade adds blobs to the beacon chain as part of EIP-4844.
 
 | Name | SSZ equivalent | Description |
 | - | - | - |
-| `BLSFieldElement` | `uint256` | `x < BLS_MODULUS` |
 | `Blob` | `Vector[BLSFieldElement, FIELD_ELEMENTS_PER_BLOB]` | |
 | `VersionedHash` | `Bytes32` | |
-| `KZGCommitment` | `Bytes48` | Same as BLS standard "is valid pubkey" check but also allows `0x00..00` for point-at-infinity |
 
 ## Constants
 
 | Name | Value |
 | - | - |
 | `BLOB_TX_TYPE` | `uint8(0x05)` |
 | `FIELD_ELEMENTS_PER_BLOB` | `4096` |
-| `BLS_MODULUS` | `52435875175126190479447740508185965837690552500527637822603658699938581184513` |
 
 ### Domain types
 
 | Name | Value |
 | - | - |
 | `DOMAIN_BLOBS_SIDECAR` | `DomainType('0x0a000000')` |
 
-## Preset
-
-### Trusted setup
-
-The trusted setup is part of the preset: during testing a `minimal` insecure variant may be used,
-but reusing the `mainnet` settings in public networks is a critical security requirement.
-
-| Name | Value |
-| - | - |
-| `KZG_SETUP_G2` | `Vector[G2Point, FIELD_ELEMENTS_PER_BLOB]`, contents TBD |
-| `KZG_SETUP_LAGRANGE` | `Vector[KZGCommitment, FIELD_ELEMENTS_PER_BLOB]`, contents TBD |
-
 ## Configuration
 
 
@@ -102,23 +83,7 @@ class BeaconBlockBody(Container):
 
 ## Helper functions
 
-### KZG core
-
-KZG core functions. These are also defined in EIP-4844 execution specs.
-
-#### `blob_to_kzg`
-
-```python
-def blob_to_kzg(blob: Blob) -> KZGCommitment:
-    computed_kzg = bls.Z1
-    for value, point_kzg in zip(blob, KZG_SETUP_LAGRANGE):
-        assert value < BLS_MODULUS
-        computed_kzg = bls.add(
-            computed_kzg,
-            bls.multiply(point_kzg, value)
-        )
-    return computed_kzg
-```
+### Misc
 
 #### `kzg_to_versioned_hash`
 
@@ -127,8 +92,6 @@ def kzg_to_versioned_hash(kzg: KZGCommitment) -> VersionedHash:
     return BLOB_COMMITMENT_VERSION_KZG + hash(kzg)[1:]
 ```
 
-### Misc
-
 #### `tx_peek_blob_versioned_hashes`
 
 This function retrieves the hashes from the `SignedBlobTransaction` as defined in EIP-4844, using SSZ offsets.

diff --git a/specs/eip4844/p2p-interface.md b/specs/eip4844/p2p-interface.md
@@ -57,6 +57,7 @@ class BlobsSidecar(Container):
     beacon_block_root: Root
     beacon_block_slot: Slot
     blobs: List[Blob, MAX_BLOBS_PER_BLOCK]
+    kzg_aggregated_proof: KZGProof
 ```
 
 ### `SignedBlobsSidecar`
@@ -114,6 +115,7 @@ The following validations MUST pass before forwarding the `signed_blobs_sidecar`
 Alias `sidecar = signed_blobs_sidecar.message`.
 - _[IGNORE]_ the `sidecar.beacon_block_slot` is for the current slot (with a `MAXIMUM_GOSSIP_CLOCK_DISPARITY` allowance) -- i.e. `blobs_sidecar.beacon_block_slot == current_slot`.
 - _[REJECT]_ the `sidecar.blobs` are all well formatted, i.e. the `BLSFieldElement` in valid range (`x < BLS_MODULUS`).
+- _[REJECT]_ The KZG proof is a correctly encoded compressed BLS G1 Point -- i.e. `bls.KeyValidate(blobs_sidecar.kzg_aggregated_proof)
 - _[REJECT]_ the beacon proposer signature, `signed_blobs_sidecar.signature`, is valid -- i.e.
 ```python
 domain = get_domain(state, DOMAIN_BLOBS_SIDECAR, blobs_sidecar.beacon_block_slot // SLOTS_PER_EPOCH)

diff --git a/specs/eip4844/polynomial-commitments.md b/specs/eip4844/polynomial-commitments.md
@@ -0,0 +1,146 @@
+# EIP-4844 -- Polynomial Commitments
+
+## Table of contents
+
+<!-- TOC -->
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+
+- [Introduction](#introduction)
+- [Custom types](#custom-types)
+- [Constants](#constants)
+- [Preset](#preset)
+  - [Trusted setup](#trusted-setup)
+- [Helper functions](#helper-functions)
+  - [BLS12-381 helpers](#bls12-381-helpers)
+    - [`bls_modular_inverse`](#bls_modular_inverse)
+    - [`div`](#div)
+    - [`lincomb`](#lincomb)
+  - [KZG](#kzg)
+    - [`blob_to_kzg`](#blob_to_kzg)
+    - [`verify_kzg_proof`](#verify_kzg_proof)
+  - [Polynomials](#polynomials)
+    - [`evaluate_polynomial_in_evaluation_form`](#evaluate_polynomial_in_evaluation_form)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+<!-- /TOC -->
+
+
+## Introduction
+
+This document specifies basic polynomial operations and KZG polynomial commitment operations as they are needed for the EIP-4844 specification. The implementations are not optimized for performance, but readability. All practical implementations should optimize the polynomial operations.
+
+## Custom types
+
+| Name | SSZ equivalent | Description |
+| - | - | - |
+| `BLSFieldElement` | `uint256` | `x < BLS_MODULUS` |
+| `KZGCommitment` | `Bytes48` | Same as BLS standard "is valid pubkey" check but also allows `0x00..00` for point-at-infinity |
+| `KZGProof` | `Bytes48` | Same as for `KZGCommitment` |
+
+## Constants
+
+| Name | Value | Notes |
+| - | - | - |
+| `BLS_MODULUS` | `52435875175126190479447740508185965837690552500527637822603658699938581184513` | Scalar field modulus of BLS12-381 |
+| `ROOTS_OF_UNITY` | `Vector[BLSFieldElement, FIELD_ELEMENTS_PER_BLOB]` | Roots of unity of order FIELD_ELEMENTS_PER_BLOB over the BLS12-381 field |
+
+## Preset
+
+### Trusted setup
+
+The trusted setup is part of the preset: during testing a `minimal` insecure variant may be used,
+but reusing the `mainnet` settings in public networks is a critical security requirement.
+
+| Name | Value |
+| - | - |
+| `KZG_SETUP_G2` | `Vector[G2Point, FIELD_ELEMENTS_PER_BLOB]`, contents TBD |
+| `KZG_SETUP_LAGRANGE` | `Vector[KZGCommitment, FIELD_ELEMENTS_PER_BLOB]`, contents TBD |
+
+## Helper functions
+
+### BLS12-381 helpers
+
+#### `bls_modular_inverse`
+
+```python
+def bls_modular_inverse(x: BLSFieldElement) -> BLSFieldElement:
+    """
+    Compute the modular inverse of x
+    i.e. return y such that x * y % BLS_MODULUS == 1 and return 0 for x == 0
+    """
+    return pow(x, -1, BLS_MODULUS) if x != 0 else 0
+```
+
+#### `div`
+
+```python
+def div(x: BLSFieldElement, y: BLSFieldElement) -> BLSFieldElement:
+    """Divide two field elements: `x` by `y`"""
+    return x * bls_modular_inverse(y) % BLS_MODULUS
+```
+
+#### `lincomb`
+
+```python
+def lincomb(points: List[KZGCommitment], scalars: List[BLSFieldElement]) -> KZGCommitment:
+    """
+    BLS multiscalar multiplication. This function can be optimized using Pippenger's algorithm and variants.
+    """
+    r = bls.Z1
+    for x, a in zip(points, scalars):
+        r = bls.add(r, bls.multiply(x, a))
+    return r
+```
+
+### KZG
+
+KZG core functions. These are also defined in EIP-4844 execution specs.
+
+#### `blob_to_kzg`
+
+```python
+def blob_to_kzg(blob: Blob) -> KZGCommitment:
+    return lincomb(KZG_SETUP_LAGRANGE, blob)
+```
+
+#### `verify_kzg_proof`
+
+```python
+def verify_kzg_proof(polynomial_kzg: KZGCommitment,
+                     x: BLSFieldElement,
+                     y: BLSFieldElement,
+                     quotient_kzg: KZGProof) -> bool:
+    """
+    Verify KZG proof that ``p(x) == y`` where ``p(x)`` is the polynomial represented by ``polynomial_kzg``.
+    """
+    # Verify: P - y = Q * (X - x)
+    X_minus_x = bls.add(KZG_SETUP_G2[1], bls.multiply(bls.G2, BLS_MODULUS - x))
+    P_minus_y = bls.add(polynomial_kzg, bls.multiply(bls.G1, BLS_MODULUS - y))
+    return bls.pairing_check([
+        [P_minus_y, bls.neg(bls.G2)],
+        [quotient_kzg, X_minus_x]
+    ])
+```
+
+### Polynomials
+
+#### `evaluate_polynomial_in_evaluation_form`
+
+```python
+def evaluate_polynomial_in_evaluation_form(poly: List[BLSFieldElement], x: BLSFieldElement) -> BLSFieldElement:
+    """
+    Evaluate a polynomial (in evaluation form) at an arbitrary point `x`
+    Uses the barycentric formula:
+       f(x) = (1 - x**WIDTH) / WIDTH  *  sum_(i=0)^WIDTH  (f(DOMAIN[i]) * DOMAIN[i]) / (x - DOMAIN[i])
+    """
+    width = len(poly)
+    assert width == FIELD_ELEMENTS_PER_BLOB
+    inverse_width = bls_modular_inverse(width)
+
+    for i in range(width):
+        r += div(poly[i] * ROOTS_OF_UNITY[i], (x - ROOTS_OF_UNITY[i]))
+    r = r * (pow(x, width, BLS_MODULUS) - 1) * inverse_width % BLS_MODULUS
+
+    return r
+```
diff --git a/specs/eip4844/validator.md b/specs/eip4844/validator.md
@@ -12,6 +12,9 @@
 - [Prerequisites](#prerequisites)
 - [Helpers](#helpers)
   - [`is_data_available`](#is_data_available)
+  - [`hash_to_bls_field`](#hash_to_bls_field)
+  - [`compute_powers`](#compute_powers)
+  - [`vector_lincomb`](#vector_lincomb)
   - [`verify_blobs_sidecar`](#verify_blobs_sidecar)
 - [Beacon chain responsibilities](#beacon-chain-responsibilities)
   - [Block proposal](#block-proposal)
@@ -40,7 +43,7 @@ Please see related Beacon Chain doc before continuing and use them as a referenc
 
 The implementation of `is_data_available` is meant to change with later sharding upgrades.
 Initially, it requires every verifying actor to retrieve the matching `BlobsSidecar`,
-and verify the sidecar with `verify_blobs`.
+and verify the sidecar with `verify_blobs_sidecar`.
 
 Without the sidecar the block may be processed further optimistically,
 but MUST NOT be considered valid until a valid `BlobsSidecar` has been downloaded.
@@ -51,19 +54,70 @@ def is_data_available(slot: Slot, beacon_block_root: Root, kzgs: Sequence[KZGCom
     verify_blobs_sidecar(slot, beacon_block_root, kzgs, sidecar)
 ```
 
+### `hash_to_bls_field`
+
+```python
+def hash_to_bls_field(x: Container) -> BLSFieldElement:
+    """
+    This function is used to generate Fiat-Shamir challenges. The output is not uniform over the BLS field.
+    """
+    return int.from_bytes(hash_tree_root(x), "little") % BLS_MODULUS
+```
+
+### `compute_powers`
+```python
+def compute_powers(x: BLSFieldElement, n: uint64) -> List[BLSFieldElement]:
+    current_power = 1
+    powers = []
+    for _ in range(n):
+        powers.append(BLSFieldElement(current_power))
+        current_power = current_power * int(x) % BLS_MODULUS
+    return powers
+```
+
+### `vector_lincomb`
+
+```python
+def vector_lincomb(vectors: List[List[BLSFieldElement]], scalars: List[BLSFieldElement]) -> List[BLSFieldElement]:
+    """
+    Given a list of vectors, compute the linear combination of each column with `scalars`, and return the resulting
+    vector.
+    """
+    r = [0]*len(vectors[0])
+    for v, a in zip(vectors, scalars):
+        for i, x in enumerate(v):
+            r[i] = (r[i] + a * x) % BLS_MODULUS
+    return [BLSFieldElement(x) for x in r]
+```
+
 ### `verify_blobs_sidecar`
 
 ```python
 def verify_blobs_sidecar(slot: Slot, beacon_block_root: Root,
-                         expected_kzgs: Sequence[KZGCommitment], blobs_sidecar: BlobsSidecar):
+                         expected_kzgs: Sequence[KZGCommitment], blobs_sidecar: BlobsSidecar) -> None:
     assert slot == blobs_sidecar.beacon_block_slot
     assert beacon_block_root == blobs_sidecar.beacon_block_root
     blobs = blobs_sidecar.blobs
+    kzg_aggregated_proof = blobs_sidecar.kzg_aggregated_proof
     assert len(expected_kzgs) == len(blobs)
-    for kzg, blob in zip(expected_kzgs, blobs):
-        assert blob_to_kzg(blob) == kzg
-```
 
+    # Generate random linear combination challenges
+    r = hash_to_bls_field([blobs, expected_kzgs])
+    r_powers = compute_powers(r, len(expected_kzgs))
+
+    # Compute commitment to aggregated polynomial
+    aggregated_poly_commitment = lincomb(expected_kzgs, r_powers)
+
+    # Create aggregated polynomial in evaluation form
+    aggregated_poly = vector_lincomb(blobs, r_powers)
+
+    # Generate challenge `x` and evaluate the aggregated polynomial at `x`
+    x = hash_to_bls_field([aggregated_poly, aggregated_poly_commitment])
+    y = evaluate_polynomial_in_evaluation_form(aggregated_poly, x)
+
+    # Verify aggregated proof
+    assert verify_kzg_proof(aggregated_poly_commitment, x, y, kzg_aggregated_proof)
+```
 
 ## Beacon chain responsibilities
 
@@ -130,4 +184,3 @@ The validator MUST hold on to blobs for `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS`
 to ensure the data-availability of these blobs throughout the network.
 
 After `MIN_EPOCHS_FOR_BLOBS_SIDECARS_REQUESTS` nodes MAY prune the blobs and/or stop serving them.
-