Save light client updates

CHANGELOG.md

@@ -27,7 +27,7 @@ The format is based on Keep a Changelog, and this project adheres to Semantic Ve
 - Updated the default `scrape-interval` in `Client-stats` to 2 minutes to accommodate Beaconcha.in API rate limits.
 - Switch to compounding when consolidating with source==target.
 - Revert block db save when saving state fails.
-- Return false from HasBlock if the block is being synced. 
+- Return false from HasBlock if the block is being synced.
 - Cleanup forkchoice on failed insertions.
 - Use read only validator for core processing to avoid unnecessary copying.
 - Use ROBlock across block processing pipeline.
@@ -52,9 +52,9 @@ The format is based on Keep a Changelog, and this project adheres to Semantic Ve
 
 ### Security
 
-## [v5.1.2](https://github.com/prysmaticlabs/prysm/compare/v5.1.1...v5.1.2) - 2024-10-16 
+## [v5.1.2](https://github.com/prysmaticlabs/prysm/compare/v5.1.1...v5.1.2) - 2024-10-16
 
-This is a hotfix release with one change. 
+This is a hotfix release with one change.
 
 Prysm v5.1.1 contains an updated implementation of the beacon api streaming events endpoint. This
 new implementation contains a bug that can cause a panic in certain conditions. The issue is
@@ -66,20 +66,20 @@ prysm REST mode validator (a feature which requires the validator to be configur
 api intead of prysm's stock grpc endpoints) or accessory software that connects to the events api,
 like https://github.com/ethpandaops/ethereum-metrics-exporter
 
-### Fixed 
+### Fixed
 
 - Recover from panics when writing the event stream [#14545](https://github.com/prysmaticlabs/prysm/pull/14545)
 
 ## [v5.1.1](https://github.com/prysmaticlabs/prysm/compare/v5.1.0...v5.1.1) - 2024-10-15
 
-This release has a number of features and improvements. Most notably, the feature flag 
-`--enable-experimental-state` has been flipped to "opt out" via `--disable-experimental-state`. 
+This release has a number of features and improvements. Most notably, the feature flag
+`--enable-experimental-state` has been flipped to "opt out" via `--disable-experimental-state`.
 The experimental state management design has shown significant improvements in memory usage at
 runtime. Updates to libp2p's gossipsub have some bandwidith stability improvements with support for
-IDONTWANT control messages. 
+IDONTWANT control messages.
 
 The gRPC gateway has been deprecated from Prysm in this release. If you need JSON data, consider the
-standardized beacon-APIs. 
+standardized beacon-APIs.
 
 Updating to this release is recommended at your convenience.
 
@@ -121,7 +121,7 @@ Updating to this release is recommended at your convenience.
 - `grpc-gateway-corsdomain` is renamed to http-cors-domain. The old name can still be used as an alias.
 - `api-timeout` is changed from int flag to duration flag, default value updated.
 - Light client support: abstracted out the light client headers with different versions.
-- `ApplyToEveryValidator` has been changed to prevent misuse bugs, it takes a closure that takes a `ReadOnlyValidator` and returns a raw pointer to a `Validator`. 
+- `ApplyToEveryValidator` has been changed to prevent misuse bugs, it takes a closure that takes a `ReadOnlyValidator` and returns a raw pointer to a `Validator`.
 - Removed gorilla mux library and replaced it with net/http updates in go 1.22.
 - Clean up `ProposeBlock` for validator client to reduce cognitive scoring and enable further changes.
 - Updated k8s-io/client-go to v0.30.4 and k8s-io/apimachinery to v0.30.4
@@ -132,7 +132,7 @@ Updating to this release is recommended at your convenience.
 - Updated Sepolia bootnodes.
 - Make committee aware packing the default by deprecating `--enable-committee-aware-packing`.
 - Moved `ConvertKzgCommitmentToVersionedHash` to the `primitives` package.
-- Updated correlation penalty for EIP-7251. 
+- Updated correlation penalty for EIP-7251.
 
 ### Deprecated
 - `--disable-grpc-gateway` flag is deprecated due to grpc gateway removal.
@@ -604,34 +604,34 @@ AVX support (eg Celeron) after the Deneb fork. This is not an issue for mainnet.
 
 - Linter: Wastedassign linter enabled to improve code quality.
 - API Enhancements:
-    - Added payload return in Wei for /eth/v3/validator/blocks.
-    - Added Holesky Deneb Epoch for better epoch management.
+  - Added payload return in Wei for /eth/v3/validator/blocks.
+  - Added Holesky Deneb Epoch for better epoch management.
 - Testing Enhancements:
-    - Clear cache in tests of core helpers to ensure test reliability.
-    - Added Debug State Transition Method for improved debugging.
-    - Backfilling test: Enabled backfill in E2E tests for more comprehensive coverage.
+  - Clear cache in tests of core helpers to ensure test reliability.
+  - Added Debug State Transition Method for improved debugging.
+  - Backfilling test: Enabled backfill in E2E tests for more comprehensive coverage.
 - API Updates: Re-enabled jwt on keymanager API for enhanced security.
 - Logging Improvements: Enhanced block by root log for better traceability.
 - Validator Client Improvements:
-    - Added Spans to Core Validator Methods for enhanced monitoring.
-    - Improved readability in validator client code for better maintenance (various commits).
+  - Added Spans to Core Validator Methods for enhanced monitoring.
+  - Improved readability in validator client code for better maintenance (various commits).
 
 ### Changed
 
 - Optimizations and Refinements:
-    - Lowered resource usage in certain processes for efficiency.
-    - Moved blob rpc validation closer to peer read for optimized processing.
-    - Cleaned up validate beacon block code for clarity and efficiency.
-    - Updated Sepolia Deneb fork epoch for alignment with network changes.
-    - Changed blob latency metrics to milliseconds for more precise measurement.
-    - Altered getLegacyDatabaseLocation message for better clarity.
-    - Improved wait for activation method for enhanced performance.
-    - Capitalized Aggregated Unaggregated Attestations Log for consistency.
-    - Modified HistoricalRoots usage for accuracy.
-    - Adjusted checking of attribute emptiness for efficiency.
+  - Lowered resource usage in certain processes for efficiency.
+  - Moved blob rpc validation closer to peer read for optimized processing.
+  - Cleaned up validate beacon block code for clarity and efficiency.
+  - Updated Sepolia Deneb fork epoch for alignment with network changes.
+  - Changed blob latency metrics to milliseconds for more precise measurement.
+  - Altered getLegacyDatabaseLocation message for better clarity.
+  - Improved wait for activation method for enhanced performance.
+  - Capitalized Aggregated Unaggregated Attestations Log for consistency.
+  - Modified HistoricalRoots usage for accuracy.
+  - Adjusted checking of attribute emptiness for efficiency.
 - Database Management:
-    - Moved --db-backup-output-dir as a deprecated flag for database management simplification.
-    - Added the Ability to Defragment the Beacon State for improved database performance.
+  - Moved --db-backup-output-dir as a deprecated flag for database management simplification.
+  - Added the Ability to Defragment the Beacon State for improved database performance.
 - Dependency Update: Bumped quic-go version from 0.39.3 to 0.39.4 for up-to-date dependencies.
 
 ### Removed
@@ -642,12 +642,12 @@ AVX support (eg Celeron) after the Deneb fork. This is not an issue for mainnet.
 ### Fixed
 
 - Bug Fixes:
-    - Fixed off by one error for improved accuracy.
-    - Resolved small typo in error messages for clarity.
-    - Addressed minor issue in blsToExecChange validator for better validation.
-    - Corrected blobsidecar json tag for commitment inclusion proof.
-    - Fixed ssz post-requests content type check.
-    - Resolved issue with port logging in bootnode.
+  - Fixed off by one error for improved accuracy.
+  - Resolved small typo in error messages for clarity.
+  - Addressed minor issue in blsToExecChange validator for better validation.
+  - Corrected blobsidecar json tag for commitment inclusion proof.
+  - Fixed ssz post-requests content type check.
+  - Resolved issue with port logging in bootnode.
 - Test Fixes: Re-enabled Slasher E2E Test for more comprehensive testing.
 
 ### Security
@@ -1074,9 +1074,9 @@ No security issues in thsi release.
   now features runtime detection, automatically enabling optimized code paths if your CPU supports it.
 - **Multiarch Containers Preview Available**: multiarch (:wave: arm64 support :wave:) containers will be offered for
   preview at the following locations:
-    - Beacon Chain: [gcr.io/prylabs-dev/prysm/beacon-chain:v4.1.0](gcr.io/prylabs-dev/prysm/beacon-chain:v4.1.0)
-    - Validator: [gcr.io/prylabs-dev/prysm/validator:v4.1.0](gcr.io/prylabs-dev/prysm/validator:v4.1.0)
-    - Please note that in the next cycle, we will exclusively use these containers at the canonical URLs.
+  - Beacon Chain: [gcr.io/prylabs-dev/prysm/beacon-chain:v4.1.0](gcr.io/prylabs-dev/prysm/beacon-chain:v4.1.0)
+  - Validator: [gcr.io/prylabs-dev/prysm/validator:v4.1.0](gcr.io/prylabs-dev/prysm/validator:v4.1.0)
+  - Please note that in the next cycle, we will exclusively use these containers at the canonical URLs.
 
 ### Added
 

api/server/structs/BUILD.bazel

@@ -36,9 +36,8 @@ go_library(
         "//math:go_default_library",
         "//proto/engine/v1:go_default_library",
         "//proto/eth/v1:go_default_library",
-        "//proto/eth/v2:go_default_library",
-        "//proto/migration:go_default_library",
         "//proto/prysm/v1alpha1:go_default_library",
+        "//runtime/version:go_default_library",
         "@com_github_ethereum_go_ethereum//common:go_default_library",
         "@com_github_ethereum_go_ethereum//common/hexutil:go_default_library",
         "@com_github_pkg_errors//:go_default_library",

api/server/structs/conversions.go

@@ -1546,3 +1546,10 @@ func EventChainReorgFromV1(event *ethv1.EventChainReorg) *ChainReorgEvent {
 		ExecutionOptimistic: event.ExecutionOptimistic,
 	}
 }
+
+func SyncAggregateFromConsensus(sa *eth.SyncAggregate) *SyncAggregate {
+	return &SyncAggregate{
+		SyncCommitteeBits:      hexutil.Encode(sa.SyncCommitteeBits),
+		SyncCommitteeSignature: hexutil.Encode(sa.SyncCommitteeSignature),
+	}
+}

api/server/structs/conversions_lightclient.go

@@ -3,125 +3,153 @@ package structs
 import (
 	"encoding/json"
 	"fmt"
-	"strconv"
 
 	"github.com/ethereum/go-ethereum/common/hexutil"
 	"github.com/pkg/errors"
-	v1 "github.com/prysmaticlabs/prysm/v5/proto/eth/v1"
-	v2 "github.com/prysmaticlabs/prysm/v5/proto/eth/v2"
-	"github.com/prysmaticlabs/prysm/v5/proto/migration"
+	"github.com/prysmaticlabs/prysm/v5/consensus-types/interfaces"
+	enginev1 "github.com/prysmaticlabs/prysm/v5/proto/engine/v1"
+	"github.com/prysmaticlabs/prysm/v5/runtime/version"
 )
 
-func LightClientUpdateFromConsensus(update *v2.LightClientUpdate) (*LightClientUpdate, error) {
-	attestedHeader, err := lightClientHeaderContainerToJSON(update.AttestedHeader)
+func LightClientUpdateFromConsensus(update interfaces.LightClientUpdate) (*LightClientUpdate, error) {
+	attestedHeader, err := lightClientHeaderToJSON(update.AttestedHeader())
 	if err != nil {
 		return nil, errors.Wrap(err, "could not marshal attested light client header")
 	}
-	finalizedHeader, err := lightClientHeaderContainerToJSON(update.FinalizedHeader)
+	finalizedHeader, err := lightClientHeaderToJSON(update.FinalizedHeader())
 	if err != nil {
 		return nil, errors.Wrap(err, "could not marshal finalized light client header")
 	}
+	finalityBranch := update.FinalityBranch()
+
+	var scBranch [][32]byte
+	if update.Version() >= version.Electra {
+		b, err := update.NextSyncCommitteeBranchElectra()
+		if err != nil {
+			return nil, err
+		}
+		scBranch = b[:]
+	} else {
+		b, err := update.NextSyncCommitteeBranch()
+		if err != nil {
+			return nil, err
+		}
+		scBranch = b[:]
+	}
 
 	return &LightClientUpdate{
 		AttestedHeader:          attestedHeader,
-		NextSyncCommittee:       SyncCommitteeFromConsensus(migration.V2SyncCommitteeToV1Alpha1(update.NextSyncCommittee)),
-		NextSyncCommitteeBranch: branchToJSON(update.NextSyncCommitteeBranch),
+		NextSyncCommittee:       SyncCommitteeFromConsensus(update.NextSyncCommittee()),
+		NextSyncCommitteeBranch: branchToJSON(scBranch),
 		FinalizedHeader:         finalizedHeader,
-		FinalityBranch:          branchToJSON(update.FinalityBranch),
-		SyncAggregate:           syncAggregateToJSON(update.SyncAggregate),
-		SignatureSlot:           strconv.FormatUint(uint64(update.SignatureSlot), 10),
+		FinalityBranch:          branchToJSON(finalityBranch[:]),
+		SyncAggregate:           SyncAggregateFromConsensus(update.SyncAggregate()),
+		SignatureSlot:           fmt.Sprintf("%d", update.SignatureSlot()),
 	}, nil
 }
 
-func LightClientFinalityUpdateFromConsensus(update *v2.LightClientFinalityUpdate) (*LightClientFinalityUpdate, error) {
-	attestedHeader, err := lightClientHeaderContainerToJSON(update.AttestedHeader)
+func LightClientFinalityUpdateFromConsensus(update interfaces.LightClientFinalityUpdate) (*LightClientFinalityUpdate, error) {
+	attestedHeader, err := lightClientHeaderToJSON(update.AttestedHeader())
 	if err != nil {
 		return nil, errors.Wrap(err, "could not marshal attested light client header")
 	}
-	finalizedHeader, err := lightClientHeaderContainerToJSON(update.FinalizedHeader)
+	finalizedHeader, err := lightClientHeaderToJSON(update.FinalizedHeader())
 	if err != nil {
 		return nil, errors.Wrap(err, "could not marshal finalized light client header")
 	}
+	finalityBranch := update.FinalityBranch()
 
 	return &LightClientFinalityUpdate{
 		AttestedHeader:  attestedHeader,
 		FinalizedHeader: finalizedHeader,
-		FinalityBranch:  branchToJSON(update.FinalityBranch),
-		SyncAggregate:   syncAggregateToJSON(update.SyncAggregate),
-		SignatureSlot:   strconv.FormatUint(uint64(update.SignatureSlot), 10),
+		FinalityBranch:  branchToJSON(finalityBranch[:]),
+		SyncAggregate:   SyncAggregateFromConsensus(update.SyncAggregate()),
+		SignatureSlot:   fmt.Sprintf("%d", update.SignatureSlot()),
 	}, nil
 }
 
-func LightClientOptimisticUpdateFromConsensus(update *v2.LightClientOptimisticUpdate) (*LightClientOptimisticUpdate, error) {
-	attestedHeader, err := lightClientHeaderContainerToJSON(update.AttestedHeader)
+func LightClientOptimisticUpdateFromConsensus(update interfaces.LightClientOptimisticUpdate) (*LightClientOptimisticUpdate, error) {
+	attestedHeader, err := lightClientHeaderToJSON(update.AttestedHeader())
 	if err != nil {
 		return nil, errors.Wrap(err, "could not marshal attested light client header")
 	}
 
 	return &LightClientOptimisticUpdate{
 		AttestedHeader: attestedHeader,
-		SyncAggregate:  syncAggregateToJSON(update.SyncAggregate),
-		SignatureSlot:  strconv.FormatUint(uint64(update.SignatureSlot), 10),
+		SyncAggregate:  SyncAggregateFromConsensus(update.SyncAggregate()),
+		SignatureSlot:  fmt.Sprintf("%d", update.SignatureSlot()),
 	}, nil
 }
 
-func branchToJSON(branchBytes [][]byte) []string {
+func branchToJSON[S [][32]byte](branchBytes S) []string {
 	if branchBytes == nil {
 		return nil
 	}
 	branch := make([]string, len(branchBytes))
 	for i, root := range branchBytes {
-		branch[i] = hexutil.Encode(root)
+		branch[i] = hexutil.Encode(root[:])
 	}
 	return branch
 }
 
-func syncAggregateToJSON(input *v1.SyncAggregate) *SyncAggregate {
-	return &SyncAggregate{
-		SyncCommitteeBits:      hexutil.Encode(input.SyncCommitteeBits),
-		SyncCommitteeSignature: hexutil.Encode(input.SyncCommitteeSignature),
-	}
-}
-
-func lightClientHeaderContainerToJSON(container *v2.LightClientHeaderContainer) (json.RawMessage, error) {
+func lightClientHeaderToJSON(header interfaces.LightClientHeader) (json.RawMessage, error) {
 	// In the case that a finalizedHeader is nil.
-	if container == nil {
+	if header == nil {
 		return nil, nil
 	}
 
-	beacon, err := container.GetBeacon()
-	if err != nil {
-		return nil, errors.Wrap(err, "could not get beacon block header")
-	}
-
-	var header any
+	var result any
 
-	switch t := (container.Header).(type) {
-	case *v2.LightClientHeaderContainer_HeaderAltair:
-		header = &LightClientHeader{Beacon: BeaconBlockHeaderFromConsensus(migration.V1HeaderToV1Alpha1(beacon))}
-	case *v2.LightClientHeaderContainer_HeaderCapella:
-		execution, err := ExecutionPayloadHeaderCapellaFromConsensus(t.HeaderCapella.Execution)
+	switch v := header.Version(); v {
+	case version.Altair:
+		result = &LightClientHeader{Beacon: BeaconBlockHeaderFromConsensus(header.Beacon())}
+	case version.Capella:
+		exInterface, err := header.Execution()
 		if err != nil {
 			return nil, err
 		}
-		header = &LightClientHeaderCapella{
-			Beacon:          BeaconBlockHeaderFromConsensus(migration.V1HeaderToV1Alpha1(beacon)),
+		ex, ok := exInterface.Proto().(*enginev1.ExecutionPayloadHeaderCapella)
+		if !ok {
+			return nil, fmt.Errorf("execution data is not %T", &enginev1.ExecutionPayloadHeaderCapella{})
+		}
+		execution, err := ExecutionPayloadHeaderCapellaFromConsensus(ex)
+		if err != nil {
+			return nil, err
+		}
+		executionBranch, err := header.ExecutionBranch()
+		if err != nil {
+			return nil, err
+		}
+		result = &LightClientHeaderCapella{
+			Beacon:          BeaconBlockHeaderFromConsensus(header.Beacon()),
 			Execution:       execution,
-			ExecutionBranch: branchToJSON(t.HeaderCapella.ExecutionBranch),
+			ExecutionBranch: branchToJSON(executionBranch[:]),
+		}
+	case version.Deneb:
+		exInterface, err := header.Execution()
+		if err != nil {
+			return nil, err
+		}
+		ex, ok := exInterface.Proto().(*enginev1.ExecutionPayloadHeaderDeneb)
+		if !ok {
+			return nil, fmt.Errorf("execution data is not %T", &enginev1.ExecutionPayloadHeaderDeneb{})
+		}
+		execution, err := ExecutionPayloadHeaderDenebFromConsensus(ex)
+		if err != nil {
+			return nil, err
 		}
-	case *v2.LightClientHeaderContainer_HeaderDeneb:
-		execution, err := ExecutionPayloadHeaderDenebFromConsensus(t.HeaderDeneb.Execution)
+		executionBranch, err := header.ExecutionBranch()
 		if err != nil {
 			return nil, err
 		}
-		header = &LightClientHeaderDeneb{
-			Beacon:          BeaconBlockHeaderFromConsensus(migration.V1HeaderToV1Alpha1(beacon)),
+		result = &LightClientHeaderDeneb{
+			Beacon:          BeaconBlockHeaderFromConsensus(header.Beacon()),
 			Execution:       execution,
-			ExecutionBranch: branchToJSON(t.HeaderDeneb.ExecutionBranch),
+			ExecutionBranch: branchToJSON(executionBranch[:]),
 		}
 	default:
-		return nil, fmt.Errorf("unsupported header type %T", t)
+		return nil, fmt.Errorf("unsupported header version %s", version.String(v))
 	}
 
-	return json.Marshal(header)
+	return json.Marshal(result)
 }

beacon-chain/blockchain/BUILD.bazel

@@ -84,7 +84,6 @@ go_library(
         "//monitoring/tracing/trace:go_default_library",
         "//proto/engine/v1:go_default_library",
         "//proto/eth/v1:go_default_library",
-        "//proto/eth/v2:go_default_library",
         "//proto/prysm/v1alpha1:go_default_library",
         "//proto/prysm/v1alpha1/attestation:go_default_library",
         "//runtime/version:go_default_library",

beacon-chain/blockchain/process_block.go

@@ -70,6 +70,7 @@ func (s *Service) postBlockProcess(cfg *postBlockProcessConfig) error {
 		defer s.handleSecondFCUCall(cfg, fcuArgs)
 	}
 	defer s.sendLightClientFeeds(cfg)
+	defer s.saveLightClientUpdate(cfg)
 	defer s.sendStateFeedOnBlock(cfg)
 	defer reportProcessingTime(startTime)
 	defer reportAttestationInclusion(cfg.roblock.Block())