Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement engine_getBlobsV1 #7553

Merged
merged 16 commits into from
Sep 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
### Additions and Improvements
- Update Java and Gradle dependecies [#7571](https://github.com/hyperledger/besu/pull/7571)
- Layered txpool: new options `--tx-pool-min-score` to remove a tx from pool when its score is lower than the specified value [#7576](https://github.com/hyperledger/besu/pull/7576)
- Add `engine_getBlobsV1` method to the Engine API [#7553](https://github.com/hyperledger/besu/pull/7553)

### Bug fixes
- Layered txpool: do not send notifications when moving tx between layers [#7539](https://github.com/hyperledger/besu/pull/7539)
Expand Down Expand Up @@ -39,7 +40,7 @@
- Correctly drops messages that exceeds local message size limit [#5455](https://github.com/hyperledger/besu/pull/7507)
- **DebugMetrics**: Fixed a `ClassCastException` occurring in `DebugMetrics` when handling nested metric structures. Previously, `Double` values within these structures were incorrectly cast to `Map` objects, leading to errors. This update allows for proper handling of both direct values and nested structures at the same level. Issue# [#7383](https://github.com/hyperledger/besu/pull/7383)
- `evmtool` was not respecting the `--genesis` setting, resulting in unexpected trace results. [#7433](https://github.com/hyperledger/besu/pull/7433)
- The genesis config override `contractSizeLimit` was not wired into code size limits [#7557](https://github.com/hyperledger/besu/pull/7557)
- The genesis config override `contractSizeLimit`q was not wired into code size limits [#7557](https://github.com/hyperledger/besu/pull/7557)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo

- Fix incorrect key filtering in LayeredKeyValueStorage stream [#7535](https://github.com/hyperledger/besu/pull/7557)

## 24.8.0
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@
import org.hyperledger.besu.ethereum.core.Util;
import org.hyperledger.besu.ethereum.eth.manager.EthContext;
import org.hyperledger.besu.ethereum.eth.manager.EthScheduler;
import org.hyperledger.besu.ethereum.eth.transactions.BlobCache;
import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionBroadcaster;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool;
Expand Down Expand Up @@ -245,7 +246,8 @@ private TransactionPool createTransactionPool() {
mock(TransactionBroadcaster.class),
ethContext,
new TransactionPoolMetrics(metricsSystem),
conf);
conf,
new BlobCache());
transactionPool.setEnabled();
return transactionPool;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@
import org.hyperledger.besu.ethereum.core.Util;
import org.hyperledger.besu.ethereum.eth.manager.EthContext;
import org.hyperledger.besu.ethereum.eth.manager.EthScheduler;
import org.hyperledger.besu.ethereum.eth.transactions.BlobCache;
import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionBroadcaster;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool;
Expand Down Expand Up @@ -233,7 +234,8 @@ private TransactionPool createTransactionPool() {
mock(TransactionBroadcaster.class),
cliqueEthContext,
new TransactionPoolMetrics(metricsSystem),
conf);
conf,
new BlobCache());

transactionPool.setEnabled();
return transactionPool;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@
import org.hyperledger.besu.ethereum.core.Util;
import org.hyperledger.besu.ethereum.eth.manager.EthContext;
import org.hyperledger.besu.ethereum.eth.manager.EthScheduler;
import org.hyperledger.besu.ethereum.eth.transactions.BlobCache;
import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionBroadcaster;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool;
Expand Down Expand Up @@ -371,7 +372,8 @@ private static ControllerAndState createControllerAndFinalState(
mock(TransactionBroadcaster.class),
ethContext,
new TransactionPoolMetrics(metricsSystem),
poolConf);
poolConf,
new BlobCache());

transactionPool.setEnabled();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
import org.hyperledger.besu.ethereum.core.MiningParameters;
import org.hyperledger.besu.ethereum.core.PrivacyParameters;
import org.hyperledger.besu.ethereum.eth.manager.EthContext;
import org.hyperledger.besu.ethereum.eth.transactions.BlobCache;
import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionBroadcaster;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool;
Expand Down Expand Up @@ -152,7 +153,8 @@ public BlockHeaderValidator.Builder createBlockHeaderRuleset(
mock(TransactionBroadcaster.class),
ethContext,
new TransactionPoolMetrics(metricsSystem),
poolConf);
poolConf,
new BlobCache());

transactionPool.setEnabled();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@
import org.hyperledger.besu.ethereum.eth.manager.EthContext;
import org.hyperledger.besu.ethereum.eth.manager.EthScheduler;
import org.hyperledger.besu.ethereum.eth.sync.backwardsync.BackwardSyncContext;
import org.hyperledger.besu.ethereum.eth.transactions.BlobCache;
import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration;
import org.hyperledger.besu.ethereum.eth.transactions.PendingTransaction;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionBroadcaster;
Expand Down Expand Up @@ -214,7 +215,8 @@ public void setUp() {
mock(TransactionBroadcaster.class),
ethContext,
new TransactionPoolMetrics(metricsSystem),
poolConf);
poolConf,
new BlobCache());

this.transactionPool.setEnabled();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@
import org.hyperledger.besu.ethereum.core.Util;
import org.hyperledger.besu.ethereum.eth.manager.EthContext;
import org.hyperledger.besu.ethereum.eth.manager.EthScheduler;
import org.hyperledger.besu.ethereum.eth.transactions.BlobCache;
import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionBroadcaster;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool;
Expand Down Expand Up @@ -480,7 +481,8 @@ private static ControllerAndState createControllerAndFinalState(
mock(TransactionBroadcaster.class),
ethContext,
new TransactionPoolMetrics(metricsSystem),
poolConf);
poolConf,
new BlobCache());

transactionPool.setEnabled();

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
import org.hyperledger.besu.ethereum.core.TransactionReceipt;
import org.hyperledger.besu.ethereum.eth.manager.EthContext;
import org.hyperledger.besu.ethereum.eth.manager.EthPeers;
import org.hyperledger.besu.ethereum.eth.transactions.BlobCache;
import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration;
import org.hyperledger.besu.ethereum.eth.transactions.PendingTransaction;
import org.hyperledger.besu.ethereum.eth.transactions.PendingTransactions;
Expand Down Expand Up @@ -121,7 +122,8 @@ public void setUp() {
batchAddedListener,
ethContext,
new TransactionPoolMetrics(metricsSystem),
TransactionPoolConfiguration.DEFAULT);
TransactionPoolConfiguration.DEFAULT,
new BlobCache());
transactionPool.setEnabled();
final BlockchainQueries blockchainQueries =
new BlockchainQueries(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
import org.hyperledger.besu.ethereum.core.TransactionReceipt;
import org.hyperledger.besu.ethereum.eth.manager.EthContext;
import org.hyperledger.besu.ethereum.eth.manager.EthPeers;
import org.hyperledger.besu.ethereum.eth.transactions.BlobCache;
import org.hyperledger.besu.ethereum.eth.transactions.ImmutableTransactionPoolConfiguration;
import org.hyperledger.besu.ethereum.eth.transactions.PendingTransaction;
import org.hyperledger.besu.ethereum.eth.transactions.PendingTransactions;
Expand Down Expand Up @@ -121,7 +122,8 @@ public void setUp() {
batchAddedListener,
ethContext,
new TransactionPoolMetrics(metricsSystem),
TransactionPoolConfiguration.DEFAULT);
TransactionPoolConfiguration.DEFAULT,
new BlobCache());
transactionPool.setEnabled();
final BlockchainQueries blockchainQueries =
new BlockchainQueries(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ public enum RpcMethod {
DEBUG_GET_RAW_BLOCK("debug_getRawBlock"),
DEBUG_GET_RAW_RECEIPTS("debug_getRawReceipts"),
DEBUG_GET_RAW_TRANSACTION("debug_getRawTransaction"),
ENGINE_GET_BLOBS_V1("engine_getBlobsV1"),
ENGINE_GET_PAYLOAD_V1("engine_getPayloadV1"),
ENGINE_GET_PAYLOAD_V2("engine_getPayloadV2"),
ENGINE_GET_PAYLOAD_V3("engine_getPayloadV3"),
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
/*
* Copyright contributors to Hyperledger Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine;

import org.hyperledger.besu.datatypes.BlobsWithCommitments;
import org.hyperledger.besu.datatypes.VersionedHash;
import org.hyperledger.besu.ethereum.ProtocolContext;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.JsonRpcRequestContext;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.exception.InvalidJsonRpcParameters;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.ExecutionEngineJsonRpcMethod;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.parameters.JsonRpcParameter;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcErrorResponse;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcResponse;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcSuccessResponse;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.RpcErrorType;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.results.BlobAndProofV1;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool;

import java.util.Arrays;
import java.util.List;
import javax.annotation.Nonnull;
import javax.annotation.Nullable;

import io.vertx.core.Vertx;

/**
* #### Specification
*
* <p>1. Given an array of blob versioned hashes client software **MUST** respond with an array of
* `BlobAndProofV1` objects with matching versioned hashes, respecting the order of versioned hashes
* in the input array.
*
* <p>2. Client software **MUST** place responses in the order given in the request, using `null`
* for any missing blobs. For instance, if the request is `[A_versioned_hash, B_versioned_hash,
* C_versioned_hash]` and client software has data for blobs `A` and `C`, but doesn't have data for
* `B`, the response **MUST** be `[A, null, C]`.
*
* <p>3. Client software **MUST** support request sizes of at least 128 blob versioned hashes. The
* client **MUST** return `-38004: Too large request` error if the number of requested blobs is too
* large.
*
* <p>4. Client software **MAY** return an array of all `null` entries if syncing or otherwise
* unable to serve blob pool data.
*
* <p>5. Callers **MUST** consider that execution layer clients may prune old blobs from their pool,
* and will respond with `null` if a blob has been pruned.
*/
public class EngineGetBlobsV1 extends ExecutionEngineJsonRpcMethod {

private final TransactionPool transactionPool;

public EngineGetBlobsV1(
final Vertx vertx,
final ProtocolContext protocolContext,
final EngineCallListener engineCallListener,
final TransactionPool transactionPool) {
super(vertx, protocolContext, engineCallListener);
this.transactionPool = transactionPool;
}

@Override
public String getName() {
return "engine_getBlobsV1";
}

@Override
public JsonRpcResponse syncResponse(final JsonRpcRequestContext requestContext) {
final VersionedHash[] versionedHashes;
try {
versionedHashes = requestContext.getRequiredParameter(0, VersionedHash[].class);
} catch (JsonRpcParameter.JsonRpcParameterException e) {
throw new InvalidJsonRpcParameters(
"Invalid versioned hashes parameter (index 0)",
RpcErrorType.INVALID_VERSIONED_HASHES_PARAMS,
e);
}

if (versionedHashes.length > 128) {
return new JsonRpcErrorResponse(
requestContext.getRequest().getId(),
RpcErrorType.INVALID_ENGINE_GET_BLOBS_V1_TOO_LARGE_REQUEST);
}

final List<BlobAndProofV1> result = getBlobV1Result(versionedHashes);

return new JsonRpcSuccessResponse(requestContext.getRequest().getId(), result);
}

private @Nonnull List<BlobAndProofV1> getBlobV1Result(final VersionedHash[] versionedHashes) {
return Arrays.stream(versionedHashes)
.map(transactionPool::getBlobQuad)
.map(this::getBlobAndProofV1)
.toList();
}

private @Nullable BlobAndProofV1 getBlobAndProofV1(final BlobsWithCommitments.BlobQuad bq) {
if (bq == null) {
return null;
}
return new BlobAndProofV1(
bq.blob().getData().toHexString(), bq.kzgProof().getData().toHexString());
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ public enum RpcErrorType implements RpcMethodError {
INVALID_ENGINE_NEW_PAYLOAD_PARAMS(INVALID_PARAMS_ERROR_CODE, "Invalid engine payload parameter"),
INVALID_ENGINE_PREPARE_PAYLOAD_PARAMS(
INVALID_PARAMS_ERROR_CODE, "Invalid engine prepare payload parameter"),
INVALID_ENGINE_GET_BLOBS_V1_TOO_LARGE_REQUEST(-38004, "Too large request"),
INVALID_ENODE_PARAMS(INVALID_PARAMS_ERROR_CODE, "Invalid enode params"),
INVALID_EXCESS_BLOB_GAS_PARAMS(
INVALID_PARAMS_ERROR_CODE, "Invalid excess blob gas params (missing or invalid)"),
Expand Down Expand Up @@ -109,6 +110,7 @@ public enum RpcErrorType implements RpcMethodError {
INVALID_TRANSACTION_LIMIT_PARAMS(INVALID_PARAMS_ERROR_CODE, "Invalid transaction limit params"),
INVALID_TRANSACTION_TRACE_PARAMS(INVALID_PARAMS_ERROR_CODE, "Invalid transaction trace params"),
INVALID_VERSIONED_HASH_PARAMS(INVALID_PARAMS_ERROR_CODE, "Invalid versioned hash params"),
INVALID_VERSIONED_HASHES_PARAMS(INVALID_PARAMS_ERROR_CODE, "Invalid versioned hashes params"),
INVALID_VOTE_TYPE_PARAMS(INVALID_PARAMS_ERROR_CODE, "Invalid vote type params"),
INVALID_WITHDRAWALS_PARAMS(INVALID_PARAMS_ERROR_CODE, "Invalid withdrawals"),

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
/*
* Copyright contributors to Hyperledger Besu.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.ethereum.api.jsonrpc.internal.results;

import com.fasterxml.jackson.annotation.JsonPropertyOrder;

/**
* The result of the eth_getBlobAndProofV1 JSON-RPC method contains an array of BlobAndProofV1.
* BlobAndProofV1 contains the blob data and the kzg proof for the blob.
*/
@JsonPropertyOrder({"blob", "proof"})
public class BlobAndProofV1 {

private final String blob;

private final String proof;

public BlobAndProofV1(final String blob, final String proof) {
this.blob = blob;
this.proof = proof;
}

public String getProof() {
return proof;
}

public String getBlob() {
return blob;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineForkchoiceUpdatedV1;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineForkchoiceUpdatedV2;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineForkchoiceUpdatedV3;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineGetBlobsV1;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineGetClientVersionV1;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineGetPayloadBodiesByHashV1;
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.methods.engine.EngineGetPayloadBodiesByRangeV1;
Expand All @@ -39,6 +40,7 @@
import org.hyperledger.besu.ethereum.api.jsonrpc.internal.results.BlockResultFactory;
import org.hyperledger.besu.ethereum.blockcreation.MiningCoordinator;
import org.hyperledger.besu.ethereum.eth.manager.EthPeers;
import org.hyperledger.besu.ethereum.eth.transactions.TransactionPool;
import org.hyperledger.besu.ethereum.mainnet.ProtocolSchedule;

import java.util.ArrayList;
Expand All @@ -60,6 +62,7 @@ public class ExecutionEngineJsonRpcMethods extends ApiGroupJsonRpcMethods {
private final Vertx consensusEngineServer;
private final String clientVersion;
private final String commit;
private final TransactionPool transactionPool;

ExecutionEngineJsonRpcMethods(
final MiningCoordinator miningCoordinator,
Expand All @@ -68,7 +71,8 @@ public class ExecutionEngineJsonRpcMethods extends ApiGroupJsonRpcMethods {
final EthPeers ethPeers,
final Vertx consensusEngineServer,
final String clientVersion,
final String commit) {
final String commit,
final TransactionPool transactionPool) {
this.mergeCoordinator =
Optional.ofNullable(miningCoordinator)
.filter(mc -> mc.isCompatibleWithEngineApi())
Expand All @@ -79,6 +83,7 @@ public class ExecutionEngineJsonRpcMethods extends ApiGroupJsonRpcMethods {
this.consensusEngineServer = consensusEngineServer;
this.clientVersion = clientVersion;
this.commit = commit;
this.transactionPool = transactionPool;
}

@Override
Expand Down Expand Up @@ -156,7 +161,9 @@ protected Map<String, JsonRpcMethod> create() {
new EnginePreparePayloadDebug(
consensusEngineServer, protocolContext, engineQosTimer, mergeCoordinator.get()),
new EngineGetClientVersionV1(
consensusEngineServer, protocolContext, engineQosTimer, clientVersion, commit)));
consensusEngineServer, protocolContext, engineQosTimer, clientVersion, commit),
new EngineGetBlobsV1(
consensusEngineServer, protocolContext, engineQosTimer, transactionPool)));

if (protocolSchedule.anyMatch(p -> p.spec().getName().equalsIgnoreCase("cancun"))) {
executionEngineApisSupported.add(
Expand Down
Loading
Loading