Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evmtool graalvm support #5192

Merged
merged 19 commits into from
Mar 21, 2023
Merged

Evmtool graalvm support #5192

merged 19 commits into from
Mar 21, 2023

Conversation

shemnon
Copy link
Contributor

@shemnon shemnon commented Mar 8, 2023

PR description

Changes and config to support using GraalVM AOT to compile and
execute evmTool. While not as long-term performant the startup time
makes filling reference tests with Besu reasonable.

Builds off of #5189

Fixed Issue(s)

Documentation

  • I thought about documentation and added the doc-change-required label to this PR if
    updates are required.

Acceptance Tests (Non Mainnet)

  • I have considered running ./gradlew acceptanceTestNonMainnet locally if my PR affects non-mainnet modules.

Changelog

Reduce the number of places that expose Log4J classes as a part of the
interfaces for methods and classes. While Log4j remains the default we
still need to be able to function when the Log4J jars are removed from
the classpath.

Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Final changes and config to support using GraalVM AOT to compile and
execute evmTool.  While not as long-term performant the startup time
makes filling reference tests with Besu reasonable.

Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
@@ -82,7 +76,7 @@ protected AbstractSECP256(final String curveName, final BigInteger prime) {
curveOrder = curve.getN();
halfCurveOrder = curveOrder.shiftRight(1);
try {
keyPairGenerator = KeyPairGenerator.getInstance(ALGORITHM, PROVIDER);
keyPairGenerator = new ECDSA();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. JCA and Bouncy Castle are very difficult to get working with GraalVM. There is a lot of introspection involved and a lot of pointless overhead to get to what this change does. We have it configured in the parameters (per ALGORITHM and PROVIDER) so that the referenced class is the only thing that can be returned. Were this a system where we change crypto configs on the fly or if ethereum accepted multiple key lengths or algos then it would be worth it. This is just overhead that makes AOT compilation impossible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm so sorry, I'm thoroughly confused because I assumed that it was possible to configure the curve used. I'll take a deeper look.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The curve kind (ECDSA vs Edwards) is the same across all current configurations. The curve parameters (secp256k1 or secp256r1) are applied to the key generator after this construction.

Also, this is not for the whole of the public key encryption used in the system, just the key pair generators. I updated the use of imports to clarify that.

// default is mainnet
return new MainnetGenesisFileModule(genesisConfig);
if (config.containsKey("ibft") || config.containsKey("clique") || config.containsKey("qbft")) {
throw new RuntimeException("Only Ethash and Merge configs accepted as genesis files");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this related to this PR?

Copy link
Contributor Author

@shemnon shemnon Mar 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This unused feature (the ability to calculate the fork from an arbitrary genesis file and world state) has a long cascading requirement that ultimately brings in all the RocksDB classes, and requires significantly more reflection config. Nobody uses the feature because it is fragile and buggy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In short, these need to be removed to use GraalVM for evmtool. I assert that no one will miss them.

Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
@atoulme
Copy link
Contributor

atoulme commented Mar 18, 2023

OK, I'm not sure if you'd like me to review this, as so many changes seem to be picked up along the way. I am happy to review if it helps.

@@ -264,6 +274,17 @@
<sha256 value="c2fd4366b1647233c99c70c17a2e90c89bf1ea60843749cca99d794525df9cd2" origin="Generated by Gradle"/>
</artifact>
</component>
<component group="com.fasterxml.jackson.core" name="jackson-annotations" version="2.13.3">
<artifact name="jackson-annotations-2.13.3.jar">
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already have 2.13.4 of these jackson dependencies - is there a reason we need this specific version?

Copy link
Contributor Author

@shemnon shemnon Mar 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the dependent libraries uses it. Verification is based on the total closure of all possible dependency references, not on the set of actually used references. You will also note that some SWT jars are also validated.

This was auto-added by the recommended gradle incantation.

@shemnon
Copy link
Contributor Author

shemnon commented Mar 20, 2023

@atoulme The current diff, as it stands, is the set of changes I would like to commit to get graalvm working. These changes do not appear in any other change list. Please review it as-is.

Copy link
Contributor

@macfarla macfarla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks ok to me

Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
@shemnon shemnon merged commit 0f97337 into hyperledger:main Mar 21, 2023
elenduuche pushed a commit to elenduuche/besu that referenced this pull request Aug 16, 2023
Changes and config to support using GraalVM AOT to compile and
execute evmTool as an alternate configuration. 

While not as long-term performant the startup time
makes filling reference tests with Besu reasonable.

Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
eum602 pushed a commit to lacchain/besu that referenced this pull request Nov 3, 2023
Changes and config to support using GraalVM AOT to compile and
execute evmTool as an alternate configuration. 

While not as long-term performant the startup time
makes filling reference tests with Besu reasonable.

Signed-off-by: Danno Ferrin <danno.ferrin@swirldslabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants