Update Astria-geth to use geth 1.14.3 #21

bharath-123 · 2024-05-29T16:42:30Z

This PR rebases astria-geth on top of geth 1.14.3. Below are the main changes which affect astria-geth:

There seems to be an overall shift in the Geth code to prefer uint256.NewInt over big.NewInt. There are some places in our codebase where I had to typecast bigInt to uint256. But this usage also seems to be pretty inconsistent overall. I suspect as geth updates, most of the codebase will eventually use uint256.
The block building process under the miner directory used to have a worker struct in pre 1.14 geth. The payloadBuilding process was as method on the worker struct. With geth 1.14 there is a major refactor to simplify the codebase by removing the worker struct. Now we need to define all the methods under the Miner struct. This required some modifications to fillAstriaTransactions and commitAstriaTransactions to be implemented on the Miner struct.
Files Changed
miner/payload_building.go
miner/worker.go
Methods like AddBalance and SubBalance which update the balance of the user in the stateDB now requires to add an extra parameter indicating the reason of change of balance. This is used for live tracing introduced in geth 1.14. This is basically an enum. I added a new value called BalanceIncreaseAstriaDepositTx which indicates a balance increase due to an astria deposit tx.
Files Changed:
core/state_transition.go
tracing/hooks.go
Geth re-worked their testing infra. Integration tests use a SimulatedBackend which mocked the geth RPC. It simulates api calls such as sendTransaction. Pre geth 1.14, when sendTransaction was called with SimulatedBackend, txs bypassed the mempool and were directly added to the block. Post geth 1.14, this is not the case anymore and the txs are sent to the mempool. This breaks certain tests for us, since we rely on txs to come from the Execution API. I have commented out the tests for now but it ll be worth adding a simulated backend which bypasses the tx for our purposes. It ll be useful to test certain codepaths.
The following tests have been commented out:
a. TestWaitDeployed
b. TestGolangBindings
c. TestEthClient/AtFunctions
d. TestForkResendTx

Testing: make test runs without problems. I deployed this astria-evm locally using the monorepo charts and was able to run spamooor scenarios successfully against it.

bharath-123 · 2024-06-04T14:36:11Z

grpc/execution/server.go

-	if merger := eth.Merger(); !merger.PoSFinalized() {
-		merger.FinalizePoS()
-	}


There is some legacy code being removed in geth.

ethereum/go-ethereum#29169

bharath-123 · 2024-06-04T14:36:46Z

grpc/execution/server_test.go

-	require.True(t, balanceDiff.Cmp(big.NewInt(1000000000000000000)) == 0, "Chain destination address balance is not correct")
+	balanceDiff := new(uint256.Int).Sub(chainDestinationAddressBalanceAfter, chainDestinationAddressBalanceBefore)
+	fmt.Printf("Balance diff: %s\n", balanceDiff.String())
+	require.True(t, balanceDiff.Cmp(uint256.NewInt(1000000000000000000)) == 0, "Chain destination address balance is not correct")


examples of some changes to uint256.Int. This is because the stateDB is now returning only uint256s.

bharath-123 · 2024-06-04T14:42:49Z

core/state_transition.go

@@ -364,7 +381,7 @@ func (st *StateTransition) preCheck() error {
 func (st *StateTransition) TransitionDb() (*ExecutionResult, error) {
 	// if this is a deposit tx, we only need to mint funds and no gas is used.
 	if st.msg.IsDepositTx {
-		st.state.AddBalance(st.msg.From, st.msg.Value)
+		st.state.AddBalance(st.msg.From, uint256.MustFromBig(st.msg.Value), tracing.BalanceIncreaseAstriaDepositTx)


example of requirement of uint256 and adding tracing reason

bharath-123 · 2024-06-04T14:43:17Z

core/state_transition.go


 		// collect base fee instead of burn
 		if rules.IsLondon && st.evm.Context.Coinbase.Cmp(common.Address{}) != 0 {
 			baseFee := new(big.Int).SetUint64(st.gasUsed())
 			baseFee.Mul(baseFee, st.evm.Context.BaseFee)
-			st.state.AddBalance(st.evm.Context.Coinbase, baseFee)
+			st.state.AddBalance(st.evm.Context.Coinbase, uint256.MustFromBig(baseFee), tracing.BalanceIncreaseRewardTransactionFee)


Update from big.int to uint256

bharath-123 · 2024-06-04T16:24:26Z

miner/payload_building.go

@@ -175,7 +176,10 @@ func (payload *Payload) ResolveFull() *engine.ExecutionPayloadEnvelope {
 }

 // buildPayload builds the payload according to the provided parameters.
-func (w *worker) buildPayload(args *BuildPayloadArgs) (*Payload, error) {
+func (miner *Miner) buildPayload(args *BuildPayloadArgs) (*Payload, error) {


key change: buildPayload is now based on Miner rather than worker.

bharath-123 · 2024-06-04T16:25:59Z

miner/worker.go

@@ -20,57 +20,18 @@ import (
 	"errors"


the worker.go file is now very simplified.

joroshiba · 2024-06-10T15:50:33Z

I think we might need some fresh updates after Elizabeth PR merge, but otherwise LGTM

Co-authored-by: Felix Lange <fjl@twurst.com>

* node: fix test if directory already exists * node: remove test

…tion (#29169) * eth: drop support for forward sync triggers and head block packets * consensus, eth: enforce always merged network * eth: fix tx looper startup and shutdown * cmd, core: fix some tests * core: remove notion of future blocks * core, eth: drop unused methods and types

bump pebble

* add buf config and generated grpc code * poc e2e grpc communication * comment out panicking code * logging help * now reads cli args. now stops grpc server on shutdown. * add mutex to GRPCServerHandler update readme containerize w/ github action to build and push to ghcr (#3) fix gh action syntax (#4) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax build and push manually (#5) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually Feature/containerize (#6) * containerize w/ github action to build and push to ghcr * fix gh action yml syntax * enable manual trigger * fix syntax * dont use github action, do it ourselves * push manually * correct multi line update SubmitTransaction to send tx to metro ethclient: ensure returned subscription is nil on error (#26976) core/state, trie: remove Try prefix in Trie accessors (#26975) This change renames StateTrie methods to remove the Try* prefix. We added the Trie methods with prefix 'Try' a long time ago, working around the problem that most existing methods of Trie did not return the database error. This weird naming convention has persisted until now. Co-authored-by: Gary Rong <garyrong0905@gmail.com> metrics/librato: ensure resp.body closed (#26969) This change ensures that we call Close on a http response body, in various places in the source code (mostly tests) core/vm: use atomic.Bool (#26951) Make use of new atomic types --------- Co-authored-by: Felix Lange <fjl@twurst.com> Co-authored-by: Martin Holst Swende <martin@swende.se> core/bloombits: use atomic type (#26993) core/state: use atomic.Bool (#26992) graphql: fix data races (#26965) Fixes multiple data races caused by the fact that resolving fields are done concurrently by the graphql library. It also enforces caching at the stateobject level for account fields. eth/tracers/native: prevent panic for LOG edge-cases (#26848) This PR fixes OOM panic in the callTracer as well as panicing on opcode validation errors (e.g. stack underflow) in callTracer and prestateTracer. Co-authored-by: Martin Holst Swende <martin@swende.se> internal/debug: add log.logfmt flag to set logging to use logfmt (#26970) docs: update outdated DeriveSha docs comment (#26968) add localnet genesis update metro-transactions dep Adding features to mempool to support our pre-ordered txs (#2) * Adding features to mempool to support our pre-ordered txs * Add framing for gRPC to execute blocks * Remove public engine API call * Fix circular dependencies * Updated to use new Init, and fill out starting attributes * Add clear astriaordered, cleanup * readme fix * add bash and jq to final docker image * make txpool interface * no more panics, update DoBlock to also update state + store block * cleanup * set post-merge at genesis * cleanup * doc update * remove txpool interface changes * cleanup * cleanup * build and push images wih tags defined by git tags * build for multiple architectures. use docker-metadata action for semver * add push: true * only build arm for git tags/releases --------- Co-authored-by: Jesse Snyder <jessetsnyder@gmail.com> Co-authored-by: elizabeth <elizabethjbinks@gmail.com> tag image with latest for builds from astria branch (#8) rename state_root to block_hash use FROM --platform=$BUILDPLATFORM to build arm images correctly (#9) use v4 of build and push action (#10) * use FROM --platform=$BUILDPLATFORM to build arm images correctly * use v4 of docker build and push fix arm builds. only build arm for tags and merges to default branch. (#12) * fix arm builds. only build arm for tags and merges to default branch. * fix gh action function syntax * more correct comment * build for semver tags Features and fixes needed for contract deployment test (#13) * add and remove tx from geth mempool so forge can deploy a contract * pass metro addr and port by flag * fixes from pr feedback * formatting implement FinalizeBlock gRPC call bump metro-transactions dep Update Protos (#16) * uses new protos * remove old protos update deps update deps no more unknown/unknown image (#19) * no more unknown/unknown image * hardcode condition Changing from prev_state_root to prev_block_hash remove metro integrate submission to sequencer log updates use env vars for chain id/tendermint endpoint rename to cometbft Make the environment variables actually work cleaner simple logging change removed sleep before getPayload by removing goroutine in buildPayload removed unneeded comment removed for loop from buildPayload Remove direct sequencer submission cleanup go.mod Migrate to the v1alpha1 API Missed pieces set genesis block as head/safe/final remove all mentions of metro Initial implementation of Execution v1alpha2 api Integrate updates remove need for consensus api Updates small updates Update grpc/execution/server.go Co-authored-by: noot <36753753+noot@users.noreply.github.com> logging remove unused dependency Mark the chain 'finalized' on startup

EIP-1559 burns the base fee. This doesn't make sense for rollups. The set fee recipient should collect it.

.github/workflows/pages.yaml

* main: fix: update with new asset proto (#34) feat: support bech32 addresses (#30) fix: Ensure all txs are removed from mempool when block building is interrupted (#29) disallow users from sending deposit txs and blob txs via the rpc (#24) Update Astria-geth to use geth 1.14.3 (#21)

bharath-123 force-pushed the bharath/update-to-1.14 branch 2 times, most recently from 4b8ecbe to b757f5b Compare May 30, 2024 10:45

bharath-123 changed the title ~~Update to geth 1.14~~ Update Astria-geth to use geth 1.14 May 30, 2024

bharath-123 marked this pull request as ready for review May 30, 2024 11:10

bharath-123 requested review from joroshiba, mycodecrafting and noot as code owners May 30, 2024 11:10

Base automatically changed from bharath/add-execution-server-tests to main May 31, 2024 06:25

bharath-123 force-pushed the bharath/update-to-1.14 branch 4 times, most recently from 2b28ac2 to 1fbd448 Compare June 4, 2024 14:19

bharath-123 commented Jun 4, 2024

View reviewed changes

miner/worker.go

@@ -20,57 +20,18 @@ import (

"errors"

Copy link

Contributor Author

bharath-123 Jun 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the worker.go file is now very simplified.

undefinedor and others added 11 commits June 10, 2024 22:19

rpc: remove deprecated method "Notifier.Closed" (#29162)

f57c74a

rlp: using unsafe.Slice instead of SliceHeader (#29067)

9b8cc25

Co-authored-by: Felix Lange <fjl@twurst.com>

core: remove useless assignments (#29065)

4e0560a

node: remove test which doesn't do a lot (#29159)

20bac3f

* node: fix test if directory already exists * node: remove test

go.mod: bump pebble db to official release (#29038)

7c92378

bump pebble

internal/ethapi: delete needless error check (#29127)

c400dda

accounts/usbwallet: revert #28945 (#29175)

c39ab8e

rpc: add more test cases for arg types (#29006)

e2829d5

accounts/usbwallet: update hid library (#29176)

1772350

accounts: remove deprecated function NewPlaintextKeyStore (#29171)

d6bf595

fjl and others added 17 commits June 10, 2024 22:19

params: release go-ethereum v1.14.3 stable

211797f

updates

464e100

txpool fixes, remove v1alpha1, use forkchoice

4572f89

some cleanup

d4d13bc

think it's good

ea3c033

chore: add some metrics specifically around astria tx execution

2570221

small refactor

7fe7a8a

feat: update protos, config for fees, updated bridging (#1)

63a8f8f

collect the base fee instead of burning it (#5)

6a0a2e2

EIP-1559 burns the base fee. This doesn't make sense for rollups. The set fee recipient should collect it.

add unit tests for execution grpc server

599d715

fix tracer test

816f273

fix tests

382432b

fix merge conflicts in github workflows

771eeba

fix tests

e00d4fd

remove i386 tests

4b3988d

minor fixes

14a96f6

bharath-123 force-pushed the bharath/update-to-1.14 branch from e8ceb69 to 14a96f6 Compare June 10, 2024 17:24

bharath-123 changed the title ~~Update Astria-geth to use geth 1.14~~ Update Astria-geth to use geth 1.14.0 Jun 11, 2024

chore: update forkdiff base to go-ethereum-1.14

a30fd3d

bharath-123 changed the title ~~Update Astria-geth to use geth 1.14.0~~ Update Astria-geth to use geth 1.14.3 Jun 11, 2024

bharath-123 and others added 3 commits June 11, 2024 17:26

minor fix

a13be6d

update forkdiff base to v1.14.3

d82e19a

fix minor merge conflicts

5f9724b

bharath-123 force-pushed the bharath/update-to-1.14 branch from a7ab4e4 to 5f9724b Compare June 11, 2024 13:48

joroshiba reviewed Jun 12, 2024

View reviewed changes

.github/workflows/pages.yaml Show resolved Hide resolved

joroshiba approved these changes Jun 19, 2024

View reviewed changes

joroshiba merged commit 2b8a8ee into main Jun 20, 2024
2 checks passed

joroshiba deleted the bharath/update-to-1.14 branch June 20, 2024 15:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Astria-geth to use geth 1.14.3 #21

Update Astria-geth to use geth 1.14.3 #21

bharath-123 commented May 29, 2024 •

edited

Loading

bharath-123 Jun 4, 2024

bharath-123 Jun 11, 2024

bharath-123 Jun 4, 2024

bharath-123 Jun 4, 2024

bharath-123 Jun 4, 2024

bharath-123 Jun 4, 2024

bharath-123 Jun 4, 2024

joroshiba commented Jun 10, 2024

Update Astria-geth to use geth 1.14.3 #21

Update Astria-geth to use geth 1.14.3 #21

Conversation

bharath-123 commented May 29, 2024 • edited Loading

bharath-123 Jun 4, 2024

Choose a reason for hiding this comment

bharath-123 Jun 11, 2024

Choose a reason for hiding this comment

bharath-123 Jun 4, 2024

Choose a reason for hiding this comment

bharath-123 Jun 4, 2024

Choose a reason for hiding this comment

bharath-123 Jun 4, 2024

Choose a reason for hiding this comment

bharath-123 Jun 4, 2024

Choose a reason for hiding this comment

bharath-123 Jun 4, 2024

Choose a reason for hiding this comment

joroshiba commented Jun 10, 2024

bharath-123 commented May 29, 2024 •

edited

Loading