Goerli Deadlock Discussion #78
Comments
|
Looking at the Gas usage here...
|
|
My Besu validator reports |
|
We have two competing heads
And 2 offline validators
So we either have to manually resync Geth and/or Infura validators. Or wait for Prysm and/or POA to online their validators. Edit: Prysm appears to be online but stuck or resyncing? |
|
Our geth node appears to be resyncing very slowly after updating to geth 1.10.0. The node went offline for some time yesterday after the update due to deletion of deprecated flags and now has been syncing goerli for about 28 hours now. Still syncing state entries with no ETA. Latest logs: |
|
Ok, Igor confirmed their node is also resyncing. |
|
Ok, Ronin is also resyncing. So whoever is done with the sync first will kick off a new block |
|
|
|
Syncing Goerli for 28 hours? Something must be amiss. Even Mantis can full sync goerli faster than that. |
|
Do geth and besu have different fork choice rules? Do we need to soft fork to the same tie breaker? |
|
We restarted our node and now it is syncing again from genesis... We may need to roll back to 1.9. Edit: We'll wait a few hours to see if this syncs successfully. |
no, the problem is (once again) the race for out of turn blocks creating competing heads with the same total difficulty and the corner case that there is a deadlock situation (unable to decide on the same best head). this was discussed in EIPs 1955, 2181, 2182, and 2183. |
|
Yeah
The gas limit is a red herring -- I skimmed through the last 12h of blocks and there were multiple streaks with a gas limit above 8mio. |
|
I think we need better fork choice rules in the protocol then. Something like (in order)
These can be soft forked without any wire changes. We may want to add something before the final arbitrator that prefer signers that are closest to in-order. But "first seen" is a always a chain halt waiting to happen. |
|
I just thought the same.
This should resolve this fairly easily. Wondering if this would even be beneficial for other consensus engines or if we should limit this to Clique only. |
|
running again :) |
|
Thanks everyone. |
|
@shemnon should this be an EIP or shall we discuss this elsewhere with the teams? |
|
let's get consensus before we post an EIP. |
|
it´s stuck again, confirmed! |
|
restarted OpenEthereum Goerli suprn0de#Z1 from zero- |
|
So I noticed that our geth nodes were on:
But then reorg'ed back to:
Which is a one less height but with the same TD. And then a and a new block What I don't understand is why did it reorg to a lower block number with the same TD, shouldn't it only reorg once it sees a higher TD? Is this basically what Danno was suggesting in #78 (comment)? Note that this is a normal node not a signer/validator. |
|
Update from Prysmatic Labs. We are having difficulty syncing with geth 1.10, we have rolled back to 1.9 and it is syncing state entries. Not sure how long it takes to sync goerli, but it has been 3h37m since we rolled back. There seem to be around 20000 state entries left. |
|
We are synced now and proposed a block. We will stay on 1.9 until it is required to upgrade for Berlin. |
|
Looks like geth always prefers shorter chains - https://github.com/ethereum/go-ethereum/blob/master/core/blockchain.go#L1596 |
We are stuck again but now both chains are the same length and same total difficulty. maybe we can agree on lowest hash? |
|
Also wondering if it's related to Swarm filling up all blocks. (Should not, but this should also be resolved asap, currently 5k pending transactions.) |
Would it be possible to patch Besu with such a forkchoice rule (diff, number, hash) so we could test it on Goerli? |
Hi guys, Swarm product owner here. The surge of blockchain transactions coming from Swarm comes from users who want to earn the right to get airdropped BZZ tokens when Swarm goes live on mainnet (see announcement). There are currently 9000+ nodes in the Swarm network, of which the vast majority is not controlled by us. Most transactions we are seeing are from users cashing out cheques. A cheque is some sort of a layer-2 construct, allowing users to pay each other without going on L1. The cashing-out of a cheque is a L1 transaction, causing a transfer of tokens (see smart-contracts). The airdrop-announcement is misunderstood by some users, who believe that each cheque is worth a certain amount of airdropped BZZ, whereas in fact the worth of the cheques is what will give you the right for airdropped BZZ. Because of this, the number of transactions arising from Swarm usage is much more than strictly needed. We attempt to rectify this misunderstanding, by clearly outlining that the frequently cashing out cheques does not yield any benefit for them and by making our users aware that the Goerli network is a public resource which should be mindfully utilized. Please let me know how the situation evolves. You can find me on the Swarm Discord as @Eknir. |
|
I have one question. To what degree is the current situation on the Goerli network caused by our users? |
|
Answered in DM (not at all) |
Online:
0x22ea9f6b28db76a7162054c05ed812deb2f519cd 5/25 ConsenSys
0x4c2ae482593505f0163cdefc073e81c63cda4107 4/25 Nethermind
0x8b24eb4e6aae906058242d83e51fb077370c4720 4/25 Infura/EG
0x9d525e28fe5830ee92d7aa799c4d21590567b595 4/25 Görli/Ronin
0xa6dd2974b96e959f2c8930024451a30afec24203 4/25 EF/Geth
0xe0a2bd4258d2768837baa26a28fe71dc079f84c7 4/25 Görli/Afri
Offline
The text was updated successfully, but these errors were encountered: