Get err "err="retrieved hash chain is invalid: invalid timestamp" and peer was removed

[nimysan]

System information

geth --version
geth version 1.13.13-stable-7f131dcb

uname:
Linux ip-172-31-69-208 6.2.0-1017-aws #17~22.04.1-Ubuntu SMP Fri Nov 17 21:07:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Expected behaviour

I start node(geth with clique) on ec2-1(172.31.70.63) and ec2-2(172.31.69.208).

once node on ec2-2 peer to node on ec2-1, I expected the blocks on ec2-1 can sync to ec2-2.

Actual behaviour

But I got failed.

Error:

DEBUG[02-26|15:39:55.097] Inserting downloaded chain items=74 firstnum=2 firsthash=672799..a36cc5 lastnum=75 lasthash=c1c855..5a45a9
INFO [02-26|15:39:55.097] Skip duplicated bad block number=2 hash=672799..a36cc5
ERROR[02-26|15:39:55.097]
########## BAD BLOCK #########
Block: 2 (0x672799a6a68e6cff47688db0a43f7c869d0e350312855916f0ad2b2f4ba36cc5)
Error: invalid timestamp
Platform: geth (devel) go1.21.6 amd64 linux
VCS: 7f131dc-
Chain config: &params.ChainConfig{ChainID:12345, HomesteadBlock:0, DAOForkBlock:, DAOForkSupport:false, EIP150Block:0, EIP155Block:0, EIP158Block:0, ByzantiumBlock:0, ConstantinopleBlock:0, PetersburgBlock:0, IstanbulBlock:0, MuirGlacierBlock:, BerlinBlock:0, LondonBlock:, ArrowGlacierBlock:, GrayGlacierBlock:, MergeNetsplitBlock:, ShanghaiTime:(*uint64)(nil), CancunTime:(*uint64)(nil), PragueTime:(*uint64)(nil), VerkleTime:(*uint64)(nil), TerminalTotalDifficulty:, TerminalTotalDifficultyPassed:false, Ethash:(*params.EthashConfig)(nil), Clique:(*params.CliqueConfig)(0xc000752230)}
Receipts:
##############################

DEBUG[02-26|15:39:55.097] Downloaded item processing failed number=2 hash=672799..a36cc5 err="invalid timestamp"
ERROR[02-26|15:39:55.097] Failed to retrieve beacon bounds for bad block reporting err="beacon sync not yet started"
DEBUG[02-26|15:39:55.097] Header download terminated peer=0b7a191b
DEBUG[02-26|15:39:55.097] Block body download terminated err="syncing canceled (requested)"
DEBUG[02-26|15:39:55.097] Receipt download terminated err="syncing canceled (requested)"
DEBUG[02-26|15:39:55.097] Synchronisation terminated elapsed=5.661ms
WARN [02-26|15:39:55.097] Synchronisation failed, dropping peer peer=0b7a191bb464cbcdad31888ad3722553837139926c11c9f84c17404aeee00b9d err="retrieved hash chain is invalid: invalid timestamp"
DEBUG[02-26|15:39:55.098] Message handling failed in eth id=0b7a191bb464cbcd conn=inbound err=EOF
DEBUG[02-26|15:39:55.098] Removing Ethereum peer peer=0b7a191b snap=true
TRACE[02-26|15:39:55.098] Unregistering sync peer peer=0b7a191b
INFO [02-26|15:39:55.098] Commit new sealing work number=2 sealhash=86696d..3385b5 txs=0 gas=0 fees=0 elapsed="86.277µs"
DEBUG[02-26|15:39:55.098] Message handling failed in snap peer=0b7a191b err=EOF
TRACE[02-26|15:39:55.098] Decrease miner recommit interval from=2s to=2s
DEBUG[02-26|15:39:55.098] Removing p2p peer peercount=0 id=0b7a191bb464cbcd duration=6.374ms req=false err="useless peer"

Steps to reproduce the behaviour

1. Genesis.json
2. Node1 on EC2-1
3. Start Node2 on EC2-2

Backtrace

DEBUG[02-26|15:39:55.097] Inserting downloaded chain               items=74 firstnum=2 firsthash=672799..a36cc5 lastnum=75 lasthash=c1c855..5a45a9
INFO [02-26|15:39:55.097] Skip duplicated bad block                number=2 hash=672799..a36cc5
ERROR[02-26|15:39:55.097] 
########## BAD BLOCK #########
Block: 2 (0x672799a6a68e6cff47688db0a43f7c869d0e350312855916f0ad2b2f4ba36cc5)
Error: invalid timestamp
Platform: geth (devel) go1.21.6 amd64 linux
VCS: 7f131dcb-
Chain config: &params.ChainConfig{ChainID:12345, HomesteadBlock:0, DAOForkBlock:<nil>, DAOForkSupport:false, EIP150Block:0, EIP155Block:0, EIP158Block:0, ByzantiumBlock:0, ConstantinopleBlock:0, PetersburgBlock:0, IstanbulBlock:0, MuirGlacierBlock:<nil>, BerlinBlock:0, LondonBlock:<nil>, ArrowGlacierBlock:<nil>, GrayGlacierBlock:<nil>, MergeNetsplitBlock:<nil>, ShanghaiTime:(*uint64)(nil), CancunTime:(*uint64)(nil), PragueTime:(*uint64)(nil), VerkleTime:(*uint64)(nil), TerminalTotalDifficulty:<nil>, TerminalTotalDifficultyPassed:false, Ethash:(*params.EthashConfig)(nil), Clique:(*params.CliqueConfig)(0xc000752230)}
Receipts: 
##############################

DEBUG[02-26|15:39:55.097] Downloaded item processing failed        number=2 hash=672799..a36cc5 err="invalid timestamp"
ERROR[02-26|15:39:55.097] Failed to retrieve beacon bounds for bad block reporting err="beacon sync not yet started"
DEBUG[02-26|15:39:55.097] Header download terminated               peer=0b7a191b
DEBUG[02-26|15:39:55.097] Block body download terminated           err="syncing canceled (requested)"
DEBUG[02-26|15:39:55.097] Receipt download terminated              err="syncing canceled (requested)"
DEBUG[02-26|15:39:55.097] Synchronisation terminated               elapsed=5.661ms
WARN [02-26|15:39:55.097] Synchronisation failed, dropping peer    peer=0b7a191bb464cbcdad31888ad3722553837139926c11c9f84c17404aeee00b9d err="retrieved hash chain is invalid: invalid timestamp"
DEBUG[02-26|15:39:55.098] Message handling failed in `eth`         id=0b7a191bb464cbcd conn=inbound err=EOF
DEBUG[02-26|15:39:55.098] Removing Ethereum peer                   peer=0b7a191b snap=true
TRACE[02-26|15:39:55.098] Unregistering sync peer                  peer=0b7a191b
INFO [02-26|15:39:55.098] Commit new sealing work                  number=2 sealhash=86696d..3385b5 txs=0 gas=0 fees=0 elapsed="86.277µs"
DEBUG[02-26|15:39:55.098] Message handling failed in `snap`        peer=0b7a191b err=EOF
TRACE[02-26|15:39:55.098] Decrease miner recommit interval         from=2s to=2s
DEBUG[02-26|15:39:55.098] Removing p2p peer                        peercount=0 id=0b7a191bb464cbcd duration=6.374ms req=false err="useless peer"

When submitting logs: please submit them as text and not screenshots.

[nimysan]

genesis.json

{
  "config": {
    "chainId": 12345,
    "homesteadBlock": 0,
    "eip150Block": 0,
    "eip155Block": 0,
    "eip158Block": 0,
    "byzantiumBlock": 0,
    "constantinopleBlock": 0,
    "petersburgBlock": 0,
    "istanbulBlock": 0,
    "berlinBlock": 0,
    "clique": {
      "period": 5,
      "epoch": 300
    }
  },
  "difficulty": "1",
  "gasLimit": "800000000",
  "extradata": "0x0000000000000000000000000000000000000000000000000000000000000000f3Fa65794aB08c1e06f01A0F9B39A19bB7349FC50000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
  "alloc": {
    "0xf3Fa65794aB08c1e06f01A0F9B39A19bB7349FC5": {
      "balance": "1000000000000000000000"
    },
    "0x6825f402a0E89032970dcad1Ca006D8F23921216": {
      "balance": "1000000000000000000000"
    },
    "0x51293994FCE33c606d0107119Ea752DEF8BE6455": {
      "balance": "1000000000000000000000"
    }
  }
}

[nimysan]

I got this error due to that the genesis.json blockPeriod does not same on EC2-1 and EC2-2. Close it.