[base node] Orphan block failed validation during chain reorg: IncorrectPreviousHash

[hansieodendaal]

This may be a serious bug - when we are trying to add this block, we already have determined that it links to the chain so it shouldn't ever fail

12:24 WARN  Orphan block 2953 (71977a234e271792325dc2216d9a84976752b70e115732032970fcb870799c3d) failed validation during chain reorg: IncorrectPreviousHash { expected: "fc6cf61474c3f949494a60b7bf06d11386210e6bd2bf024985374bee4eeaede5", block_hash: "71977a234e271792325dc2216d9a84976752b70e115732032970fcb870799c3d" }
12:24 WARN  Peer f88e0eafd2ed7f879fa8698857 sent an invalid block: Validation error: Expected block previous hash to be fc6cf61474c3f949494a60b7bf06d11386210e6bd2bf024985374bee4eeaede5, but was 71977a234e271792325dc2216d9a84976752b70e115732032970fcb870799c3d

[SWvheerden]

might be related:

19:29 v0.38.0, esmeralda, State: Starting header sync, Tip: 1080 (Fri, 02 Sep 2022 17:28:51 +0000), Mempool: 407tx (10996326g, +/- 87blks), Connections: 8, Banned: 1, Messages (last 60s): 59, Rpc: 4/100
19:29 �[0;31;1mERROR�[0m Failed to apply DB transaction: InvalidOperation("The header at height 1080 already exists. Existing header hash: 1ed0699f1b0debb33cad1e0adac3a7b94608c73e181265d1f0d9d5f47d5f1590")
19:29 �[0;31;1mERROR�[0m error=Chain storage error: You tried to execute an invalid Database operation: The header at height 1080 already exists. Existing header hash: 1ed0699f1b0debb33cad1e0adac3a7b94608c73e181265d1f0d9d5f47d5f1590
19:29 �[0;31;1mERROR�[0m Failed to synchronize headers from peer `c1050429d43f8bbb8a88c0bab7`: Chain storage error: You tried to execute an invalid Database operation: The header at height 1080 already exists. Existing header hash: 1ed0699f1b0debb33cad1e0adac3a7b94608c73e181265d1f0d9d5f47d5f1590**

Node was at 1079, started header sync, just as the tip increased to 1080

[SWvheerden]

Another instance of this happening:

14:02 �[0;33mWARN�[0m  Block sync failed: Chain storage error: Key df680100 in deleted_txo_mmr_position_to_height_index already exists
14:02 �[0;33mWARN�[0m  Orphan block 686 (4abcfac81283989d0c7abe737ad2baf3e20046b5a4638e48235000af14912aaa) failed validation during chain reorg: IncorrectPreviousHash { expected: "68840ec3dc835f6925b300a050522336816158efb2cc523698e7875bd959ae69", block_hash: "4abcfac81283989d0c7abe737ad2baf3e20046b5a4638e48235000af14912aaa" }
14:02 �[0;33mWARN�[0m  Peer b49822d0d1fb08416ae53f3c84 sent an invalid block: Validation error: Expected block previous hash to be 68840ec3dc835f6925b300a050522336816158efb2cc523698e7875bd959ae69, but was 4abcfac81283989d0c7abe737ad2baf3e20046b5a4638e48235000af14912aaa
14:02 �[0;31;1mERROR�[0m Failed to handle incoming block message: CommsInterfaceError(ChainStorageError(ValidationError { source: IncorrectPreviousHash { expected: "68840ec3dc835f6925b300a050522336816158efb2cc523698e7875bd959ae69", block_hash: "4abcfac81283989d0c7abe737ad2baf3e20046b5a4638e48235000af14912aaa" } }))

[SWvheerden]

pretty sure both cases are caused by a race condition between the RWLOCK of the db and the db.set_disable_add_block_flag() command.

[stringhandler]

Sounds like this is related to #4668

[SWvheerden]

fixed in #4677