Skip to content
This repository has been archived by the owner on Nov 6, 2020. It is now read-only.

Invalid old blocks #9306

Closed
ascjones opened this issue Aug 8, 2018 · 5 comments · Fixed by #9531
Closed

Invalid old blocks #9306

ascjones opened this issue Aug 8, 2018 · 5 comments · Fixed by #9531
Labels
F2-bug 🐞 The client fails to follow expected behavior. M4-core ⛓ Core client code / Rust. P2-asap 🌊 No need to stop dead in your tracks, however issue should be addressed as soon as possible.
Milestone
2.2

Comments

@ascjones
Copy link
Contributor

ascjones commented Aug 8, 2018 •
edited
Loading

I'm running:

  • Which Parity version?: 2.0.1
  • Which operating system?: Linux
  • How installed?: binaries
  • Are you fully synchronized?: yes
  • Which network are you connected to?: ethereum
  • Did you try to restart the node?: yes

After warp sync completes, old/ancient blocks are downloaded. On my node, starting at #1 some of those blocks are invalid, followed by valid blocks. Originally this was true only for #1, all other blocks that I queried were valid. After a fresh sync, the same occurred except the range #1..#57 was invalid:

Block 1 ❌
Expected 0x88e96d4537bea4d9c05d12549907b32561d3bf31f45aae734cdc119f13406cb6

curl -sX POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["0x1", false], "id": 75}' localhost:8545 | jq ".result.hash"
"0x49077beab2fa25f648dab021318e203f9f4dc6aed7e4b6b26526f63673b8ef41"

Block 57 ❌
Expected 0xf394e8a094b2c701e66c8074ee7da247b676f71ba1633a68435f5e2aee975c5c

curl -sX POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["0x39", false], "id": 75}' localhost:8545 | jq ".result.hash"
"0xa638e3dd16207e5cebab1b057e0e9b97762321df22b7cd28ff0a6cdc7b78a7d1"

Block 58 ✔️
Expected 0xcd5b5c4cecd7f18a13fe974255badffd58e737dc67596d56bc01f063dd282e9e

curl -sX POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["0x3a", false], "id": 75}' localhost:8545 | jq ".result.hash"
"0xcd5b5c4cecd7f18a13fe974255badffd58e737dc67596d56bc01f063dd282e9e"

It appears that one of our bootnodes is the source of these bad blocks. @ngotchac has been investigating that.

I'm going to run a fresh sync with sync tracing enabled to get more detailed logs.
@ascjones ascjones added F2-bug 🐞 The client fails to follow expected behavior. M4-core ⛓ Core client code / Rust. labels Aug 8, 2018
@ascjones ascjones added this to the 2.1 milestone Aug 8, 2018
@ascjones
Copy link
Contributor Author

ascjones commented Aug 16, 2018 •
edited
Loading

I've managed to reproduce this, except this time #1 -> #54 are good and #55, #56, #57 are bad. Here are the logs for the latest downloaded old block:

2018-08-14 19:54:05  IO Worker #0 INFO import         #54    2/750 peers   5 MiB chain 3 GiB db 0 bytes queue 7 MiB sync  RPC:  0 conn,    0 req/s,   47 µs
2018-08-14 19:54:40  IO Worker #1 INFO import         #54    2/750 peers   111 KiB chain 3 GiB db 0 bytes queue 7 MiB sync  RPC:  0 conn,    0 req/s,   47 µs
2018-08-14 19:55:15  IO Worker #0 INFO import         #54    2/750 peers   120 KiB chain 3 GiB db 0 bytes queue 7 MiB sync  RPC:  0 conn,    0 req/s,   47 µs
2018-08-14 19:56:10  IO Worker #0 INFO import         #54    2/750 peers   175 KiB chain 3 GiB db 0 bytes queue 7 MiB sync  RPC:  0 conn,    0 req/s,   58 µs
2018-08-14 19:56:45  IO Worker #3 INFO import         #54    2/750 peers   249 KiB chain 3 GiB db 0 bytes queue 7 MiB sync  RPC:  0 conn,    0 req/s,   58 µs
2018-08-14 19:57:30  IO Worker #2 INFO import         #54    2/750 peers   408 KiB chain 3 GiB db 0 bytes queue 7 MiB sync  RPC:  0 conn,    0 req/s,   58 µs
2018-08-14 19:59:40  IO Worker #2 INFO import         #54    2/750 peers   520 KiB chain 3 GiB db 0 bytes queue 7 MiB sync  RPC:  0 conn,    0 req/s,   58 µs
2018-08-14 20:05:30  IO Worker #3 INFO import      #10795   85/750 peers   5 MiB chain 2 GiB db 0 bytes queue 1 MiB sync  RPC:  0 conn,    0 req/s,    0 µs
...
2018-08-14 21:31:16  IO Worker #1 INFO import    #1917030   129/750 peers   4 MiB chain 2 GiB db 0 bytes queue 5 MiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:31:51  IO Worker #1 INFO import    #1919316   128/750 peers   6 MiB chain 2 GiB db 0 bytes queue 13 MiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:33:01  IO Worker #2 INFO import          #2   129/750 peers   316 KiB chain 2 GiB db 0 bytes queue 321 KiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:33:36  IO Worker #2 INFO import          #2   133/750 peers   5 MiB chain 2 GiB db 0 bytes queue 321 KiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:34:11  IO Worker #3 INFO import         #54   134/750 peers   5 MiB chain 2 GiB db 0 bytes queue 586 KiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:34:46  IO Worker #3 INFO import         #54   131/750 peers   7 MiB chain 2 GiB db 0 bytes queue 586 KiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:35:46  IO Worker #3 INFO import         #54   129/750 peers   5 MiB chain 2 GiB db 0 bytes queue 3 MiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:36:21  IO Worker #1 INFO import         #54   130/750 peers   5 MiB chain 2 GiB db 0 bytes queue 3 MiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:36:56  IO Worker #0 INFO import         #54   134/750 peers   5 MiB chain 2 GiB db 0 bytes queue 3 MiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:37:31  IO Worker #1 INFO import         #54   132/750 peers   5 MiB chain 2 GiB db 0 bytes queue 3 MiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:38:06  IO Worker #0 INFO import         #54   135/750 peers   7 MiB chain 2 GiB db 0 bytes queue 3 MiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:39:31  IO Worker #0 INFO import         #54   124/750 peers   7 MiB chain 2 GiB db 0 bytes queue 3 MiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:40:16  IO Worker #2 INFO import         #57   124/750 peers   6 MiB chain 2 GiB db 0 bytes queue 3 MiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:41:26  IO Worker #2 INFO import         #57   107/750 peers   6 MiB chain 2 GiB db 0 bytes queue 4 MiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:43:46  IO Worker #1 INFO import         #57   120/750 peers   5 MiB chain 2 GiB db 0 bytes queue 5 MiB sync  RPC:  0 conn,    0 req/s,  184 µs
2018-08-14 21:44:21  IO Worker #3 INFO import         #57   120/750 peers   6 MiB chain 2 GiB db 0 bytes queue 5 MiB sync  RPC:  0 conn,    0 req/s,  184 µs
...
2018-08-15 16:37:44  IO Worker #1 INFO import         #57   38/750 peers   5 MiB chain 2 GiB db 0 bytes queue 185 MiB sync  RPC:  0 conn,    0 req/s,  432 µs
2018-08-15 16:38:19  IO Worker #1 INFO import         #57   40/750 peers   5 MiB chain 2 GiB db 0 bytes queue 185 MiB sync  RPC:  0 conn,    0 req/s,  432 µs
2018-08-15 16:38:54  IO Worker #3 INFO import         #57   39/750 peers   6 MiB chain 2 GiB db 0 bytes queue 185 MiB sync  RPC:  0 conn,    0 req/s,  432 µs

So it downloads #1 -> #54 (bad), then later downloads good blocks up until #1923590 (near the forkBlock - coincidence?). Then receives again blocks #2 -> #57 which contain the dodgy 3 blocks.

Here is where it gets the 3 dodgy ones:

2018-08-14 21:40:07  IO Worker #0 TRACE sync  68 -> BlockHeaders (128 entries), state = Idle, set = OldBlocks
2018-08-14 21:40:07  IO Worker #0 TRACE sync  Skipping existing block header 54 (0xb10d31a7d35e1031817f095bf910267401bbd215fbe6eb902c3ff4585ecadc32)
2018-08-14 21:40:07  IO Worker #0 TRACE sync  Skipping existing block header 58 (0xcd5b5c4cecd7f18a13fe974255badffd58e737dc67596d56bc01f063dd282e9e)
2018-08-14 21:40:07  IO Worker #0 TRACE sync  Skipping existing block header 59 (0x4d9423080290a650eaf6db19c87c76dff83d1b4ab64aefe6e5c5aa2d1f4b6623)
...
2018-08-14 21:40:07  IO Worker #0 TRACE sync  New header: 3379b8e9b4c1989994b4bd5b2e7313eb772b227045ac8e8271101a38080876fd
2018-08-14 21:40:07  IO Worker #0 TRACE sync  New header: 9ca0372f3bbc90aecb7cf8cddc3c5e82869e02ee4c0598b514141cca7b6c4557
2018-08-14 21:40:07  IO Worker #0 TRACE sync  New header: a638e3dd16207e5cebab1b057e0e9b97762321df22b7cd28ff0a6cdc7b78a7d1
2018-08-14 21:40:07  IO Worker #0 TRACE sync  Inserted 3 headers
2018-08-14 21:40:07  IO Worker #0 TRACE sync  Drained 3 blocks, new head :Some(0xa638e3dd16207e5cebab1b057e0e9b97762321df22b7cd28ff0a6cdc7b78a7d1)
2018-08-14 21:40:07  IO Worker #0 TRACE sync  Block queued 0x3379b8e9b4c1989994b4bd5b2e7313eb772b227045ac8e8271101a38080876fd
2018-08-14 21:40:07  IO Worker #0 TRACE sync  Block queued 0x9ca0372f3bbc90aecb7cf8cddc3c5e82869e02ee4c0598b514141cca7b6c4557
2018-08-14 21:40:07  IO Worker #0 TRACE sync  Block queued 0xa638e3dd16207e5cebab1b057e0e9b97762321df22b7cd28ff0a6cdc7b78a7d1
2018-08-14 21:40:07  IO Worker #0 TRACE sync  Imported 3 of 3

Continuing to investigate.

@ascjones
Copy link
Contributor Author

Roughly here is how it ends up in this state:

  1. Fresh warp sync, snapshot sync would complete
  2. Old blocks download would begin from genesis
  3. Bad set of blocks #1 -> #54 downloaded at first
  4. Eventually finds a good peer and overwrites bad blocks with good blocks
  5. Continues syncing, and at some point:
    Bad block 0x6457f27aaef51ed65de71ff01ccdc0a88776797543d0a267cad4b1a78fb61942 : BlockImportError(Other("The queue is full (4096)")
    2018-08-14 21:31:53 IO Worker #1 TRACE sync Restarting
  6. The restart causes the sync to start again from genesis
  7. We receive a set of blocks #1..#57 where #1..#54 are good but #55..#57 are bad.
  8. Old blocks syncing gets stuck on #57, and we have a few bad old blocks.

Putting together a PR to prevent it getting into this state.

@ascjones ascjones mentioned this issue Aug 20, 2018
Merged
@ascjones
Copy link
Contributor Author

ascjones commented Aug 21, 2018 •
edited
Loading

Old blocks still failed to sync correctly, but this time via a different path. It could be caused by my change since things start going bad after it gets the queue full error.

  • Peers drop to 0
2018-08-20 19:55:01  IO Worker #0 TRACE sync  Syncing with peers: 10 active, 12 confirmed, 12 total
2018-08-20 19:55:01  IO Worker #1 TRACE sync  Syncing with peers: 9 active, 11 confirmed, 11 total
2018-08-20 19:55:01  IO Worker #0 TRACE sync  Syncing with peers: 8 active, 10 confirmed, 10 total
2018-08-20 19:55:01  IO Worker #1 TRACE sync  Syncing with peers: 7 active, 9 confirmed, 9 total
2018-08-20 19:55:01  IO Worker #0 TRACE sync  Syncing with peers: 6 active, 8 confirmed, 8 total
2018-08-20 19:55:01  IO Worker #2 TRACE sync  Syncing with peers: 8 active, 8 confirmed, 8 total
2018-08-20 19:55:01  IO Worker #2 TRACE sync  Syncing with peers: 8 active, 8 confirmed, 8 total
2018-08-20 19:55:01  IO Worker #1 TRACE sync  Syncing with peers: 5 active, 7 confirmed, 7 total
2018-08-20 19:55:01  IO Worker #0 TRACE sync  Syncing with peers: 4 active, 6 confirmed, 6 total
2018-08-20 19:55:01  IO Worker #1 TRACE sync  Syncing with peers: 3 active, 5 confirmed, 5 total
2018-08-20 19:55:01  IO Worker #0 TRACE sync  Syncing with peers: 2 active, 4 confirmed, 4 total
2018-08-20 19:55:01  IO Worker #3 TRACE sync  Syncing with peers: 2 active, 3 confirmed, 3 total
2018-08-20 19:55:01  IO Worker #2 TRACE sync  Syncing with peers: 2 active, 2 confirmed, 2 total
2018-08-20 19:55:01  IO Worker #2 TRACE sync  Syncing with peers: 1 active, 1 confirmed, 1 total
2018-08-20 19:55:01  IO Worker #2 TRACE sync  Syncing with peers: 0 active, 0 confirmed, 0 total
  • Attempt to import lots of blocks, only import 4096 into queue
2018-08-20 19:55:21  IO Worker #0 TRACE sync  Imported 4099 of 34163
  • Repeatedly requests headers from the same peer, going backwards in steps of 1 * 16, then exponentially back to zero
29414640-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Considering peer 17, force=false, td=Some(0x14c0f895d60dd5e79be), our td=6125438561124561615294, latest=0x1a7b…6a6c, have_latest=true, state=Idle
29414641-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Starting sync with better chain
29414642:2018-08-20 19:55:25  IO Worker #1 TRACE sync  17 <- GetBlockHeaders: 256 entries starting from 0x3d0e…d95f, set = OldBlocks
29414643-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Syncing with peers: 2 active, 4 confirmed, 4 total
29414644-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Skipping busy peer 17
29414645-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Skipping busy peer 8
29414646-2018-08-20 19:55:25  IO Worker #1 DEBUG sync  17 -> Dispatching packet: 4
29414647:2018-08-20 19:55:25  IO Worker #1 TRACE sync  17 -> BlockHeaders (0 entries), state = Idle, set = OldBlocks
29414648-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Imported 0 of 0
29414649-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Sync round complete
--
29414652-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Searching common header from the last round 1258878 (0x014e…fe92)
29414653-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Starting sync with better chain
29414654:2018-08-20 19:55:25  IO Worker #1 TRACE sync  17 <- GetBlockHeaders: 256 entries starting from 0x014e…fe92, set = OldBlocks
29414655-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Syncing with peers: 2 active, 4 confirmed, 4 total
29414656-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Skipping busy peer 17
29414657-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Skipping busy peer 8
29414658-2018-08-20 19:55:25  IO Worker #2 DEBUG sync  17 -> Dispatching packet: 4
29414659:2018-08-20 19:55:25  IO Worker #2 TRACE sync  17 -> BlockHeaders (0 entries), state = Idle, set = OldBlocks
29414660-2018-08-20 19:55:25  IO Worker #2 TRACE sync  Imported 0 of 0
29414661-2018-08-20 19:55:25  IO Worker #2 TRACE sync  Sync round complete
--
29414664-2018-08-20 19:55:25  IO Worker #2 TRACE sync  Searching common header from the last round 1258877 (0x6479…ae9c)
29414665-2018-08-20 19:55:25  IO Worker #2 TRACE sync  Starting sync with better chain
29414666:2018-08-20 19:55:25  IO Worker #2 TRACE sync  17 <- GetBlockHeaders: 256 entries starting from 0x6479…ae9c, set = OldBlocks
29414667-2018-08-20 19:55:25  IO Worker #2 TRACE sync  Syncing with peers: 2 active, 4 confirmed, 4 total
29414668-2018-08-20 19:55:25  IO Worker #2 TRACE sync  Skipping busy peer 17
--
29414672-2018-08-20 19:55:25  IO Worker #1 TRACE sync  == Disconnecting 13: Parity/v1.11.8-stable-c754a02-20180725/x86_64-linux-gnu/rustc1.27.2
29414673-2018-08-20 19:55:25  IO Worker #1 DEBUG sync  17 -> Dispatching packet: 4
29414674:2018-08-20 19:55:25  IO Worker #1 TRACE sync  17 -> BlockHeaders (0 entries), state = Idle, set = OldBlocks
29414675-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Imported 0 of 0
29414676-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Sync round complete
--
29414679-2018-08-20 19:55:25  IO Worker #1 TRACE sync  Searching common header from the last round 1258876 (0x73c4…45e2)


2018-08-20 19:55:26  IO Worker #0 TRACE sync  Searching common header in the blockchain 1258863 (0x1f9c…22f6)
2018-08-20 19:55:26  IO Worker #0 TRACE sync  Searching common header in the blockchain 1258862 (0x1a54…23d8)
2018-08-20 19:55:26  IO Worker #3 TRACE sync  Searching common header in the blockchain 1258860 (0x180a…a789)
2018-08-20 19:55:26  IO Worker #0 TRACE sync  Searching common header in the blockchain 1258856 (0x3f11…b270)
2018-08-20 19:55:26  IO Worker #0 TRACE sync  Searching common header in the blockchain 1258848 (0x64ab…441d)
2018-08-20 19:55:26  IO Worker #0 TRACE sync  Searching common header in the blockchain 1258832 (0x2701…ce20)
2018-08-20 19:55:26  IO Worker #3 TRACE sync  Searching common header in the blockchain 1258800 (0xd0ec…ca03)
2018-08-20 19:55:26  IO Worker #0 TRACE sync  Searching common header in the blockchain 1258736 (0x6cf4…ca33)
2018-08-20 19:55:26  IO Worker #0 TRACE sync  Searching common header in the blockchain 1258608 (0x0076…85ab)
2018-08-20 19:55:26  IO Worker #0 TRACE sync  Searching common header in the blockchain 1258352 (0x7419…62b6)
2018-08-20 19:55:26  IO Worker #3 TRACE sync  Searching common header in the blockchain 1257840 (0xa393…8785)
2018-08-20 19:55:26  IO Worker #3 TRACE sync  Searching common header in the blockchain 1256816 (0x6645…3a57)
2018-08-20 19:55:26  IO Worker #0 TRACE sync  Searching common header in the blockchain 1254768 (0x99ad…54cb)
2018-08-20 19:55:26  IO Worker #0 TRACE sync  Searching common header in the blockchain 1250672 (0xca85…b231)
2018-08-20 19:55:26  IO Worker #3 TRACE sync  Searching common header in the blockchain 1242480 (0x6f3e…9d11)
2018-08-20 19:55:26  IO Worker #1 TRACE sync  Searching common header in the blockchain 1226096 (0x8971…50a4)
2018-08-20 19:55:26  IO Worker #1 TRACE sync  Searching common header in the blockchain 1193328 (0xda85…b19f)
2018-08-20 19:55:26  IO Worker #0 TRACE sync  Searching common header in the blockchain 1127792 (0x48c2…7450)
2018-08-20 19:55:26  IO Worker #2 TRACE sync  Searching common header in the blockchain 996720 (0xaca3…6093)
2018-08-20 19:55:27  IO Worker #1 TRACE sync  Searching common header in the blockchain 734576 (0xc527…0c2e)
2018-08-20 19:55:27  IO Worker #2 TRACE sync  Searching common header in the blockchain 210288 (0xd4e5…8fa3)
2018-08-20 19:56:21  IO Worker #2 TRACE sync  Searching common header in the blockchain 33020 (0x5cc6…c005)
2018-08-20 19:56:21  IO Worker #0 TRACE sync  Searching common header in the blockchain 33019 (0x2956…093a)
2018-08-20 19:56:21  IO Worker #0 TRACE sync  Searching common header in the blockchain 33017 (0xb22b…1a2b)
2018-08-20 19:56:22  IO Worker #2 TRACE sync  Searching common header in the blockchain 33013 (0xebe5…80b3)
2018-08-20 19:56:47  IO Worker #0 TRACE sync  Searching common header in the blockchain 33005 (0xd157…d779)
2018-08-20 19:56:48  IO Worker #3 TRACE sync  Searching common header in the blockchain 32989 (0xb9d8…e6ab)
2018-08-20 19:56:48  IO Worker #3 TRACE sync  Searching common header in the blockchain 32957 (0x7553…5f11)
2018-08-20 19:56:50  IO Worker #2 TRACE sync  Searching common header in the blockchain 32893 (0xcae7…1f5b)
2018-08-20 19:56:50  IO Worker #2 TRACE sync  Searching common header in the blockchain 32765 (0x49e8…c402)
2018-08-20 19:56:51  IO Worker #1 TRACE sync  Searching common header in the blockchain 32509 (0x87e6…36b6)
2018-08-20 19:57:12  IO Worker #2 TRACE sync  Searching common header in the blockchain 31997 (0x46b9…5160)
2018-08-20 19:58:30  IO Worker #1 TRACE sync  Searching common header in the blockchain 30973 (0x27a4…1ba9)
2018-08-20 19:58:54  IO Worker #1 TRACE sync  Searching common header in the blockchain 28925 (0x8465…6988)
2018-08-20 19:58:58  IO Worker #0 TRACE sync  Searching common header in the blockchain 24829 (0x33fc…0cb2)
2018-08-20 19:59:33  IO Worker #0 TRACE sync  Searching common header in the blockchain 16637 (0xf6ba…5489)
2018-08-20 20:01:44  IO Worker #0 TRACE sync  Searching common header in the blockchain 6182737 (0x69b0…6d38)
2018-08-20 20:02:05  IO Worker #3 TRACE sync  Searching common header in the blockchain 253 (0xd4e5…8fa3)

And hey presto we get stuck again as before:

2018-08-20 19:55:07  IO Worker #2 INFO import    #1254780    0/750 peers   6 MiB chain 3 GiB db 0 bytes queue 193 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 19:55:42  IO Worker #0 INFO import       #5715    7/750 peers   5 MiB chain 3 GiB db 0 bytes queue 62 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 19:56:18  IO Worker #3 DEBUG sync  54 -> Dispatching packet: 62018-08-20 19:56:18  IO Worker #2 INFO import      #47184   29/750 peers   5 MiB chain 3 GiB db 0 bytes queue 76 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 19:56:52  IO Worker #2 INFO import      #31997   21/750 peers   6 MiB chain 3 GiB db 0 bytes queue 56 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 19:57:27  IO Worker #2 INFO import      #51935   33/750 peers   6 MiB chain 3 GiB db 0 bytes queue 69 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 19:58:02  IO Worker #0 INFO import      #66914   36/750 peers   6 MiB chain 3 GiB db 0 bytes queue 60 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 19:58:37  IO Worker #1 INFO import      #29179   41/750 peers   6 MiB chain 3 GiB db 0 bytes queue 88 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 19:59:12  IO Worker #2 INFO import      #16764   46/750 peers   6 MiB chain 3 GiB db 0 bytes queue 85 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 19:59:47  IO Worker #1 INFO import       #1269   48/750 peers   6 MiB chain 3 GiB db 0 bytes queue 84 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 20:00:23  IO Worker #3 INFO import      #17658   40/750 peers   6 MiB chain 3 GiB db 0 bytes queue 70 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 20:00:58  IO Worker #2 DEBUG sync  58 -> Dispatching packet: 72018-08-20 20:00:58  IO Worker #3 INFO import      #17658   34/750 peers   6 MiB chain 3 GiB db 0 bytes queue 70 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 20:01:33  IO Worker #2 INFO import      #17658   22/750 peers   5 MiB chain 3 GiB db 0 bytes queue 71 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 20:02:07  IO Worker #0 INFO import        #381   15/750 peers   6 MiB chain 3 GiB db 0 bytes queue 62 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 20:02:42  IO Worker #3 INFO import      #54364   26/750 peers   6 MiB chain 3 GiB db 0 bytes queue 69 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 20:03:17  IO Worker #3 INFO import      #31354   32/750 peers   5 MiB chain 3 GiB db 0 bytes queue 81 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 20:03:53  IO Worker #2 TRACE sync  == Disconnecting 18: Parity/v1.11.7-stable-085035f-20180717/x86_64-linux-gnu/rustc1.27.12018-08-20 20:03:53  IO Worker #0 INFO import      #48864   30/750 peers   5 MiB chain 3 GiB db 0 bytes queue 71 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 20:04:27  IO Worker #3 INFO import      #52569   34/750 peers   6 MiB chain 3 GiB db 0 bytes queue 61 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 20:05:22  IO Worker #2 INFO import         #57   13/750 peers   6 MiB chain 3 GiB db 0 bytes queue 85 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 20:05:57  IO Worker #2 INFO import         #57   10/750 peers   6 MiB chain 3 GiB db 0 bytes queue 85 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 20:06:32  IO Worker #1 INFO import         #57    6/750 peers   6 MiB chain 3 GiB db 0 bytes queue 86 MiB sync  RPC:  0 conn,    0 req/s,  168 µs
2018-08-20 20:07:07  IO Worker #1 INFO import         #57    6/750 peers   6 MiB chain 3 GiB db 0 bytes queue 90 MiB sync  RPC:  0 conn,    0 req/s,  168 µs

@5chdn 5chdn added the P2-asap 🌊 No need to stop dead in your tracks, however issue should be addressed as soon as possible. label Aug 23, 2018
@5chdn
Copy link
Contributor

5chdn commented Aug 23, 2018

Thanks, this is an important task. 👍

@ascjones
Copy link
Contributor Author

Tried running @andresilva's fix #9407. While it no longer retracts, it does get stuck at #1914466, again suspiciously close to the fork block (see #9306 (comment))

@boris-savic boris-savic mentioned this issue Sep 6, 2018
Closed
@5chdn 5chdn modified the milestones: 2.1, 2.2 Sep 11, 2018
@ascjones ascjones mentioned this issue Sep 11, 2018
Merged
@jam10o-new jam10o-new mentioned this issue Sep 21, 2018
Closed
@ascjones ascjones mentioned this issue Oct 15, 2018
Merged
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
F2-bug 🐞 The client fails to follow expected behavior. M4-core ⛓ Core client code / Rust. P2-asap 🌊 No need to stop dead in your tracks, however issue should be addressed as soon as possible.
Projects
None yet
Milestone
2.2
Development

Successfully merging a pull request may close this issue.

Fix ancient blocks sync openethereum/parity-ethereum
2 participants
@ascjones @5chdn