WS Subscriptions Missing Events and Block Headers

[calebrate]

• When subscribed to new headers and events via ws api, parity will push incomplete event sequence and also headers and events are coming out of order.
• Chain reorg block headers are not double pushed (replacement header missing).
• Imported X + another X blocks will push in sequence (X, X-1, X-2...) instead of (...X-2, X-1, X)
• Events from "synced" blocks between "imported" ones, in case parity missed some, are not pushed at all

This is a completely unreliable behaviour for checking for events and removals.

I'm running:

• Which Parity version?: v1.9.7-stable
• Which operating system?: Windows
• How installed?: via installer
• Are you fully synchronized?: yes
• Which network are you connected to?: ethereum
• Did you try to restart the node?: yes

[calebrate]

For the example Parity output:

2018-04-29 12:13:34  Imported #5525801 c392…a35b (272 txs, 7.83 Mgas, 1989.34 ms, 35.09 KiB)
2018-04-29 12:13:34    #5030334   13/25 peers    280 KiB chain  101 MiB db  0 bytes queue   31 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:13:34    #5030334   13/25 peers    280 KiB chain  101 MiB db  0 bytes queue   31 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:14:10    #5031136    5/25 peers      4 KiB chain  101 MiB db  0 bytes queue   31 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:14:42    #5032033   16/25 peers      5 MiB chain  101 MiB db  0 bytes queue   37 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:15:22    #5034438   18/25 peers      2 MiB chain  101 MiB db  0 bytes queue   64 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:15:56    #5037228   13/25 peers      4 KiB chain  101 MiB db  0 bytes queue   10 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:16:27    #5037228   17/25 peers      4 KiB chain  101 MiB db  0 bytes queue   46 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:17:02    #5037228   15/25 peers      4 KiB chain  101 MiB db  0 bytes queue   60 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:17:12  Syncing #5525801 c392…a35b     0 blk/s    0 tx/s   0 Mgas/s      0+    0 Qed  #5525801   18/25 peers      4 KiB chain  101 MiB db  0 bytes queue   60 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:17:22  Syncing #5525810 34ef…e56e     0 blk/s  112 tx/s   6 Mgas/s      0+    5 Qed  #5525816   21/25 peers    484 KiB chain  102 MiB db  599 KiB queue   64 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:17:35  Imported #5525816 01bb…9f49 (152 txs, 7.88 Mgas, 898.11 ms, 26.30 KiB)
2018-04-29 12:17:42  Imported #5525817 94ad…564e (75 txs, 7.99 Mgas, 435.55 ms, 34.77 KiB)
2018-04-29 12:17:59    #5037228   16/25 peers    761 KiB chain  101 MiB db  0 bytes queue   82 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:18:01    #5037612   16/25 peers    761 KiB chain  101 MiB db  0 bytes queue   73 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:18:01  Imported #5525818 fd87…4d4e (70 txs, 7.64 Mgas, 577.83 ms, 32.63 KiB)
2018-04-29 12:18:26  Imported #5525819 c889…5352 (290 txs, 7.97 Mgas, 1201.62 ms, 36.39 KiB)
2018-04-29 12:18:38    #5039528   12/25 peers    854 KiB chain  101 MiB db  0 bytes queue   43 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:18:39    #5039528    3/25 peers    854 KiB chain  101 MiB db  0 bytes queue   43 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:19:12    #5040042    8/25 peers      1 MiB chain  101 MiB db  0 bytes queue   42 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:19:51    #5040170   10/25 peers      2 MiB chain  101 MiB db  0 bytes queue   46 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:20:24    #5041195   13/25 peers      5 MiB chain  101 MiB db  0 bytes queue   45 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:21:18    #5044594   21/25 peers      5 MiB chain  101 MiB db  0 bytes queue   36 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:21:18    #5044594   21/25 peers      5 MiB chain  101 MiB db  0 bytes queue   36 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:21:18    #5044594   21/25 peers      5 MiB chain  101 MiB db  0 bytes queue   36 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:21:18    #5044594   21/25 peers      5 MiB chain  101 MiB db  0 bytes queue   36 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:21:27  Syncing #5525821 98c2…513b     0 blk/s   37 tx/s   1 Mgas/s      0+    8 Qed  #5525831   12/25 peers      5 MiB chain  102 MiB db    1 MiB queue   45 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:21:46  Imported #5525832 253b…56c6 (105 txs, 7.99 Mgas, 750.95 ms, 18.73 KiB)
2018-04-29 12:21:57    #5045358   19/25 peers    208 KiB chain  102 MiB db  0 bytes queue   40 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:22:28  Imported #5525837 db73…3014 (69 txs, 3.23 Mgas, 1178.84 ms, 11.48 KiB) + another 3 block(s) containing 243 tx(s)
2018-04-29 12:22:32    #5045358    4/25 peers    318 KiB chain  100 MiB db  0 bytes queue   53 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:22:40  Imported #5525839 3ba5…c11b (87 txs, 4.30 Mgas, 1270.41 ms, 16.17 KiB) + another 1 block(s) containing 348 tx(s)
2018-04-29 12:22:59  Imported #5525840 4023…14bc (64 txs, 2.66 Mgas, 187.73 ms, 10.45 KiB)
2018-04-29 12:23:07    #5045358   12/25 peers    425 KiB chain   99 MiB db  0 bytes queue   56 MiB sync  RPC:  1 conn,  0 req/s,  37 µs
2018-04-29 12:23:54  Imported #5525844 d17b…caf2 (86 txs, 4.09 Mgas, 1305.36 ms, 14.38 KiB) + another 2 block(s) containing 437 tx(s)

The output of

web3.eth.subscribe('newBlockHeaders').on('data',function(header){
    console.log(header.number)
})

is:

5525801
5525816
5525817
5525818
5525819
5525832
5525834
5525837
5525835
5525836
5525839
5525838
5525840
5525842
5525843
5525844

(Many missing and some out of order)

[zhjgit]

I'm currently experiencing the same problem, subscribing to a newBlockHeaders event, but when my node imports a block, it's always lost. What's the solution?

[Tbaut]

Please open a dedicated issue with example for us to reproduce.