make block queue into a more generic verification queue and fix block heap size calculation #2095
Conversation
rphmeier
added
A0-pleasereview 🤓
|
The latest push implements a header queue, but it will give warnings that the |
|
This also fixes calculation of heap size for blocks in the blocks queue (previously hard-coded to be zero, which was wrong) |
rphmeier
changed the title
|
looks fairly reasonable to me. @arkpar could you take a look? |
|
Doing a full sync with this right now, strangely it seems to have broken the limits on the queue size (or exposed an existing issue) -- I saw 100MiB of blocks (48k) queued at once with the default maximums: 50MiB and 30k blocks. I haven't changed the It might just be that it sees that the block queue isn't quite full and then imports all the blocks it has, bumping it over the limit. |
|
On the plus side, it's definitely queuing way fewer blocks (while syncing ~700k it has typically 15-20k at any given time) while keeping the usual queue size slightly over 50MiB. So the footprint is definitely more in line with the specified options now. |
| match verify_block_unordered(un.header, un.bytes, engine) { | ||
| Ok(verified) => Ok(verified), | ||
| Err(e) => { | ||
| warn!(target: "client", "Stage 2 block verification failed for {}\n Error: {:?}", hash, e); |
There was a problem hiding this comment.
Word "Error" should be removed from this message. It is confusing for users.
| match verify_block_basic(&input.header, &input.bytes, engine) { | ||
| Ok(()) => Ok(input), | ||
| Err(e) => { | ||
| warn!(target: "client", "Stage 1 block verification failed for {:?}", input.hash()); |
There was a problem hiding this comment.
Please add error info to the message
arkpar
added
A6-mustntgrumble 💦
rphmeier
removed
A6-mustntgrumble 💦
|
Changes Unknown when pulling 6abadd6 on header_queue into * on master*. |
* master: Fixing Delegate Call in JIT (#2378) Prioritizing re-imported transactions (#2372) Revert #2172, pretty much. (#2387) correct sync memory usage calculation (#2385) Update gitlab-ci Fix the traceAddress field in transaction traces. (#2373) Removing extras data from retracted blocks. (#2375) fixed #2263, geth keys with ciphertext shorter than 32 bytes (#2318) Expanse compatibility (#2369) Specify column cache sizes explicitly; default fallback of 2MB (#2358) Canonical state cache (master) (#2311) make block queue into a more generic verification queue and fix block heap size calculation (#2095) Hash Content RPC method (#2355) Reorder transaction_by_hash to favour canon search (#2332) DIV optimization (#2327) Error when deserializing invalid hex (#2339) Changed http:// to https:// on some links (#2349) add a test fix migration system, better errors # Conflicts: # .gitlab-ci.yml
in preparation for header-only sync. verification pipelines (currently assumed to be 3-stage) are defined in the
kindsmodule.