-
Notifications
You must be signed in to change notification settings - Fork 246
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix flaky tests #1570
Fix flaky tests #1570
Conversation
As we have set ban_time to 0, explicitly wait for 1ms to ensure bans will be removed in the later call of clear_stale Signed-off-by: linning <linningde25@gmail.com>
We only keep track of the state of the last K blocks, if there are more than K blocks left to process we can skip some blocks as their result will simply be dumpped after processing, this also help to prevent the bundle validator from stuck on some pruned blocks because these block will be skip as the chain grows. Signed-off-by: linning <linningde25@gmail.com>
In CI confirmation depth is 5, but in practice none of the blocks should be pruned because they will not be deep enough after changes #1541, none of the blocks should be finalized and pruned unless you have a few gigabytes worth of blockchain history. |
There are some differences between finality and canonicalization: and the blocks of non-canonical branches may get pruned upon canonicalization: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The reasoning of banned tx makes sense to me. I'm not pretty sure about the second commit, but the code change is fine. BTW, I think we can at least improve the log to include the block number so that next time we can have more context to investigate.
Signed-off-by: linning <linningde25@gmail.com>
38148f9
to
af29acd
Compare
This PR tries to fix some flaky tests:
#1399
I have dug into prune_known() and seems what it does is indeed what we expected, and I have another guess about the failure:
https://github.com/subspace/subspace/blob/317f83a58670e51298703d9e480a004f6986df5b/domains/client/domain-executor/src/tests.rs#L917-L920
namely, after the block that contains the
submit_bundle
tx is imported and before the call ofprune_tx_from_pool
, the background workertxpool-notifications
handled the imported block, which will ban all the tx of the block, thus the tx is already banned before we callingprune_tx_from_pool
.the solution is to wait 1ms before calling
clear_stale
inprune_tx_from_pool
, because we have already setban_time
to 0, wait a few more times will ensure the bans will be removed inclear_stale
as the ban time must be passed.#1219
This PR is not a complete fix for this issue as we still not figure out why the last K blocks will be pruned such fast (may be related to configuration, local storage of the node, etc).
But this PR should workaround most of the damage as the pruned block will be skipped as the chain grows, and things will back to normal.
Code contributor checklist: