don't run filtered tmp processor when reingestion is enabled #3
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
when a reingestion range is being run, the processor runner was still running
TransactionFilteredTmpProcessor
which is inserting and deleting tx's tohistory_transactions_filtered_tmp
on every flush. This step is un-necessary during reingestion since TransactionFilteredTmpProcessor` only needs to be maintained with transactions from 'forward' ingestion to support tx sub queue processing.In addtion, during some reingestion runs with parallel workers, observed some long running/hanging DELETE stmts left on the db server which were slowing down overall reingestion workers.
this behavior may be related to the deadlocks situation that was noted and resolved by stellar#5263.
This pr proposes additional changes related to reingestion sql performance on top of the deadlock fix in stellar#5263:
EnableIngestionFiltering
flag and global config variable, it is obsolete, as ingestion filter was permanently enabled, the flag provides no value.TransactionFilteredTmpProcessor
to referenceReingestionEnabled
instead ofEnableIngestionFiltering
, this allows the parallel reingestion workers to skip doing that, thus avoiding that additional sql activity