Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pending blocks ops #2886

Merged
merged 67 commits into from
Dec 31, 2019
Merged

pending blocks ops #2886

merged 67 commits into from
Dec 31, 2019

Conversation

ayrat555
Copy link
Contributor

The rebased version of #2819

Motivation

A lot of problems are created by tracking the importing progress on multiple tables.
Querying and updating involves as such heavy operations and this is also the cause for inconsistencies.

The proposed solution is to use a new table to track the progress of block's data indexing.
By using a specific table, whose rows are deleted when there are no more pending operations on the relative block, we make joins quick and keep them to a minimum.
This is also meant to simplify all inserts and updates and only perform deletes when strictly necessary.

pasqu4le and others added 29 commits November 28, 2019 15:58
Problem: a lot of problems are created by tracking the importing progress on multiple tables.
Querying and updating involves as such heavy operations and this is also the cause for inconsistencies.

Solution: use a new table to track the progress of block's data indexing.
By using a specific table, whose rows are deleted when there are no more pending operations on the relative block, we make joins quick and keep them to a minimum.
This is also meant to simplify all inserts and updates and only perform deletes when strictly necessary.
Problem: a lot of problems are created by tracking the importing progress on multiple tables.
Querying and updating involves as such heavy operations and this is also the cause for inconsistencies.

Solution: use a new table to track the progress of block's data indexing.
By using a specific table, whose rows are deleted when there are no more pending operations on the relative block, we make joins quick and keep them to a minimum.
This is also meant to simplify all inserts and updates and only perform deletes when strictly necessary.
@coveralls
Copy link

coveralls commented Nov 28, 2019

Pull Request Test Coverage Report for Build 4d96dd8c-7568-4c8d-9edb-3bf9ee38f98f

  • 114 of 128 (89.06%) changed or added relevant lines in 18 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.3%) to 75.229%

Changes Missing Coverage Covered Lines Changed/Added Lines %
apps/explorer/lib/explorer/chain/internal_transaction.ex 7 8 87.5%
apps/explorer/lib/explorer/chain/import/runner/internal_transactions.ex 45 48 93.75%
apps/explorer/lib/explorer/chain/pending_block_operation.ex 5 9 55.56%
apps/indexer/lib/indexer/fetcher/internal_transaction.ex 13 19 68.42%
Files with Coverage Reduction New Missed Lines %
apps/explorer/lib/explorer/chain/cache/transaction_count.ex 2 63.64%
Totals Coverage Status
Change from base Build 417aecc0-4a59-4ae6-9f89-9397232b4d9e: 0.3%
Covered Lines: 5333
Relevant Lines: 7089

💛 - Coveralls

…lidation

remove pending ops on invalidation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants