Skip to content

Commit

Permalink
sql backend w/ receipt log indexing (#883)
Browse files Browse the repository at this point in the history
* Receipt log sql indexer

* Wip unify worker and backend

* Wip lay out service

* Remove old crate

* mapping sync works (wip genesis)

* Genesis block indexing

* General rpc update to async

* WIP filter logs sql

* Fix mapping-sync tests

* WIP filter logs sql

* Test filter_logs

* Fix tests

* Make it build and pass tests again

* Use sql backend in rpc

* Run ts tests with sql + ci

* Fix key-value

* editorconfig

* taplo

* clippy

* Always fire interval first

* Try move indexing

* Always fire interval first

* await on logs task

* wal journal mode

* Additional logging

* Add pragmas + optimize vec insert

* More logs

* More logs

* Move block metadata state read to blocking thread

# Conflicts:
#	client/db/src/sql/mod.rs

* Tweak filter_logs query

* Update indexes

* Additional metadata logging

* Fix trailing wildcard bug + add more tests

# Conflicts:
#	client/db/src/sql/mod.rs

* Improve topic input sanitization

* Move `block_number` column from `logs` to `blocks`

# Conflicts:
#	client/db/src/sql/mod.rs

* Add and handle `is_canon` column

# Conflicts:
#	client/db/src/sql/mod.rs

* Use ethereum digest in sync worker tests

# Conflicts:
#	client/mapping-sync/Cargo.toml

* Proper way of handling canonicalization + tests

# Conflicts:
#	client/db/src/sql/mod.rs

* Add progress handler timeout for sqlx queries

# Conflicts:
#	template/node/src/cli.rs

* improve query building

* add cli params to tweak sql, fixes non canon import bug and sql query

# Conflicts:
#	client/db/src/sql/mod.rs
#	client/mapping-sync/src/sql/mod.rs
#	template/node/src/cli.rs

* add tests, fix redundant cloning

# Conflicts:
#	Cargo.lock
#	client/db/Cargo.toml

* improve docs and tests

* fix gaps in sync

# Conflicts:
#	client/mapping-sync/src/sql/mod.rs

* Fix tests

* formatting

* editorconfig

* fmt

* add sql backend pool size config

* Tmp disable threads pragma

* Revert "Tmp disable threads pragma"

This reverts commit abaeb11.

* Try CI fix, low cache, pool size and greater timeout

* revert timeout param

* add support for purge, move db in sql directory

* bump

* fix resume indexing, create indices at start, use spawn_blocking

* simplify num_ops_timeout type, add docs

* use channles for indexing operations

* fix bugs, loop for check missing blocks, set canon query

* cleanup, add docs

* fix filter query

* insert null for missing topics, check unindexed blocks

* fix tests, add worker config

* merge with v0.9.38

* fmt

* fmt

* fmt

* clippy

* clippy

* cleanup

* use upstream sqlx

* lint

* remove nightly feature

* apply code suggestions

* rebase

* disallow SyncStrategy::Normal for the time being

* emit notifications from sql backend

* update tests

---------

Co-authored-by: Nisheeth Barthwal <nbaztec@gmail.com>
  • Loading branch information
tgmichel and nbaztec authored May 31, 2023
1 parent a81265d commit aa57359
Show file tree
Hide file tree
Showing 36 changed files with 5,029 additions and 848 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@ jobs:
run: cd ts-tests && npm run fmt-check
- name: Run functional tests
run: cd ts-tests && npm run build && npm run test
- name: Run functional tests (sql)
run: cd ts-tests && npm run build && npm run test-sql

lint:
name: 'Run lints'
Expand Down
Loading

0 comments on commit aa57359

Please sign in to comment.