Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support concurrent writes to disjoint tables in the same transaction #112

Closed
cberner opened this issue Dec 30, 2021 · 2 comments · Fixed by #523
Closed

Support concurrent writes to disjoint tables in the same transaction #112

cberner opened this issue Dec 30, 2021 · 2 comments · Fixed by #523

Comments

@cberner
Copy link
Owner

cberner commented Dec 30, 2021

No description provided.

@cberner
Copy link
Owner Author

cberner commented Jan 15, 2022

This does not look promising from benchmarking two concurrent database writes. Running the lmdb_benchmark script in two threads leads to:

redb: Bulk loaded 100000 items in 964ms
redb: Bulk loaded 100000 items in 970ms
redb: Wrote 100 individual items in 1235ms
redb: Wrote 100 individual items in 1236ms
redb: Wrote 100 x 1000 items in 6547ms
redb: Wrote 100 x 1000 items in 6547ms
redb: Random read 100000 items in 60ms
redb: Random read 100000 items in 61ms
redb: Random read 100000 items in 56ms
redb: Random read 100000 items in 56ms
redb: Random read 100000 items in 56ms
redb: Random read 100000 items in 56ms
redb: Random range read 100000 starts in 64ms
redb: Random range read 100000 starts in 64ms
redb: Random range read 100000 starts in 65ms
redb: Random range read 100000 starts in 64ms
redb: Random range read 100000 starts in 64ms
redb: Random range read 100000 starts in 64ms
redb: Removed 50000 items in 415ms
redb: Removed 50000 items in 418ms

vs a single thread:

redb: Bulk loaded 100000 items in 457ms
redb: Wrote 100 individual items in 776ms
redb: Wrote 100 x 1000 items in 4139ms
redb: Random read 100000 items in 60ms
redb: Random read 100000 items in 54ms
redb: Random read 100000 items in 54ms
redb: Random range read 100000 starts in 63ms
redb: Random range read 100000 starts in 63ms
redb: Random range read 100000 starts in 63ms
redb: Removed 50000 items in 257ms

showing no gain for bulk inserts, and only a ~1.5x throughput improvement for fsync bound workloads

@cberner cberner closed this as completed Jan 15, 2022
@cberner cberner reopened this Sep 11, 2022
@cberner
Copy link
Owner Author

cberner commented Sep 11, 2022

Seems worth taking another look at this given the low throughput findings for large databases in #344

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant