Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deletions scale poorly #356

Closed
cberner opened this issue Sep 11, 2022 · 1 comment
Closed

Deletions scale poorly #356

cberner opened this issue Sep 11, 2022 · 1 comment

Comments

@cberner
Copy link
Owner

cberner commented Sep 11, 2022

Deletions should be about the same speed as insertions. However they seem to scale poorly. See the benchmarking done in #344 for more details

@cberner
Copy link
Owner Author

cberner commented Sep 19, 2022

Actually, I think this is just the effects of the database file becoming larger than memory, and the deletions being the last part of the benchmark and hence starting from a full database.

I did additional benchmarks each starting with 16M insertions, and then deletions of that data or insertions of more data. The results show that deletions are slightly slower than insertions, but scale as expected:

Delete:

10k:
redb: Removed 10000 items in 5694ms

100k:
redb: Removed 100000 items in 31043ms

1M:
redb: Removed 1000000 items in 166154ms

2M:
redb: Removed 2000000 items in 343368ms

4M:
redb: Removed 4000000 items in 756057ms

8M:
redb: Removed 8000000 items in 1368607ms

16M:
redb: Removed 16000000 items in 1686246ms


Additional inserts (with seed = 37347842):

10k:
redb: Bulk loaded 10000 items in 4158ms

100k:
redb: Bulk loaded 100000 items in 26200ms

1M:
redb: Bulk loaded 1000000 items in 124340ms

2M:
redb: Bulk loaded 2000000 items in 250383ms

4M:
redb: Bulk loaded 4000000 items in 594556ms

8M:
redb: Bulk loaded 8000000 items in 1390448ms

16M:
redb: Bulk loaded 16000000 items in 3648112ms

@cberner cberner closed this as completed Sep 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant