-
Notifications
You must be signed in to change notification settings - Fork 57
SplinterDB Integration Investigations
This work was performed in Fall 2022. Following is the Abstract of a paper submitted to VLDB 2023 Industrial Track, but was not accepted for publication. However, the results are encouraging.
Relational databases make extensive use of B-tree indexes to accelerate query performance. B-Trees offer excellent point-query performance but inserting data into a large B-tree can be costly, especially for a workload of random inserts. In fact, high-velocity tables are often not indexed at all due to the performance limitations of B-tree insertions.
This paper describes how SplinterDB, a novel, open-source, write-optimized key-value store developed at VMware, can be used as an index within a relational database, such as PostgreSQL, to increase ingestion speed without sacrificing query performance. In our performance evaluation, we show that a SplinterDBindexed table can ingest data up to 24× faster than a B-tree-indexed table. In fact, SplinterDB ingests data half as fast as a table with no index at all, whereas B-tree indexes can impose a 36× slowdown compared to an unindexed table. Furthermore, SplinterDB matched the point-query performance of Postgres’s B-tree and outperformed the builtin B-tree by 50% on large-range queries, although it was about 3× slower on short-range queries.
Our results suggest that a SplinterDB-based index could become a compelling alternative to B-tree indexes for supporting higher ingestion rates like those needed for real-time applications in relational databases like PostgreSQL.
The full technical paper can be found here, SplinterDB-Postgres-vFinal.pdf