Skip to content

Commit

Permalink
add 2024Jan benchmark
Browse files Browse the repository at this point in the history
Signed-off-by: Keming <kemingyang@tensorchord.ai>
  • Loading branch information
kemingy committed Jan 29, 2024
1 parent dc8cd88 commit 94b0005
Show file tree
Hide file tree
Showing 4 changed files with 21 additions and 0 deletions.
21 changes: 21 additions & 0 deletions src/faqs/benchmark.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Benchmark

`pgvecto.rs` has been focused on performance from the beginning. We have continuously monitored the performance of `pgvecto.rs` and compared it with other vector search libraries. The following benchmark results are from January 2024.

The test is done on Google Cloud [n2-standard-8](https://cloud.google.com/compute/docs/general-purpose-machines#n2_series) (8 vCPUs, 32 GB RAM) with the `laion-768-5m-ip` dataset.

## `pgvecto.rs` v.s. `pgvector`

With the HNSW index, `pgvecto.rs` can achieve **2.5x** responses per second as `pgvector` can do with a slightly better precision of around 97%. This advantage increases when higher precision is required.

![pgvecto.rs_vs_pgvector](./images/2024Jan_pgvectors_compare.png)

With the filter enabled, `pgvecto.rs` can always reach more than **2x** responses per second as `pgvector` can do with different filter probabilities.

![pgvecto.rs_vs_pgvector_filter](./images/2024Jan_pgvectors_filter_compare.png)

## `pgvecto.rs` with different quantization methods

`pgvecto.rs` supports several [quantization methods](../usage/quantization.md). These methods can help to reduce memory usage at different scales. The following figure shows the performance of `pgvecto.rs` with different quantization methods.

![pgvecto.rs_quantization](./images/2024Jan_pgvectors_quantization.png)
Binary file added src/faqs/images/2024Jan_pgvectors_compare.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 94b0005

Please sign in to comment.