feat: support sparse vector #299

silver-ymz · 2024-01-23T07:46:49Z

part of #252

Uasge

CREATE TABLE t (val svector(6));

INSERT INTO t (val) SELECT ARRAY[0, random(), 0, 0, random(), random()]::real[]::vector::svector FROM generate_series(1, 1000);

INSERT INTO t (val) SELECT "[0, 1, 1, 0, 0, 0]";

CREATE INDEX ON t USING vectors (val svector_l2_ops)
WITH (options = "[indexing.hnsw]");

CREATE INDEX ON t USING vectors (val svector_dot_ops)
WITH (options = "[indexing.hnsw]");

CREATE INDEX ON t USING vectors (val svector_cos_ops)
WITH (options = "[indexing.ivf]");

Design

Storage

The storage part is in crates/service/src/prelude/storage.

DenseMmap is original RawMmap, sparse storage struct is following

pub struct SparseMmap {
    vectors: MmapArray<SparseF32Element>,
    offsets: MmapArray<u32>,
    payload: MmapArray<Payload>,
    dims: u16,
}

Index

Hnsw doesn't make any modification. For ivf, it expands all vectors to dense vector as the subsamples of kmeans, and all kmeans algorithm is used on the dense vector. After getting centroids, it will calculate other vectors in sparse format.

SQ and PQ aren't implemented now.

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

silver-ymz · 2024-01-23T13:50:13Z

PTAL @VoVAllen @usamoi

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

src/datatype/svecf32.rs

crates/service/src/algorithms/flat.rs

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

src/datatype/svecf32.rs

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

crates/service/src/prelude/global/mod.rs

crates/service/src/prelude/scalar/sparse_f32.rs

crates/service/src/prelude/storage/mod.rs

src/datatype/svecf32.rs

crates/service/src/algorithms/quantization/product.rs

src/prelude/error.rs

src/datatype/svecf32.rs

crates/service/src/prelude/storage/sparse.rs

crates/service/src/algorithms/quantization/scalar.rs

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

usamoi · 2024-02-02T07:02:47Z

#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Serialize, Deserialize)]
pub struct SparseF32Element {
    pub index: u32,
    pub value: F32,
}

It'll be better if we use two arrays (array of index: u16 and value: F32). We'll save 25% memory on vectors.

VoVAllen · 2024-02-02T07:26:58Z

Probably need some experiments to see the best calculation method/storage layout. In pyanns, it stored the data along with the indices like [1, 0.1, 2, 0.1]. Therefore the read is totally sequential. Seperate them needs a seperate random access.

VoVAllen · 2024-02-02T07:27:45Z

Also there should be chances to use simd to optimize the computation. if statement inside calculation should be much slower

usamoi · 2024-02-02T07:30:38Z

Seperate them needs a seperate random access

We will read both two arrays at the same time so it's still sequential.

there should be chances to use simd to optimize the computation

This layout should be nice for SIMD.

VoVAllen · 2024-02-02T09:36:24Z

I'm actually thinking of bitset like encoding for indices such as roaringbitmap. Then you can get the intersection of the set easily, and then select out the value to compute. Not sure how much improvement we can have here. Is it worth trying?

usamoi · 2024-02-02T09:39:36Z

I'm actually thinking of bitset like encoding for indices such as roaringbitmap. Then you can get the intersection of the set easily, and then select out the value to compute. Not sure how much improvement we can have here. Is it worth trying?

Roaring bitmap is designed for a large data. How could we take advantage of it?

VoVAllen · 2024-02-05T15:32:18Z

Do we need to store the vector norm for L2/cos distance in sparse vector? @usamoi

VoVAllen · 2024-02-05T15:33:16Z

We can handle the branchless/SIMD optimization in other PR since it doesn't affect internal layout

VoVAllen · 2024-02-05T16:21:19Z

I just feel bitset-like structure is easier to use SIMD. Find this might be useful as a referenece https://github.com/UNSW-database/simd_set_operations , with his thesis including thorough analysis

After reading the thesis, I think our final target should be https://github.com/UNSW-database/simd_set_operations/blob/main/setops/src/intersect/broadcast.rs#L489-L561. On AVX512 machine, it's pretty competitive with the array representation.

usamoi · 2024-02-06T09:01:15Z

Do we have the reference of format for svector_from_kv_string?

usamoi · 2024-02-06T09:02:15Z

src/datatype/svecf32.rs

+        let values_ptr = pgrx::pg_sys::pq_getmsgbytes(buf, values_bytes as _);
+        let mut output = SVecf32::new_zeroed_in_postgres(len as usize);
+        output.dims = dims;
+        std::ptr::copy(indexes_ptr, output.indexes_mut() as _, indexes_bytes);


Let's detect possible data corruption for indexes.

done in new commit

usamoi · 2024-02-06T09:03:12Z

src/datatype/svecf32.rs

+            ptr.add(offset).cast()
+        }
+    }
+    fn indexes_mut(&mut self) -> *mut u16 {


_vectors_svecf32_recv needs it. I updated the returned value from a pointer to a reference.

usamoi · 2024-02-06T09:03:18Z

src/datatype/svecf32.rs

+    fn indexes_mut(&mut self) -> *mut u16 {
+        self.phantom.as_mut_ptr().cast()
+    }
+    fn values_mut(&mut self) -> *mut F32 {


_vectors_svecf32_recv needs it. I updated the returned value from a pointer to a reference.

usamoi · 2024-02-06T09:04:11Z

src/datatype/svecf32.rs

+    fn indexes(&self) -> *const u16 {
+        self.phantom.as_ptr().cast()
+    }
+    fn values(&self) -> *const F32 {


Why not return a reference?

done in new commit

usamoi · 2024-02-06T09:04:15Z

src/datatype/svecf32.rs

+    pub fn len(&self) -> usize {
+        self.len as usize
+    }
+    fn indexes(&self) -> *const u16 {


Why not return a reference?

done in new commit

usamoi · 2024-02-06T09:05:01Z

src/datatype/svecf32.rs

+        }
+        .friendly();
+    }
+    SVecf32::new_in_postgres(SparseF32Ref {


We need to check if the dimensions are in [1, 65535].

done in new commit

usamoi · 2024-02-06T09:06:50Z

src/prelude/error.rs

@@ -70,6 +70,14 @@ INFORMATION: left_dimensions = {left_dimensions}, right_dimensions = {right_dime
        left_dimensions: u16,
        right_dimensions: u16,
    },
+    #[error("\


Use BadLiteral.

done in new commit

usamoi · 2024-02-06T09:13:18Z

src/datatype/svecf32.rs

+#[repr(C, align(8))]
+pub struct SVecf32 {
+    varlena: u32,
+    len: u16,


Can you swap the position of len and dims?
I'm considering removing from_datum trick, and dims is a more meaningful information if we choose one to be in header.

done in new commit

usamoi · 2024-02-06T09:15:05Z

crates/service/src/index/indexing/mod.rs

+        }
+    }
+
+    pub fn dims(&self) -> u16 {


We do not need it, do we?

removed in new commit

usamoi · 2024-02-06T09:15:54Z

crates/service/src/index/indexing/mod.rs

@@ -130,11 +161,11 @@ impl<S: G> DynamicIndexing<S> {
        }
    }

-    pub fn vector(&self, i: u32) -> &[S::Scalar] {
+    pub fn content(&self, i: u32) -> S::VectorRef<'_> {


Let's keep the name of vector and open. Afterall we already use the two names everywhere.

done in new commit

VoVAllen · 2024-02-06T09:35:36Z

Do we have the reference of format for svector_from_kv_string?

I don't think we need from kv string capability now. User can easily transform kv to the array in any language. The kv format I think is rare

VoVAllen · 2024-02-06T09:36:09Z

And I think we don't need to worry about the sparse computation now. We can always improve it later

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

src/sql/finalize.sql

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

usamoi · 2024-02-18T07:14:16Z

crates/service/src/prelude/scalar/sparse_f32.rs

+use std::fmt::Display;
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
+pub struct SparseF32Element {


Will remove it.

usamoi · 2024-02-18T07:51:05Z

Please fix the implementation of comparsion operators.

silver-ymz added 5 commits January 23, 2024 15:44

basic support

2fd58c0

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

cargo clippy

76508cd

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

optimize for ivf

07f940c

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

Merge branch 'main' into feat/sparse

209e5d8

patch for merge

7b883fe

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

silver-ymz marked this pull request as ready for review January 23, 2024 13:46

silver-ymz changed the title ~~[WIP] feat: support sparse vector~~ feat: support sparse vector Jan 23, 2024

set vector cast to non-implicit

351f8fe

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

silver-ymz mentioned this pull request Jan 23, 2024

doc: sparse vector tensorchord/pgvecto.rs-docs#34

Merged

usamoi reviewed Jan 24, 2024

View reviewed changes

src/datatype/svecf32.rs Show resolved Hide resolved

usamoi reviewed Jan 24, 2024

View reviewed changes

crates/service/src/algorithms/flat.rs Outdated Show resolved Hide resolved

add dimension in sparse vector

62dce6a

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

silver-ymz changed the base branch from main to 0.3 January 29, 2024 07:36

usamoi reviewed Jan 29, 2024

View reviewed changes

src/datatype/svecf32.rs Outdated Show resolved Hide resolved

silver-ymz added 3 commits January 29, 2024 22:43

support creating sparse vector from spilt array and kv string

1c1e411

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

Merge branch '0.3' into feat/sparse

96786b5

cargo clippy

822e3ca

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

silver-ymz requested a review from usamoi January 29, 2024 14:56

silver-ymz changed the base branch from 0.3 to main January 29, 2024 14:57

Merge branch 'main' into feat/sparse

df8a9a7

usamoi reviewed Jan 31, 2024

View reviewed changes

silver-ymz added 2 commits February 2, 2024 14:32

move vector type from Storage trait to G trait

d23f32b

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

merge main into sparse

001cbd7

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

usamoi mentioned this pull request Feb 6, 2024

Tracking issue for schema #255

Open

14 tasks

usamoi reviewed Feb 6, 2024

View reviewed changes

silver-ymz added 3 commits February 7, 2024 18:41

refactor

9105277

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

merge main into sparse

567d511

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

cargo fmt

128b7f4

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

VoVAllen reviewed Feb 8, 2024

View reviewed changes

src/sql/finalize.sql Outdated Show resolved Hide resolved

update to_svector arguments name

3e55f63

Signed-off-by: Mingzhuo Yin <yinmingzhuo@gmail.com>

silver-ymz force-pushed the feat/sparse branch from 3c56ad3 to 3e55f63 Compare February 8, 2024 09:47

usamoi reviewed Feb 18, 2024

View reviewed changes

usamoi approved these changes Feb 18, 2024

View reviewed changes

usamoi added this pull request to the merge queue Feb 18, 2024

Merged via the queue into tensorchord:main with commit d7a490c Feb 18, 2024
7 checks passed

silver-ymz deleted the feat/sparse branch February 18, 2024 08:17

feat: support sparse vector #299

feat: support sparse vector #299

Conversation

silver-ymz commented Jan 23, 2024 • edited Loading

Uasge

Design

Storage

Index

silver-ymz commented Jan 23, 2024

usamoi commented Feb 2, 2024

VoVAllen commented Feb 2, 2024

VoVAllen commented Feb 2, 2024

usamoi commented Feb 2, 2024

VoVAllen commented Feb 2, 2024

usamoi commented Feb 2, 2024

VoVAllen commented Feb 5, 2024

VoVAllen commented Feb 5, 2024

VoVAllen commented Feb 5, 2024

usamoi commented Feb 6, 2024

usamoi Feb 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

silver-ymz Feb 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

usamoi Feb 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VoVAllen commented Feb 6, 2024

VoVAllen commented Feb 6, 2024

Choose a reason for hiding this comment

usamoi commented Feb 18, 2024

silver-ymz commented Jan 23, 2024 •

edited

Loading

usamoi Feb 6, 2024 •

edited

Loading

silver-ymz Feb 7, 2024 •

edited

Loading

usamoi Feb 6, 2024 •

edited

Loading