[Feature]: Multiple Vector Search

[joshuayyy]

Is there an existing issue for this?

• I have searched the existing issues

Is your feature request related to a problem? Please describe.

I have been using your product for quite some time now and I must say, it's truly a fantastic product. However, I've encountered a problem that led me to propose a feature request.

Firstly, at present, Milvus only supports single vector searches. This restricts its application in my use case, where each image is associated with two different vectors. In my database, each image is represented by two vectors: a semantic vector of the image and a descriptive vector derived from the image's title. Both vectors are generated using CLIP. I want to search using an image's semantic vector, but I would like this input vector to be compared against both the semantic and descriptive vectors of the images in the database. The current functionality doesn't allow a single input vector to be compared with multiple vectors of the same entity in the database.

Secondly, I am looking to perform a search operation involving multiple images. For instance, I may want to search for images that are similar to both image A and image B. However, the current system does not support such multi-image search operations.

Describe the solution you'd like.

I propose two enhancements to the existing Milvus system:

Extend the current search functionality to accommodate multiple vector comparisons for a single entity. Specifically, it would mean the capability to compare an input semantic vector with both the semantic and descriptive vectors of images in the database. Furthermore, to mitigate the inherent semantic gap between images and their textual descriptions, the results from the descriptive vectors should be given appropriate weightage when the input is a semantic vector.

Introduce the capability to conduct multi-image searches. This would enable users to input multiple semantic vectors and search for images similar to all the input images simultaneously. Furthermore, it would be ideal to allow adjustments of weights for each input image, to control the influence of each image in the search results.

Describe an alternate solution.

No response

Anything else? (Additional Context)

...

[xiaofan-luan]

/assign @xige-16
/assign @czs007

for your reference

[xiaofan-luan]

multi vector is on the roadmap of Milvus 2.4
including features like

1. build index on multiple index
2. handoff with multiple index
3. search with vector field specified
4. ranking between multiple search result

[CitronBaBa]

Is there a documentation for this feature? As I understand it's searching two vector fields, the current hybrid search doc is about vector search with scalar field filtering, https://milvus.io/docs/hybridsearch.md#Conduct-a-hybrid-vector-search.

[xige-16]

Is there a documentation for this feature? As I understand it's searching two vector fields, the current hybrid search doc is about vector search with scalar field filtering, https://milvus.io/docs/hybridsearch.md#Conduct-a-hybrid-vector-search.

To be precise, this should be called a filtered search. Documentation for hybrid search, which supports multi-vector queries, will be available soon. @CitronBaBa

[ssling0817]

Hello, I have updated the pymilvus to version 2.4.1 and tried to connect to an existing milvus server, but the error message is still there. Does the server also need to be updated? Or updating code is enough. Thanks!

MilvusException: <MilvusException: (code=65535, message=multiple vector fields is not supported, fields name: summary_embeddings, description_embeddings)>