v2.5.0

draft

What's Changed

• update raft to 24.10 by @Presburger in #914
• Fix group search by @alexanderguzhva in #913
• fix : add config check for open and closed range settings by @foxspy in #915
• fix: knowhere ut bugfix by @foxspy in #918
• raft index supports cosine similarity by normalizing the input data. by @Presburger in #924
• compensate for the missing acceleration functions in ARM NEON. by @Presburger in #922
• Allows FAISS_HNSW_FLAT index to load & search on hnswlib flat index by @alexanderguzhva in #929
• enhance : use faiss hnsw implementation to replace hnswlib by @foxspy in #930
• enable refine for FAISS HNSW indices by default, if available by @alexanderguzhva in #932
• fix: extend ut time and fix some compile warnings by @foxspy in #931
• improve sparse vector index mmap: to mmap almost everything by @zhengbuqian in #928
• fix: update faiss hnsw unit test for compatibility by @foxspy in #934
• Add SparseInvertedIndexNodeCC by @zhengbuqian in #933
• sparse mmap mode to use a disk file as backed mmap mmeory by @zhengbuqian in #935
• fix: add hnsw pq/prq refine check by @foxspy in #936
• use MAP_PRIVATE for mmapped file by @zhengbuqian in #938
• fix : fix code checker error in milvus pipeline by @foxspy in #939
• enhance : optimize config check error msg by @foxspy in #940
• enhance: BF supports ids of base_data starting from a specific value by @cqy123456 in #941
• sparse RangeSearch/AnnIterator to return raw distance by @zhengbuqian in #944
• fix : optimize faiss hnsw refine check error msg by @foxspy in #945
• fix : update knowhere index version by @foxspy in #947
• enhance : update vector index defeault value by @foxspy in #946
• fix: BitsetViewSelectorHelper miss id_offset by @cqy123456 in #948
• Add optimized distiance functions for PowerPC by @carll99 in #894
• enhance: optimize get norms function with simd and reduce some calculations by @cqy123456 in #950

New Contributors

• @carll99 made their first contribution in #894

Full Changelog: v2.3.12...v2.5.0

v2.3.12

Features

• async build support for cardinal by @Presburger in #873
• provide a static checking method for index config by @foxspy in #888

Improvements

• add a bit-based structure to mark index-specific features by @foxspy in #887

Bug Fixes

• fix: copy the query vector to resolve the use-after-free issue for ivf-iterator by @alwayslove2013 in #901

Full Changelog: v2.3.11...v2.3.12

v2.3.11

What's Changed

• FAISS HNSW range search support by @alexanderguzhva in #853
• Update knowhere default thread priority by @congqixia in #868
• Remove cache for graph search by @liliu-z in #872
• fix raft compile error by @Presburger in #876
• Iterator support for diskann by @alwayslove2013 in #874

New Contributors

• @congqixia made their first contribution in #868

Full Changelog: v2.3.10...v2.3.11

v2.3.10

What's Changed

Features

• Iterator support for faiss hnsw indices by @alexanderguzhva in #823
• Add knowhere static interface for milvus to estimate loading resource by @foxspy in #836

Improvements

• Skip coverage check for trace code by @cydrain in #858
• Enhance BM25 performance issue by @zhengbuqian in #851
• Add more unittest for src/common by @cydrain in #850

Bug Fixes

• Set scoped omp to fix IVF index build degrade by @cydrain in #863
• Fix load ivf_flat with cosine metric loading error by @cqy123456 in #841
• Fix omp threads spawned by @cydrain in #835
• Fix Apple clang compile error by @Presburger in #828

Full Changelog: v2.3.9...v2.3.10

v2.3.9

What's Changed

Features

• faiss hnsw supports GetVectorByIds by @alexanderguzhva in #801

Improvements

• Use SIMD to accelerate the float vector normalization procedure by @cydrain in #797 #802
• Add API description for index_node by @liliu-z in #800
• Optimize error messages when encountering unsupported data types or invalid parameters by @cydrain in #808 #811
• Add CPU model limitation when creating SCANN index by @chasingegg in #816

Bug Fixes

• Patch to handle all-zero vectors within cosine metric by @cydrain in #803

Full Changelog: v2.3.8...v2.3.9

v2.3.8

Features

• Add support for Iterator on IVFSQ8 #772

Improvements

• Provide a static API to check if any index type contains raw vector data #751
• Iterator now supports returning results in order of distance through parameters #755 #766
• Expose Knowhere's config validation API through C interface #760

Bug Fixes

• Fix the issue where sparse INVERTED_INDEX couldn't handle empty vectors #775

v2.3.7

Features

• Support using CAGRA for indexing and HNSW for searching

Improvements

• Provide an API to check if any index type supports mmap #698
• When HNSW search results are fewer than k, use brute force search to ensure the number of results #711
• Add more monitor metrics #717 #732
• Support SIMD acceleration of FP16/BF16 computations #723

Changes

• RangeSearch terminates early based on topk, and won't return more than topk results #700
• Sparse Index now supports zero vectors in indexing and querying #704
  • Zero vectors in the index cannot be retrieved
  • Zero vector queries will return 0 results

Bug Fixes

• Fixed a bug where IVFSQ8 was using L2 to calculate IP distance when dim is divisible by 16 #703
• Fixed an issue where sparse INVERTED_INDEX wasn't properly using build parameters #721
• Fixed a problem where excessive threading was causing slow indexing speed for IVF series indices #741

v2.2.7

What's Changed

• fix: [cherry-pick]Replace gettid() with syscall(SYS_gettid) by @xige-16 in #697

New Contributors

• @xige-16 made their first contribution in #697

Full Changelog: v2.2.6...v2.2.7

v2.3.6

What's Changed

• Add data type conversion test by @cydrain in #653
• Update BruteForce ut for multi data type by @cydrain in #656
• Clean build warnings by @cydrain in #657
• Support numpy array to sparse dataset in pyknowhere api by @chasingegg in #654
• cagra support train on gpu, search on cpu, use hnswlib by @Presburger in #661
• Add following APIs to facilitate DataSet conversion by @cydrain in #659
• Update all APIs' input parameter to 'const DataSetPtr' by @cydrain in #667
• not check diskann range_search params max_k < min_k if with_cardinal by @alwayslove2013 in #664
• fix aix compilation by @alexanderguzhva in #665
• propagate faiss changes from (24 Jun 2024) by @alexanderguzhva in #663
• [range search] iterator stops only if exceeds radius several times consecutively. by @alwayslove2013 in #671
• [skip e2e] Automated Update Cardinal Commit by @sre-ci-robot in #673
• refactor memory management by @alexanderguzhva in #569
• remove additional metric_type definition from diskann_config by @alwayslove2013 in #674
• [fix] serialize on empty index by @foxspy in #677
• [skip e2e] Automated Update Cardinal Commit by @sre-ci-robot in #680
• a theoretical fix for binary data type by @alexanderguzhva in #676
• Fix uneven loading of gpu memory when use multi device by @Presburger in #681
• [skip ci] Update cardinal version by @foxspy in #684
• fix possible memory leak caused by iterator by @zhengbuqian in #689
• fix diskann range_search config: search_list_size should for_range_search by @alwayslove2013 in #694

Full Changelog: v2.3.5...v2.3.6

v2.3.5

What's Changed

• fix: make GetIsSparse const by @zhengbuqian in #627
• check the combination of index type and data type by @cqy123456 in #634
• add sparse support type in knowherecheck by @cqy123456 in #635
• [skip ci] Add diskann benchmark test by @cydrain in #637
• register index(binary_hnsw) in knowhere by @cqy123456 in #638
• [skip e2e] : remove iterator/sparse ut limit for cardinal by @foxspy in #636
• Params for iterator by @alwayslove2013 in #625
• [skip ci] Fix diskann benchmark fail by @cydrain in #640
• update raft to 24.04 by @Presburger in #628
• [skip ci] Update range search benchmark result display by @cydrain in #646
• [skip ci] Update script prepare_gpu_build.sh for nightly build by @cydrain in #648
• Update index and data type check function by @cqy123456 in #649

Full Changelog: v2.3.4...v2.3.5