Skip to content

Commit

Permalink
Update kNN search guide with knn parallelization (#100705)
Browse files Browse the repository at this point in the history
Relates to PR #98204
  • Loading branch information
mayya-sharipova authored Oct 11, 2023
1 parent 09c8cf5 commit b582276
Showing 1 changed file with 9 additions and 4 deletions.
13 changes: 9 additions & 4 deletions docs/reference/how-to/knn-search.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -83,14 +83,19 @@ since it relies on separate data structures to perform the search. Before
using the <<include-exclude, `excludes`>> parameter, make sure to review the
downsides of omitting fields from `_source`.

Another option is to use <<synthetic-source,synthetic `_source`>> if all
your index fields support it.


[discrete]
=== Reduce the number of index segments

{es} shards are composed of segments, which are internal storage elements in
the index. For approximate kNN search, {es} stores the dense vector values of
each segment as an HNSW graph. kNN search must check each segment, searching
through one HNSW graph after another. This means kNN search can be
significantly faster if there are fewer segments. By default, {es} periodically
the index. For approximate kNN search, {es} stores the vector values of
each segment as a separate HNSW graph, so kNN search must check each segment.
The recent parallelization of kNN search made it much faster to search across
multiple segments, but still kNN search can be up to several times
faster if there are fewer segments. By default, {es} periodically
merges smaller segments into larger ones through a background
<<index-modules-merge, merge process>>. If this isn't sufficient, you can take
explicit steps to reduce the number of index segments.
Expand Down

0 comments on commit b582276

Please sign in to comment.