Skip to content
This repository has been archived by the owner on Apr 4, 2023. It is now read-only.

Optimise the ExactWords sub-criterion within Exactness #709

Merged
merged 2 commits into from
Jan 2, 2023
Merged

Conversation

loiclec
Copy link
Contributor

@loiclec loiclec commented Nov 24, 2022

Pull Request

Related issue

Fixes (partially) meilisearch/meilisearch#3116

What does this PR do?

  1. Reduces the algorithmic complexity of finding the documents containing N exact words from something that is exponential to something that is polynomial.
  2. Cache intermediary results between different calls to the exactness criterion.

Performance Results

On the smol_songs.csv dataset, a request containing 10 common words now takes about 60ms instead of 5 seconds to execute. For example, this is the case with this (admittedly nonsensical) request: Rock You Hip Hop Folk World Country Electronic Love The.

@loiclec loiclec requested a review from ManyTheFish November 24, 2022 15:10
@loiclec loiclec added no breaking The related changes are not breaking (DB nor API) performance Related to the performance in term of search/indexation speed or RAM/CPU/Disk consumption labels Nov 24, 2022
Copy link
Member

@ManyTheFish ManyTheFish left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @loiclec, nice PR,
I requested some small changes that can be discussed if you feel that there are not relevant,

Thanks!

milli/src/search/criteria/exactness.rs Outdated Show resolved Hide resolved
milli/src/search/criteria/exactness.rs Outdated Show resolved Hide resolved
milli/src/search/criteria/exactness.rs Outdated Show resolved Hide resolved
milli/src/search/criteria/exactness.rs Outdated Show resolved Hide resolved
milli/src/search/criteria/exactness.rs Outdated Show resolved Hide resolved
1. Cache some results between calls to next()
2. Compute the combinations of exact words more efficiently
@loiclec loiclec requested a review from irevoire January 2, 2023 12:18
@irevoire irevoire requested a review from ManyTheFish January 2, 2023 12:19
@loiclec loiclec dismissed ManyTheFish’s stale review January 2, 2023 12:24

Many is on holiday and can't re-review the PR. I applied his suggestions :)

@loiclec
Copy link
Contributor Author

loiclec commented Jan 2, 2023

Thank you!
bors merge

@bors
Copy link
Contributor

bors bot commented Jan 2, 2023

Build succeeded!

And happy new year! 🎉

@bors bors bot merged commit 9519e60 into main Jan 2, 2023
@bors bors bot deleted the exactness-perf branch January 2, 2023 12:45
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
no breaking The related changes are not breaking (DB nor API) performance Related to the performance in term of search/indexation speed or RAM/CPU/Disk consumption
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants