-
Notifications
You must be signed in to change notification settings - Fork 65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use lazy initialization for priority queue of hits and scores to improve latencies by 20% #746
Merged
martin-gaievski
merged 2 commits into
opensearch-project:main
from
martin-gaievski:hybridq_optimization_replace_pq_updatetop_by_insertwithoverflow
May 21, 2024
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
martin-gaievski
changed the title
Use lazy initialization for priority queue of hits and scores to improve latencies by 10%
Use lazy initialization for priority queue of hits and scores to improve latencies by 20%
May 10, 2024
martin-gaievski
force-pushed
the
hybridq_optimization_replace_pq_updatetop_by_insertwithoverflow
branch
from
May 10, 2024 00:21
32a620a
to
8ccee40
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #746 +/- ##
============================================
- Coverage 85.02% 84.86% -0.16%
- Complexity 790 792 +2
============================================
Files 60 60
Lines 2430 2425 -5
Branches 410 409 -1
============================================
- Hits 2066 2058 -8
- Misses 202 207 +5
+ Partials 162 160 -2 ☔ View full report in Codecov by Sentry. |
martin-gaievski
requested review from
heemin32,
navneet1v,
VijayanB,
vamshin,
jmazanec15,
naveentatikonda,
junqiu-lei,
sean-zheng-amazon,
model-collapse,
zane-neo,
ylwu-amzn,
jngz-es,
vibrantvarun and
zhichao-aws
as code owners
May 10, 2024 15:17
navneet1v
reviewed
May 16, 2024
src/main/java/org/opensearch/neuralsearch/search/HybridTopScoreDocCollector.java
Outdated
Show resolved
Hide resolved
src/main/java/org/opensearch/neuralsearch/search/HybridTopScoreDocCollector.java
Show resolved
Hide resolved
Signed-off-by: Martin Gaievski <gaievski@amazon.com>
Signed-off-by: Martin Gaievski <gaievski@amazon.com>
martin-gaievski
force-pushed
the
hybridq_optimization_replace_pq_updatetop_by_insertwithoverflow
branch
from
May 21, 2024 16:37
98b588b
to
4d0c69d
Compare
navneet1v
approved these changes
May 21, 2024
VijayanB
reviewed
May 21, 2024
VijayanB
approved these changes
May 21, 2024
martin-gaievski
added
backport 2.x
Label will add auto workflow to backport PR to 2.x branch
v2.15.0
hybrid search
hybrid query performance optimization
labels
May 21, 2024
martin-gaievski
merged commit May 21, 2024
940a7ea
into
opensearch-project:main
82 of 83 checks passed
VijayanB
pushed a commit
that referenced
this pull request
May 29, 2024
…ove latencies by 20% (#746) * Optimize PQ for hybrid scores Signed-off-by: Martin Gaievski <gaievski@amazon.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
backport 2.x
Label will add auto workflow to backport PR to 2.x branch
hybrid query performance optimization
hybrid search
performance
Make it fast!
v2.15.0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
In this change we're improving hybrid query latencies by changing allocation of query hits objects for hybrid scores. Currently those objects are pre-populated, then for new element top of the priority queue is updated and heap is rebalanced. We're changing this to lazy initialization - priority queue is initially empty, for each hit we're inserting element. If we reach the capacity then element with low score is pushed out of the heap. As per benchmark results this alone gives 10% improvement.
With several other minor changes like pre calculate and store size instead of calling size() for each doc and each sub-query we can get 10% percent.
Below are metrics for hybrid query latency that I've collected after this change. There are based on 2.x (2.14) version and noaa OSB workload, all times are in ms:
following are the baseline results
Following framegraph has been captured after this change:
Issues Resolved
#745
#704
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.