-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove mmap.extensions setting #9392
Conversation
Gradle Check (Jenkins) Run Completed with:
|
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:Checks if related components are compatible with change 8d153ab Incompatible componentsIncompatible components: [https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/security-analytics.git] Skipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git] |
Compatibility status:Checks if related components are compatible with change 8d153ab Incompatible componentsIncompatible components: [https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/security-analytics.git] Skipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git] |
Gradle Check (Jenkins) Run Completed with:
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #9392 +/- ##
============================================
- Coverage 71.86% 71.80% -0.07%
+ Complexity 62757 62726 -31
============================================
Files 5161 5161
Lines 294370 294338 -32
Branches 42579 42574 -5
============================================
- Hits 211548 211340 -208
- Misses 65386 65510 +124
- Partials 17436 17488 +52 ☔ View full report in Codecov by Sentry. |
Signed-off-by: David Zane <davizane@amazon.com>
@dzane17 @jainankitk This will break k-NN. We should update ensure k-NN gets updated before merging this change (either open an issue with a specific suggestion or create a PR directly). |
❌ Gradle check result for bc0d3dd: null Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
@navneet1v - Is this the only place KNN plugin is using this setting? Also, are there any file extensions that KNN needs to read as NIOFS? If not, MMAP is the new default unless overridden using NIOFS setting. cc: @vamshin |
Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: David Zane <38449481+dzane17@users.noreply.github.com>
KNN don't use any files as NIOFS. |
@navneet1v - Great, thanks for confirming. Can you also create PR to remove the |
@andrross LGTY? |
Per opensearch-project/OpenSearch#9392, mmap will be a default setting for all lucene segment files Signed-off-by: Heemin Kim <heemin@amazon.com>
Per opensearch-project/OpenSearch#9392, mmap will be a default setting for all lucene segment files Signed-off-by: Heemin Kim <heemin@amazon.com>
Per opensearch-project/OpenSearch#9392, mmap will be a default setting for all lucene segment files Signed-off-by: Heemin Kim <heemin@amazon.com>
* Remove mmap.extensions setting in favor of nio.extensions Signed-off-by: David Zane <davizane@amazon.com> * Update CHANGELOG-3.0.md Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: David Zane <38449481+dzane17@users.noreply.github.com> --------- Signed-off-by: David Zane <davizane@amazon.com> Signed-off-by: David Zane <38449481+dzane17@users.noreply.github.com> Co-authored-by: Andriy Redko <drreta@gmail.com>
* Remove mmap.extensions setting in favor of nio.extensions Signed-off-by: David Zane <davizane@amazon.com> * Update CHANGELOG-3.0.md Co-authored-by: Andriy Redko <drreta@gmail.com> Signed-off-by: David Zane <38449481+dzane17@users.noreply.github.com> --------- Signed-off-by: David Zane <davizane@amazon.com> Signed-off-by: David Zane <38449481+dzane17@users.noreply.github.com> Co-authored-by: Andriy Redko <drreta@gmail.com>
Description
Removes
index.store.hybrid.mmap.extensions
setting in OS 3.0.Follow up to #8508
Background
Currently OpenSearch code contains an explicit list of file extensions which load using mmap from hybridfs. Other file extensions default to nio. This PR flips the logic to keep a list of nio file extensions while all others default to mmap. This will prevent any future regressions in case Lucene adds new segment file type that should be using mmap.
Related Issues
Resolves #8297
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.