Add allowlist setting for search-pipeline-common processors #14562
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
andrross
requested review from
anasalkouz,
Bukhtawar,
CEHENKLE,
dblock,
dbwiddis,
gbbafna,
kotwanikunal,
mch2,
msfroh,
nknize,
owaiskazi19,
reta,
Rishikesh1159,
sachinpkale,
saratvemulapalli,
shwetathareja,
sohami and
VachaShah
as code owners
andrross
commented
Jun 26, 2024
modules/ingest-common/src/main/java/org/opensearch/ingest/common/IngestCommonModulePlugin.java
Show resolved
Hide resolved
|
❌ Gradle check result for bc8ecc7: null Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
reta
reviewed
Jun 26, 2024
...on/src/main/java/org/opensearch/search/pipeline/common/SearchPipelineCommonModulePlugin.java
Show resolved
Hide resolved
reta
approved these changes
Jun 26, 2024
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #14562 +/- ##
============================================
+ Coverage 71.71% 71.74% +0.03%
- Complexity 62179 62222 +43
============================================
Files 5125 5125
Lines 292444 292472 +28
Branches 42257 42258 +1
============================================
+ Hits 209720 209831 +111
+ Misses 65434 65351 -83
Partials 17290 17290 ☔ View full report in Codecov by Sentry. |
Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com>
andrross
force-pushed
the
search-pipeline-allowlist
branch
from
bc8ecc7 to
d4a1af7
Compare
reta
approved these changes
Jun 27, 2024
|
❌ Gradle check result for d4a1af7: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
opensearch-trigger-bot bot
pushed a commit
that referenced
this pull request
Jun 27, 2024
Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to #14439 Signed-off-by: Andrew Ross <andrross@amazon.com> (cherry picked from commit d9e9944) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
This was referenced Jun 27, 2024
reta
pushed a commit
that referenced
this pull request
Jun 27, 2024
…14582) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to #14439 (cherry picked from commit d9e9944) Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
bharath-techie
added a commit
to bharath-techie/OpenSearch
that referenced
this pull request
Jul 10, 2024
* Fix flaky test in range aggregation yaml test (opensearch-project#14486) Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Use CODECOV_TOKEN (opensearch-project#14536) Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * [Tiered Caching] Moving query recomputation logic outside of write lock (opensearch-project#14187) * Moving query recompute out of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * [Tiered Caching] Moving query recomputation logic outside of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding java doc for the completable map Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Changes to call future handler only once per key Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing spotless check Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle fail Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments to refactor unit test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * minor UT refactor Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> * Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510) Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> * Add doc for debugging rest tests (opensearch-project#14491) * add doc for debugging rest tests Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Update TESTING.md Co-authored-by: Marc Handalian <handalm@amazon.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Address comment Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Marc Handalian <handalm@amazon.com> * Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462) Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> * [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549) * Add bwc version 2.15.1 Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix auto-generated version Signed-off-by: Andrew Ross <andrross@amazon.com> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com> * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550) * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> * Add allowlist setting for ingest-common processors (opensearch-project#14479) Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com> * Fix file cache initialization (opensearch-project#14004) * fix file cache initialization Signed-off-by: panguixin <panguixin@bytedance.com> * changelog Signed-off-by: panguixin <panguixin@bytedance.com> * add test Signed-off-by: panguixin <panguixin@bytedance.com> --------- Signed-off-by: panguixin <panguixin@bytedance.com> * Add Ashish Singh as maintainer (opensearch-project#14567) Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com> * Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517) * Bump com.azure:azure-storage-common in /plugins/repository-azure Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1. - [Release notes](https://github.com/Azure/azure-sdk-for-java/releases) - [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1) --- updated-dependencies: - dependency-name: com.azure:azure-storage-common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Add allowlist setting for search-pipeline-common processors (opensearch-project#14562) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com> * Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581) (cherry picked from commit 0095fd1) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427) * Add unittests for RemoteClusterStateAttributesManager Signed-off-by: Shivansh Arora <hishiv@amazon.com> * Add Ashish Singh to codeowners (opensearch-project#14592) Signed-off-by: Ashish Singh <ssashish@amazon.com> * Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554) Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Add @internalapi annotation to japicmp exclusions (opensearch-project#14597) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552) * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:update changelog Signed-off-by: vatsal <vatsal.v.anand@gmail.com> --------- Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> * Star tree mapping changes (opensearch-project#14605) * Star tree mapping changes with feature flag --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun security check Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> * Correct typo in method name (opensearch-project#14621) Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200) * Refactor FilterPath parse function (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Implement unit tests for FilterPathTests (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Write warn log if Filter is empty; Add comments (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unnecessary log statement Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unused logger Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless apply Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove incorrect changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings Signed-off-by: Sooraj Sinha <soosinha@amazon.com> * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> * Fix assertion failure while deleting remote backed index (opensearch-project#14601) Signed-off-by: Sachin Kale <kalsac@amazon.com> * OnHeap Star Tree Implementation Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed major nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * includes Count Aggregator Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * handling for missing doc values Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressing review comments Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * rebasing with main Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * support for empty sequential doc values iterator Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * min and max star tree aggregators Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * star tree file formats * Star tree codec changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Adding tests Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Addressing comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * addressing review comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Star tree merge changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * fix annotations * star-tree file formats reader and javadoc fixes * read for composite index values Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * doc values file format Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Signed-off-by: panguixin <panguixin@bytedance.com> Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> Signed-off-by: Bharathwaj G <bharath78910@gmail.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com> Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: kkewwei <kkewwei@163.com> Co-authored-by: Marc Handalian <handalm@amazon.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com> Co-authored-by: panguixin <panguixin@bytedance.com> Co-authored-by: Bukhtawar Khan <bukhtawa@amazon.com> Co-authored-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: Liyun Xiu <xiliyun@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
bharath-techie
added a commit
to bharath-techie/OpenSearch
that referenced
this pull request
Jul 11, 2024
* Fix flaky test in range aggregation yaml test (opensearch-project#14486) Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Use CODECOV_TOKEN (opensearch-project#14536) Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * [Tiered Caching] Moving query recomputation logic outside of write lock (opensearch-project#14187) * Moving query recompute out of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * [Tiered Caching] Moving query recomputation logic outside of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding java doc for the completable map Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Changes to call future handler only once per key Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing spotless check Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle fail Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments to refactor unit test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * minor UT refactor Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> * Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510) Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> * Add doc for debugging rest tests (opensearch-project#14491) * add doc for debugging rest tests Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Update TESTING.md Co-authored-by: Marc Handalian <handalm@amazon.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Address comment Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Marc Handalian <handalm@amazon.com> * Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462) Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> * [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549) * Add bwc version 2.15.1 Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix auto-generated version Signed-off-by: Andrew Ross <andrross@amazon.com> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com> * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550) * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> * Add allowlist setting for ingest-common processors (opensearch-project#14479) Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com> * Fix file cache initialization (opensearch-project#14004) * fix file cache initialization Signed-off-by: panguixin <panguixin@bytedance.com> * changelog Signed-off-by: panguixin <panguixin@bytedance.com> * add test Signed-off-by: panguixin <panguixin@bytedance.com> --------- Signed-off-by: panguixin <panguixin@bytedance.com> * Add Ashish Singh as maintainer (opensearch-project#14567) Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com> * Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517) * Bump com.azure:azure-storage-common in /plugins/repository-azure Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1. - [Release notes](https://github.com/Azure/azure-sdk-for-java/releases) - [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1) --- updated-dependencies: - dependency-name: com.azure:azure-storage-common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Add allowlist setting for search-pipeline-common processors (opensearch-project#14562) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com> * Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581) (cherry picked from commit 0095fd1) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427) * Add unittests for RemoteClusterStateAttributesManager Signed-off-by: Shivansh Arora <hishiv@amazon.com> * Add Ashish Singh to codeowners (opensearch-project#14592) Signed-off-by: Ashish Singh <ssashish@amazon.com> * Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554) Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Add @internalapi annotation to japicmp exclusions (opensearch-project#14597) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552) * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:update changelog Signed-off-by: vatsal <vatsal.v.anand@gmail.com> --------- Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> * Star tree mapping changes (opensearch-project#14605) * Star tree mapping changes with feature flag --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun security check Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> * Correct typo in method name (opensearch-project#14621) Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200) * Refactor FilterPath parse function (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Implement unit tests for FilterPathTests (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Write warn log if Filter is empty; Add comments (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unnecessary log statement Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unused logger Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless apply Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove incorrect changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings Signed-off-by: Sooraj Sinha <soosinha@amazon.com> * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> * Fix assertion failure while deleting remote backed index (opensearch-project#14601) Signed-off-by: Sachin Kale <kalsac@amazon.com> * OnHeap Star Tree Implementation Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed major nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * includes Count Aggregator Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * handling for missing doc values Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressing review comments Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * rebasing with main Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * support for empty sequential doc values iterator Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * min and max star tree aggregators Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * star tree file formats * Star tree codec changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Adding tests Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Addressing comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * addressing review comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Star tree merge changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * fix annotations * star-tree file formats reader and javadoc fixes * read for composite index values Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * doc values file format Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Signed-off-by: panguixin <panguixin@bytedance.com> Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> Signed-off-by: Bharathwaj G <bharath78910@gmail.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com> Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: kkewwei <kkewwei@163.com> Co-authored-by: Marc Handalian <handalm@amazon.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com> Co-authored-by: panguixin <panguixin@bytedance.com> Co-authored-by: Bukhtawar Khan <bukhtawa@amazon.com> Co-authored-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: Liyun Xiu <xiliyun@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
bharath-techie
added a commit
to bharath-techie/OpenSearch
that referenced
this pull request
Jul 11, 2024
* Fix flaky test in range aggregation yaml test (opensearch-project#14486) Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Use CODECOV_TOKEN (opensearch-project#14536) Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> * [Tiered Caching] Moving query recomputation logic outside of write lock (opensearch-project#14187) * Moving query recompute out of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * [Tiered Caching] Moving query recomputation logic outside of write lock Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Adding java doc for the completable map Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Changes to call future handler only once per key Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing spotless check Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Added changelog Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Fixing gradle fail Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comments to refactor unit test Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * minor UT refactor Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> * Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510) Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> * Add doc for debugging rest tests (opensearch-project#14491) * add doc for debugging rest tests Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Update TESTING.md Co-authored-by: Marc Handalian <handalm@amazon.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> * Address comment Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Marc Handalian <handalm@amazon.com> * Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462) Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> * [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549) * Add bwc version 2.15.1 Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> * Fix auto-generated version Signed-off-by: Andrew Ross <andrross@amazon.com> --------- Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com> * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550) * Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> * Addressing comment Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> --------- Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> * Add allowlist setting for ingest-common processors (opensearch-project#14479) Add a new static setting that lets an operator choose specific ingest processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com> * Fix file cache initialization (opensearch-project#14004) * fix file cache initialization Signed-off-by: panguixin <panguixin@bytedance.com> * changelog Signed-off-by: panguixin <panguixin@bytedance.com> * add test Signed-off-by: panguixin <panguixin@bytedance.com> --------- Signed-off-by: panguixin <panguixin@bytedance.com> * Add Ashish Singh as maintainer (opensearch-project#14567) Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com> * Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517) * Bump com.azure:azure-storage-common in /plugins/repository-azure Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1. - [Release notes](https://github.com/Azure/azure-sdk-for-java/releases) - [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1) --- updated-dependencies: - dependency-name: com.azure:azure-storage-common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> --------- Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * Add allowlist setting for search-pipeline-common processors (opensearch-project#14562) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com> * Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581) (cherry picked from commit 0095fd1) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427) * Add unittests for RemoteClusterStateAttributesManager Signed-off-by: Shivansh Arora <hishiv@amazon.com> * Add Ashish Singh to codeowners (opensearch-project#14592) Signed-off-by: Ashish Singh <ssashish@amazon.com> * Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554) Signed-off-by: Liyun Xiu <xiliyun@amazon.com> * Add @internalapi annotation to japicmp exclusions (opensearch-project#14597) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552) * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Fix issue 14519:update changelog Signed-off-by: vatsal <vatsal.v.anand@gmail.com> --------- Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> * Star tree mapping changes (opensearch-project#14605) * Star tree mapping changes with feature flag --------- Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610) * Bump com.microsoft.azure:msal4j in /plugins/repository-azure Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0. - [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases) - [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt) - [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0) --- updated-dependencies: - dependency-name: com.microsoft.azure:msal4j dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> * Update changelog Signed-off-by: dependabot[bot] <support@github.com> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> * [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564) * Fix testInvalidInput flakiness Signed-off-by: Peter Alfonsi <petealft@amazon.com> * Addressed andrross's comment Signed-off-by: Peter Alfonsi <petealft@amazon.com> * rerun security check Signed-off-by: Peter Alfonsi <petealft@amazon.com> --------- Signed-off-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> * Correct typo in method name (opensearch-project#14621) Signed-off-by: vatsal <vatsal.v.anand@gmail.com> * Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200) * Refactor FilterPath parse function (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Implement unit tests for FilterPathTests (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Write warn log if Filter is empty; Add comments (opensearch-project#12067) Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Add changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unnecessary log statement Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove unused logger Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Spotless apply Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> * Remove incorrect changelog Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> --------- Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> * Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513) * Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata * Correct the check for deciding upload of hashes of consistent settings Signed-off-by: Sooraj Sinha <soosinha@amazon.com> * Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613) Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> * Fix assertion failure while deleting remote backed index (opensearch-project#14601) Signed-off-by: Sachin Kale <kalsac@amazon.com> * OnHeap Star Tree Implementation Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressed major nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * includes Count Aggregator Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * handling for missing doc values Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * addressing review comments Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * rebasing with main Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * support for empty sequential doc values iterator Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * nits Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * min and max star tree aggregators Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * star tree file formats * Star tree codec changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Adding tests Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Addressing comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * addressing review comments Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * Star tree merge changes Signed-off-by: Bharathwaj G <bharath78910@gmail.com> * fix annotations * star-tree file formats reader and javadoc fixes * read for composite index values Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> * doc values file format Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> --------- Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com> Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com> Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com> Signed-off-by: kkewwei kkewwei@163.com Signed-off-by: kkewwei <kkewwei@163.com> Signed-off-by: Peter Alfonsi <petealft@amazon.com> Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com> Signed-off-by: panguixin <panguixin@bytedance.com> Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com> Signed-off-by: Andriy Redko <andriy.redko@aiven.io> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Liyun Xiu <xiliyun@amazon.com> Signed-off-by: vatsal <vatsal.v.anand@gmail.com> Signed-off-by: Daniel Widdis <widdis@gmail.com> Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com> Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com> Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Sachin Kale <kalsac@amazon.com> Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com> Signed-off-by: Bharathwaj G <bharath78910@gmail.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com> Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com> Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com> Co-authored-by: kkewwei <kkewwei@163.com> Co-authored-by: Marc Handalian <handalm@amazon.com> Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com> Co-authored-by: Peter Alfonsi <petealft@amazon.com> Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com> Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com> Co-authored-by: Andrew Ross <andrross@amazon.com> Co-authored-by: panguixin <panguixin@bytedance.com> Co-authored-by: Bukhtawar Khan <bukhtawa@amazon.com> Co-authored-by: Andriy Redko <andriy.redko@aiven.io> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> Co-authored-by: Shivansh Arora <hishiv@amazon.com> Co-authored-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: Liyun Xiu <xiliyun@amazon.com> Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com> Co-authored-by: Daniel Widdis <widdis@gmail.com> Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com> Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com> Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Sachin Kale <sachinpkale@gmail.com> Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
harshavamsi
pushed a commit
to harshavamsi/OpenSearch
that referenced
this pull request
Jul 12, 2024
…ch-project#14562) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com>
3 tasks
kkewwei
pushed a commit
to kkewwei/OpenSearch
that referenced
this pull request
Jul 24, 2024
…ch-project#14562) (opensearch-project#14582) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 (cherry picked from commit d9e9944) Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: kkewwei <kkewwei@163.com>
wdongyu
pushed a commit
to wdongyu/OpenSearch
that referenced
this pull request
Aug 22, 2024
…ch-project#14562) Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows: - If the allowlist setting is not defined, all installed processors are enabled. This is the status quo. - If the allowlist setting is defined as the empty set, then all processors are disabled. - If the allowlist setting contains the names of valid processors, only those processors are enabled. - If the allowlist setting contains a name of a processor that does not exist, then the server will fail to start with an IllegalStateException listing which processors were defined in the allowlist but are not installed. - If the allowlist setting is changed between server restarts then any ingest pipeline using a now-disabled processor will fail. This is the same experience if a pipeline used a processor defined by a plugin but then that plugin were to be uninstalled across restarts. A distinct setting exists for each of request, response, and search phase results processors. Related to opensearch-project#14439 Signed-off-by: Andrew Ross <andrross@amazon.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add a new static setting that lets an operator choose specific search pipeline processors to enable by name. The behavior is as follows:
A distinct setting exists for each of request, response, and search phase results processors.
Related to #14439
Check List
API changes companion pull request created, if applicable.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.