Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Refactor] Replace recursive FilterPath.parse with iterative approach #12067

Closed
2 tasks
cwperks opened this issue Jan 29, 2024 · 3 comments · Fixed by #14200
Closed
2 tasks

[Refactor] Replace recursive FilterPath.parse with iterative approach #12067

cwperks opened this issue Jan 29, 2024 · 3 comments · Fixed by #14200
Assignees
@deshsidd
Labels
bug Something isn't working good first issue Good for newcomers Search Search query, autocomplete ...etc

Comments

@cwperks
Copy link
Member

cwperks commented Jan 29, 2024

Describe the bug

This issue is related to a recent pull request: #11060

There is another instance of recursion used in the FilterPath.parse method which appears inefficient and unnecessary. It appears to be ignoring repeat . until the last is located and could more optimally skip until the last . in the sequence instead of recursing.

Additional Context

In #11060, there was a recursive method for evaluating core's own Regex class. The method simpleMatchWithNormalizedString was refactored from a recursive approach to an iterative approach for more predictable space and runtime complexity.

  • Refactor FilterPath.parse to use iterative method
  • Update FilterPathTests to cover more scenarios

Related component

Search

To Reproduce

Should maintain same behavior as now

Expected behavior

Should maintain same behavior as now

Additional Details

No response
@cwperks cwperks added bug Something isn't working untriaged labels Jan 29, 2024
@github-actions github-actions bot added the Search Search query, autocomplete ...etc label Jan 29, 2024
@peternied
Copy link
Member

[Triage - attendees 1 2 3 4 5 6 7 8]
@cwperks Thanks for filing, we'd gladly review a pull request.

@peternied peternied added good first issue Good for newcomers and removed untriaged labels Jan 31, 2024
@reta reta mentioned this issue Jan 31, 2024
Closed
robinf95 added a commit to robinf95/OpenSearch that referenced this issue Feb 1, 2024
@robinf95
Refactor FilterPath parse function (opensearch-project#12067)
5abffd7 
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
robinf95 added a commit to robinf95/OpenSearch that referenced this issue Feb 1, 2024
@robinf95
Refactor FilterPath parse function (opensearch-project#12067)
43f5867 
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
@robinf95 robinf95 mentioned this issue Feb 1, 2024
Closed
8 tasks
robinf95 added a commit to robinf95/OpenSearch that referenced this issue Feb 5, 2024
@robinf95
Implement unit tests for FilterPathTests (opensearch-project#12067)
d0bb649 
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
robinf95 added a commit to robinf95/OpenSearch that referenced this issue Feb 5, 2024
@robinf95
Write warn log if Filter is empty; Add comments (opensearch-project#1…
0be57d1 
…2067)

Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
robinf95 added a commit to robinf95/OpenSearch that referenced this issue Feb 5, 2024
@robinf95
Write warn log if Filter is empty; Add comments (opensearch-project#1…
997d2fa 
…2067)

Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
robinf95 added a commit to robinf95/OpenSearch that referenced this issue Feb 5, 2024
@robinf95
Write warn log if Filter is empty; Add comments (opensearch-project#1…
c154bd7 
…2067)

Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
@getsaurabh02
Copy link
Member

We discussed this in the Search community meeting today. @jainankitk is going to take a stab at it this week.

@jainankitk
Copy link
Collaborator

jainankitk commented May 29, 2024 •
edited
Loading

@niyatiagg - Can you help with this issue?

@deshsidd deshsidd mentioned this issue Jun 11, 2024
Merged
3 tasks
@sandeshkr419 sandeshkr419 assigned deshsidd and unassigned jainankitk Jun 12, 2024
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this issue Jun 12, 2024
@robinf95 @deshsidd
Refactor FilterPath parse function (opensearch-project#12067)
1bfd4c9 
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this issue Jun 12, 2024
@robinf95 @deshsidd
Implement unit tests for FilterPathTests (opensearch-project#12067)
22b3cd5 
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this issue Jun 12, 2024
@robinf95 @deshsidd
Write warn log if Filter is empty; Add comments (opensearch-project#1…
cd1c7bf 
…2067)

Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this issue Jun 12, 2024
@robinf95 @deshsidd
Refactor FilterPath parse function (opensearch-project#12067)
046850e 
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this issue Jun 12, 2024
@robinf95 @deshsidd
Implement unit tests for FilterPathTests (opensearch-project#12067)
9dbfb75 
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this issue Jun 12, 2024
@robinf95 @deshsidd
Write warn log if Filter is empty; Add comments (opensearch-project#1…
b362196 
…2067)

Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this issue Jun 12, 2024
@robinf95 @deshsidd
Refactor FilterPath parse function (opensearch-project#12067)
b17cac9 
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this issue Jun 12, 2024
@robinf95 @deshsidd
Implement unit tests for FilterPathTests (opensearch-project#12067)
15ec467 
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this issue Jul 2, 2024
@robinf95 @deshsidd
Refactor FilterPath parse function (opensearch-project#12067)
800e039 
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this issue Jul 2, 2024
@robinf95 @deshsidd
Implement unit tests for FilterPathTests (opensearch-project#12067)
4f8ab61 
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this issue Jul 2, 2024
@robinf95 @deshsidd
Write warn log if Filter is empty; Add comments (opensearch-project#1…
778e92d 
…2067)

Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
dblock pushed a commit that referenced this issue Jul 2, 2024
@deshsidd @robinf95
Refactoring FilterPath.parse by using an iterative approach instead o…
0742453 
…f recursion. (#14200)

* Refactor FilterPath parse function (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
@github-project-automation github-project-automation bot moved this from 🆕 New to ✅ Done in Search Project Board Jul 2, 2024
opensearch-trigger-bot bot pushed a commit that referenced this issue Jul 2, 2024
@github-actions @robinf95
Refactoring FilterPath.parse by using an iterative approach instead o…
1788244 
…f recursion. (#14200)

* Refactor FilterPath parse function (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
(cherry picked from commit 0742453)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
dblock pushed a commit that referenced this issue Jul 2, 2024
@opensearch-trigger-bot @github-actions @robinf95
Refactoring FilterPath.parse by using an iterative approach instead o…
489a942 
…f recursion. (#14200) (#14629)

* Refactor FilterPath parse function (#12067)


* Implement unit tests for FilterPathTests (#12067)


* Write warn log if Filter is empty; Add comments (#12067)


* Add changelog



* Remove unnecessary log statement



* Remove unused logger



* Spotless apply



* Remove incorrect changelog



---------



(cherry picked from commit 0742453)

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
kaushalmahi12 pushed a commit to kaushalmahi12/OpenSearch that referenced this issue Jul 3, 2024
@deshsidd @robinf95 @kaushalmahi12
Refactoring FilterPath.parse by using an iterative approach instead o…
821c44d 
…f recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
kaushalmahi12 pushed a commit to kaushalmahi12/OpenSearch that referenced this issue Jul 8, 2024
@deshsidd @robinf95 @kaushalmahi12
Refactoring FilterPath.parse by using an iterative approach instead o…
6abdba2 
…f recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
akolarkunnu pushed a commit to akolarkunnu/OpenSearch that referenced this issue Jul 9, 2024
@deshsidd @robinf95 @akolarkunnu
Refactoring FilterPath.parse by using an iterative approach instead o…
ff6a6c2 
…f recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
bharath-techie added a commit to bharath-techie/OpenSearch that referenced this issue Jul 10, 2024
@bharath-techie @bowenlan-amzn
@prudhvigodithi @sgup432 @kkewwei @mch2 @peteralfonsi @opensearch-trigger-bot @opensearch-ci-bot @andrross @bugmakerrrrrr @Bukhtawar @reta @dependabot @shiv0408 @ashking94 @chishui @imvtsl @dbwiddis @deshsidd @robinf95 @RS146BIJAY @soosinha @sachinpkale @sarthakaggarwal97
Startree file formats codec merge (#29)
7c6b6f9 
* Fix flaky test in range aggregation yaml test (opensearch-project#14486)

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Use CODECOV_TOKEN (opensearch-project#14536)

Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>

* [Tiered Caching]  Moving query recomputation logic outside of write lock (opensearch-project#14187)

* Moving query recompute out of write lock

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* [Tiered Caching] Moving query recomputation logic outside of write lock

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding java doc for the completable map

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Changes to call future handler only once per key

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing spotless check

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Added changelog

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comments

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing gradle fail

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comments to refactor unit test

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* minor UT refactor

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>

* Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510)

Signed-off-by: kkewwei kkewwei@163.com

Signed-off-by: kkewwei kkewwei@163.com
Signed-off-by: kkewwei <kkewwei@163.com>

* Add doc for debugging rest tests (opensearch-project#14491)

* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Update TESTING.md

Co-authored-by: Marc Handalian <handalm@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Address comment

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Marc Handalian <handalm@amazon.com>

* Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462)

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>

* [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549)

* Add bwc version 2.15.1

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Fix auto-generated version

Signed-off-by: Andrew Ross <andrross@amazon.com>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550)

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comment

Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>

* Add allowlist setting for ingest-common processors (opensearch-project#14479)

Add a new static setting that lets an operator choose specific ingest
processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Fix file cache initialization (opensearch-project#14004)

* fix file cache initialization

Signed-off-by: panguixin <panguixin@bytedance.com>

* changelog

Signed-off-by: panguixin <panguixin@bytedance.com>

* add test

Signed-off-by: panguixin <panguixin@bytedance.com>

---------

Signed-off-by: panguixin <panguixin@bytedance.com>

* Add Ashish Singh as maintainer (opensearch-project#14567)

Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>

* Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517)

* Bump com.azure:azure-storage-common in /plugins/repository-azure

Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1)

---
updated-dependencies:
- dependency-name: com.azure:azure-storage-common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Add allowlist setting for search-pipeline-common processors (opensearch-project#14562)

Add a new static setting that lets an operator choose specific search
pipeline processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

A distinct setting exists for each of request, response, and search phase
results processors.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581)

(cherry picked from commit 0095fd1)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427)

* Add unittests for RemoteClusterStateAttributesManager

Signed-off-by: Shivansh Arora <hishiv@amazon.com>

* Add Ashish Singh to codeowners (opensearch-project#14592)

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554)

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

* Add @internalapi annotation to japicmp exclusions (opensearch-project#14597)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552)

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:update changelog

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

---------

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>

* Star tree mapping changes (opensearch-project#14605)

* Star tree mapping changes with feature flag
---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>

* OnHeap Star Tree Implementation

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressed nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressed major nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* includes Count Aggregator

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* handling for missing doc values

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressing review comments

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* rebasing with main

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* support for empty sequential doc values iterator

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* min and max star tree aggregators

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* star tree file formats

* Star tree codec changes

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Adding tests

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Addressing comments

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* addressing review comments

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Star tree merge changes

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* fix annotations

* star-tree file formats reader and javadoc fixes

* read for composite index values

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* doc values file format

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com>
Signed-off-by: kkewwei kkewwei@163.com
Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>
Signed-off-by: panguixin <panguixin@bytedance.com>
Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com>
Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>
Co-authored-by: kkewwei <kkewwei@163.com>
Co-authored-by: Marc Handalian <handalm@amazon.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: panguixin <panguixin@bytedance.com>
Co-authored-by: Bukhtawar Khan <bukhtawa@amazon.com>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Ashish Singh <ssashish@amazon.com>
Co-authored-by: Liyun Xiu <xiliyun@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
bharath-techie added a commit to bharath-techie/OpenSearch that referenced this issue Jul 11, 2024
@bharath-techie @bowenlan-amzn
@prudhvigodithi @sgup432 @kkewwei @mch2 @peteralfonsi @opensearch-trigger-bot @opensearch-ci-bot @andrross @bugmakerrrrrr @Bukhtawar @reta @dependabot @shiv0408 @ashking94 @chishui @imvtsl @dbwiddis @deshsidd @robinf95 @RS146BIJAY @soosinha @sachinpkale @sarthakaggarwal97
Startree file formats codec merge (#29)
36b72ab 
* Fix flaky test in range aggregation yaml test (opensearch-project#14486)

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Use CODECOV_TOKEN (opensearch-project#14536)

Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>

* [Tiered Caching]  Moving query recomputation logic outside of write lock (opensearch-project#14187)

* Moving query recompute out of write lock

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* [Tiered Caching] Moving query recomputation logic outside of write lock

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding java doc for the completable map

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Changes to call future handler only once per key

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing spotless check

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Added changelog

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comments

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing gradle fail

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comments to refactor unit test

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* minor UT refactor

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>

* Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510)

Signed-off-by: kkewwei kkewwei@163.com

Signed-off-by: kkewwei kkewwei@163.com
Signed-off-by: kkewwei <kkewwei@163.com>

* Add doc for debugging rest tests (opensearch-project#14491)

* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Update TESTING.md

Co-authored-by: Marc Handalian <handalm@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Address comment

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Marc Handalian <handalm@amazon.com>

* Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462)

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>

* [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549)

* Add bwc version 2.15.1

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Fix auto-generated version

Signed-off-by: Andrew Ross <andrross@amazon.com>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550)

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comment

Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>

* Add allowlist setting for ingest-common processors (opensearch-project#14479)

Add a new static setting that lets an operator choose specific ingest
processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Fix file cache initialization (opensearch-project#14004)

* fix file cache initialization

Signed-off-by: panguixin <panguixin@bytedance.com>

* changelog

Signed-off-by: panguixin <panguixin@bytedance.com>

* add test

Signed-off-by: panguixin <panguixin@bytedance.com>

---------

Signed-off-by: panguixin <panguixin@bytedance.com>

* Add Ashish Singh as maintainer (opensearch-project#14567)

Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>

* Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517)

* Bump com.azure:azure-storage-common in /plugins/repository-azure

Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1)

---
updated-dependencies:
- dependency-name: com.azure:azure-storage-common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Add allowlist setting for search-pipeline-common processors (opensearch-project#14562)

Add a new static setting that lets an operator choose specific search
pipeline processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

A distinct setting exists for each of request, response, and search phase
results processors.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581)

(cherry picked from commit 0095fd1)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427)

* Add unittests for RemoteClusterStateAttributesManager

Signed-off-by: Shivansh Arora <hishiv@amazon.com>

* Add Ashish Singh to codeowners (opensearch-project#14592)

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554)

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

* Add @internalapi annotation to japicmp exclusions (opensearch-project#14597)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552)

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:update changelog

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

---------

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>

* Star tree mapping changes (opensearch-project#14605)

* Star tree mapping changes with feature flag
---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>

* OnHeap Star Tree Implementation

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressed nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressed major nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* includes Count Aggregator

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* handling for missing doc values

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressing review comments

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* rebasing with main

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* support for empty sequential doc values iterator

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* min and max star tree aggregators

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* star tree file formats

* Star tree codec changes

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Adding tests

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Addressing comments

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* addressing review comments

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Star tree merge changes

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* fix annotations

* star-tree file formats reader and javadoc fixes

* read for composite index values

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* doc values file format

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com>
Signed-off-by: kkewwei kkewwei@163.com
Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>
Signed-off-by: panguixin <panguixin@bytedance.com>
Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com>
Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>
Co-authored-by: kkewwei <kkewwei@163.com>
Co-authored-by: Marc Handalian <handalm@amazon.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: panguixin <panguixin@bytedance.com>
Co-authored-by: Bukhtawar Khan <bukhtawa@amazon.com>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Ashish Singh <ssashish@amazon.com>
Co-authored-by: Liyun Xiu <xiliyun@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
bharath-techie added a commit to bharath-techie/OpenSearch that referenced this issue Jul 11, 2024
@bharath-techie @bowenlan-amzn
@prudhvigodithi @sgup432 @kkewwei @mch2 @peteralfonsi @opensearch-trigger-bot @opensearch-ci-bot @andrross @bugmakerrrrrr @Bukhtawar @reta @dependabot @shiv0408 @ashking94 @chishui @imvtsl @dbwiddis @deshsidd @robinf95 @RS146BIJAY @soosinha @sachinpkale @sarthakaggarwal97
Startree file formats codec merge (#29)
c696370 
* Fix flaky test in range aggregation yaml test (opensearch-project#14486)

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Use CODECOV_TOKEN (opensearch-project#14536)

Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>

* [Tiered Caching]  Moving query recomputation logic outside of write lock (opensearch-project#14187)

* Moving query recompute out of write lock

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* [Tiered Caching] Moving query recomputation logic outside of write lock

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding java doc for the completable map

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Changes to call future handler only once per key

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing spotless check

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Added changelog

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comments

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing gradle fail

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comments to refactor unit test

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* minor UT refactor

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>

* Fix Flaky Test ClusterRerouteIT.testDelayWithALargeAmountOfShards (opensearch-project#14510)

Signed-off-by: kkewwei kkewwei@163.com

Signed-off-by: kkewwei kkewwei@163.com
Signed-off-by: kkewwei <kkewwei@163.com>

* Add doc for debugging rest tests (opensearch-project#14491)

* add doc for debugging rest tests

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Update TESTING.md

Co-authored-by: Marc Handalian <handalm@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* Address comment

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Marc Handalian <handalm@amazon.com>

* Fix flaky DefaultCacheStatsHolderTests (opensearch-project#14462)

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>

* [AUTO] [main] Add bwc version 2.15.1. (opensearch-project#14549)

* Add bwc version 2.15.1

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* Fix auto-generated version

Signed-off-by: Andrew Ross <andrross@amazon.com>

---------

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently (opensearch-project#14550)

* Fix flaky test TieredSpilloverCacheTests.testComputeIfAbsentConcurrently

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comment

Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>

* Add allowlist setting for ingest-common processors (opensearch-project#14479)

Add a new static setting that lets an operator choose specific ingest
processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Fix file cache initialization (opensearch-project#14004)

* fix file cache initialization

Signed-off-by: panguixin <panguixin@bytedance.com>

* changelog

Signed-off-by: panguixin <panguixin@bytedance.com>

* add test

Signed-off-by: panguixin <panguixin@bytedance.com>

---------

Signed-off-by: panguixin <panguixin@bytedance.com>

* Add Ashish Singh as maintainer (opensearch-project#14567)

Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>

* Allow @internalapi annotation on classes not meant to be constructed outside of the OpenSearch core (opensearch-project#14575)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 in /plugins/repository-azure (opensearch-project#14517)

* Bump com.azure:azure-storage-common in /plugins/repository-azure

Bumps [com.azure:azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.21.2 to 12.25.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-java/releases)
- [Commits](Azure/azure-sdk-for-java@azure-storage-common_12.21.2...azure-storage-blob_12.25.1)

---
updated-dependencies:
- dependency-name: com.azure:azure-storage-common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* Add allowlist setting for search-pipeline-common processors (opensearch-project#14562)

Add a new static setting that lets an operator choose specific search
pipeline processors to enable by name. The behavior is as follows:

- If the allowlist setting is not defined, all installed processors are
  enabled. This is the status quo.
- If the allowlist setting is defined as the empty set, then all processors
  are disabled.
- If the allowlist setting contains the names of valid processors, only those
  processors are enabled.
- If the allowlist setting contains a name of a processor that does not exist,
  then the server will fail to start with an IllegalStateException
  listing which processors were defined in the allowlist but are not
  installed.
- If the allowlist setting is changed between server restarts then any
  ingest pipeline using a now-disabled processor will fail. This is the
  same experience if a pipeline used a processor defined by a plugin but
  then that plugin were to be uninstalled across restarts.

A distinct setting exists for each of request, response, and search phase
results processors.

Related to opensearch-project#14439

Signed-off-by: Andrew Ross <andrross@amazon.com>

* Bump Apache Lucene to 9.11.1 (opensearch-project#14576) (opensearch-project#14581)

(cherry picked from commit 0095fd1)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Add unittests for RemoteClusterStateAttributesManager (opensearch-project#14427)

* Add unittests for RemoteClusterStateAttributesManager

Signed-off-by: Shivansh Arora <hishiv@amazon.com>

* Add Ashish Singh to codeowners (opensearch-project#14592)

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Add batching processor base type AbstractBatchingProcessor (opensearch-project#14554)

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

* Add @internalapi annotation to japicmp exclusions (opensearch-project#14597)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is mis… (opensearch-project#14552)

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Parsing a GetResult returns NPE if found field is missing. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix wildcart import. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:Fix spotless issues. Signed-off-by: Vatsal <vatsal.v.anand@gmail.com>

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Fix issue 14519:update changelog

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

---------

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>

* Star tree mapping changes (opensearch-project#14605)

* Star tree mapping changes with feature flag
---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>

* OnHeap Star Tree Implementation

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressed nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressed major nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* includes Count Aggregator

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* handling for missing doc values

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* addressing review comments

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* rebasing with main

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* support for empty sequential doc values iterator

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* nits

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* min and max star tree aggregators

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* star tree file formats

* Star tree codec changes

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Adding tests

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Addressing comments

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* addressing review comments

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* Star tree merge changes

Signed-off-by: Bharathwaj G <bharath78910@gmail.com>

* fix annotations

* star-tree file formats reader and javadoc fixes

* read for composite index values

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

* doc values file format

Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Prudhvi Godithi <pgodithi@amazon.com>
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sagar <99425694+sgup432@users.noreply.github.com>
Signed-off-by: kkewwei kkewwei@163.com
Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: Sagar Upadhyaya <upasagar@amazon.com>
Signed-off-by: panguixin <panguixin@bytedance.com>
Signed-off-by: Bukhtawar Khan <bukhtawa@amazon.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Prudhvi Godithi <pgodithi@amazon.com>
Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sagar Upadhyaya <upasagar@amazon.com>
Co-authored-by: kkewwei <kkewwei@163.com>
Co-authored-by: Marc Handalian <handalm@amazon.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-ci-bot <83309141+opensearch-ci-bot@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: panguixin <panguixin@bytedance.com>
Co-authored-by: Bukhtawar Khan <bukhtawa@amazon.com>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Ashish Singh <ssashish@amazon.com>
Co-authored-by: Liyun Xiu <xiliyun@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
harshavamsi pushed a commit to harshavamsi/OpenSearch that referenced this issue Jul 12, 2024
@deshsidd @robinf95 @harshavamsi
Refactoring FilterPath.parse by using an iterative approach instead o…
3b240be 
…f recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
amberzsy pushed a commit to amberzsy/OpenSearch-public that referenced this issue Jul 14, 2024
@deshsidd @robinf95 @zshuyi
Refactoring FilterPath.parse by using an iterative approach instead o…
2c4471b 
…f recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
mch2 pushed a commit that referenced this issue Jul 16, 2024
@kaushalmahi12 @dependabot
@peteralfonsi @imvtsl @deshsidd @robinf95 @RS146BIJAY @soosinha @sachinpkale @cwperks @bharath-techie
Add changes to propagate queryGroupId across child requests and nodes (…
d33d24e 
…#14614)

* add query group header propagator

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* apply spotless check

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add new propagator in ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* spotlessApply

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Correct typo in method name (#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (#14200)

* Refactor FilterPath parse function (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add PR link changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix assertion failure while deleting remote backed index (#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (#14635)

* Allow system index warning

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review comments

Signed-off-by: Craig Perkins <cwperx@amazon.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Star tree codec changes (#14514)

---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (#14672)

* Bump com.github.spullara.mustache.java:compiler

Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14.
- [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14)

---
updated-dependencies:
- dependency-name: com.github.spullara.mustache.java:compiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14673)

* Bump net.minidev:accessors-smart in /plugins/repository-azure

Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:accessors-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* move query group thread context propagator out of ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
opensearch-trigger-bot bot pushed a commit that referenced this issue Jul 16, 2024
@github-actions @dependabot
@peteralfonsi @imvtsl @deshsidd @robinf95 @RS146BIJAY @soosinha @sachinpkale @cwperks @bharath-techie
Add changes to propagate queryGroupId across child requests and nodes (…
874f4ed 
…#14614)

* add query group header propagator

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* apply spotless check

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add new propagator in ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* spotlessApply

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Correct typo in method name (#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (#14200)

* Refactor FilterPath parse function (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add PR link changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix assertion failure while deleting remote backed index (#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (#14635)

* Allow system index warning

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review comments

Signed-off-by: Craig Perkins <cwperx@amazon.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Star tree codec changes (#14514)

---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (#14672)

* Bump com.github.spullara.mustache.java:compiler

Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14.
- [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14)

---
updated-dependencies:
- dependency-name: com.github.spullara.mustache.java:compiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14673)

* Bump net.minidev:accessors-smart in /plugins/repository-azure

Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:accessors-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* move query group thread context propagator out of ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
(cherry picked from commit d33d24e)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
mch2 pushed a commit that referenced this issue Jul 22, 2024
@opensearch-trigger-bot @github-actions
@dependabot @peteralfonsi @imvtsl @deshsidd @robinf95 @RS146BIJAY @soosinha @sachinpkale @cwperks @bharath-techie
Add changes to propagate queryGroupId across child requests and nodes (…
7755055 
…#14614) (#14763)

* add query group header propagator



* apply spotless check



* add new propagator in ThreadContext



* spotlessApply



* address comments



* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...



* Updating SHAs



* Update changelog



---------






* [Bugfix] Fix ICacheKeySerializerTests flakiness (#14564)

* Fix testInvalidInput flakiness



* Addressed andrross's comment



* rerun security check



---------





* Correct typo in method name (#14621)




* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (#14200)

* Refactor FilterPath parse function (#12067)


* Implement unit tests for FilterPathTests (#12067)


* Write warn log if Filter is empty; Add comments (#12067)


* Add changelog



* Remove unnecessary log statement



* Remove unused logger



* Spotless apply



* Remove incorrect changelog



---------





* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(#14612)




* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings




* add changelog



* add PR link changelog



* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (#14613)




* Fix assertion failure while deleting remote backed index (#14601)




* Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (#14635)

* Allow system index warning



* Add to CHANGELOG



* Address code review comments



---------




* Star tree codec changes (#14514)

---------



* Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (#14672)

* Bump com.github.spullara.mustache.java:compiler

Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14.
- [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14)

---
updated-dependencies:
- dependency-name: com.github.spullara.mustache.java:compiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...



* Updating SHAs



* Update changelog



---------






* Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14673)

* Bump net.minidev:accessors-smart in /plugins/repository-azure

Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:accessors-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...



* Updating SHAs



* Update changelog



---------






* move query group thread context propagator out of ThreadContext



---------






















(cherry picked from commit d33d24e)

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
kaushalmahi12 added a commit to kaushalmahi12/OpenSearch that referenced this issue Jul 23, 2024
@kaushalmahi12 @dependabot
@peteralfonsi @imvtsl @deshsidd @robinf95 @RS146BIJAY @soosinha @sachinpkale @cwperks @bharath-techie
Add changes to propagate queryGroupId across child requests and nodes (…
5991a63 
…opensearch-project#14614)

* add query group header propagator

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* apply spotless check

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add new propagator in ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* spotlessApply

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add PR link changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (opensearch-project#14635)

* Allow system index warning

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review comments

Signed-off-by: Craig Perkins <cwperx@amazon.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Star tree codec changes (opensearch-project#14514)

---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (opensearch-project#14672)

* Bump com.github.spullara.mustache.java:compiler

Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14.
- [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14)

---
updated-dependencies:
- dependency-name: com.github.spullara.mustache.java:compiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (opensearch-project#14673)

* Bump net.minidev:accessors-smart in /plugins/repository-azure

Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:accessors-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* move query group thread context propagator out of ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
kaushalmahi12 added a commit to kaushalmahi12/OpenSearch that referenced this issue Jul 23, 2024
@kaushalmahi12 @dependabot
@peteralfonsi @imvtsl @deshsidd @robinf95 @RS146BIJAY @soosinha @sachinpkale @cwperks @bharath-techie
Add changes to propagate queryGroupId across child requests and nodes (…
0d70e36 
…opensearch-project#14614)

* add query group header propagator

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* apply spotless check

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add new propagator in ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* spotlessApply

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add PR link changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (opensearch-project#14635)

* Allow system index warning

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review comments

Signed-off-by: Craig Perkins <cwperx@amazon.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Star tree codec changes (opensearch-project#14514)

---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (opensearch-project#14672)

* Bump com.github.spullara.mustache.java:compiler

Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14.
- [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14)

---
updated-dependencies:
- dependency-name: com.github.spullara.mustache.java:compiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (opensearch-project#14673)

* Bump net.minidev:accessors-smart in /plugins/repository-azure

Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:accessors-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* move query group thread context propagator out of ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
kkewwei pushed a commit to kkewwei/OpenSearch that referenced this issue Jul 24, 2024
@opensearch-trigger-bot @github-actions
@robinf95 @kkewwei
Refactoring FilterPath.parse by using an iterative approach instead o…
c79ab10 
…f recursion. (opensearch-project#14200) (opensearch-project#14629)

* Refactor FilterPath parse function (opensearch-project#12067)

* Implement unit tests for FilterPathTests (opensearch-project#12067)

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)

* Add changelog

* Remove unnecessary log statement

* Remove unused logger

* Spotless apply

* Remove incorrect changelog

---------

(cherry picked from commit 0742453)

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Signed-off-by: kkewwei <kkewwei@163.com>
kkewwei pushed a commit to kkewwei/OpenSearch that referenced this issue Jul 24, 2024
@opensearch-trigger-bot @github-actions
@dependabot @peteralfonsi @imvtsl @deshsidd @robinf95 @RS146BIJAY @soosinha @sachinpkale @cwperks @bharath-techie @kkewwei
Add changes to propagate queryGroupId across child requests and nodes (…
48b29b1 
…opensearch-project#14614) (opensearch-project#14763)

* add query group header propagator

* apply spotless check

* add new propagator in ThreadContext

* spotlessApply

* address comments

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

* Updating SHAs

* Update changelog

---------

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

* Addressed andrross's comment

* rerun security check

---------

* Correct typo in method name (opensearch-project#14621)

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)

* Implement unit tests for FilterPathTests (opensearch-project#12067)

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)

* Add changelog

* Remove unnecessary log statement

* Remove unused logger

* Spotless apply

* Remove incorrect changelog

---------

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

* add changelog

* add PR link changelog

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

* Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (opensearch-project#14635)

* Allow system index warning

* Add to CHANGELOG

* Address code review comments

---------

* Star tree codec changes (opensearch-project#14514)

---------

* Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (opensearch-project#14672)

* Bump com.github.spullara.mustache.java:compiler

Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14.
- [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14)

---
updated-dependencies:
- dependency-name: com.github.spullara.mustache.java:compiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

* Updating SHAs

* Update changelog

---------

* Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (opensearch-project#14673)

* Bump net.minidev:accessors-smart in /plugins/repository-azure

Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:accessors-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

* Updating SHAs

* Update changelog

---------

* move query group thread context propagator out of ThreadContext

---------

(cherry picked from commit d33d24e)

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: kkewwei <kkewwei@163.com>
mch2 pushed a commit that referenced this issue Jul 30, 2024
@kaushalmahi12 @shiv0408
@Arpit-Bandejiya @gaobinlong @RS146BIJAY @ansjcy @cwperks @sandeshkr419 @reta @zelinh @lukas-vlcek @kkewwei @SwethaGuptha @soosinha @ahmedsobeh @dependabot @opensearch-trigger-bot @github-actions @deshsidd @peteralfonsi @imvtsl @robinf95 @sachinpkale @bharath-techie @shourya035 @mgodwan @sarthakaggarwal97 @sgup432 @skumawat2025 @DaniilRoman @rishabh6788 @dblock @ashking94 @bowenlan-amzn @Pranshu-S @rajiv-kv @sumitasr @dbwiddis @shailendra0811 @imyp92 @ebraminio @chishui @finnegancarroll @neetikasinghal @jed326 @gbbafna @andrross
Add queryGroupId to search workload tasks at co-ordinator and data no…
eb306d2 
…de level (#14708)

* add logic to add headers to Task

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add logic to add queryGroupId to task headers

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove redundant code

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add changelog entry

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix precommit

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add UTs for RemoteIndexMetadataManager (#14660)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes (#10959)

* Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes
Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Add more test

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* modify change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Fix test failure

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Change the indexAnalyzer used by prefix field

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Skip old version for yaml test

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Fix test failure

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify yaml test description

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Remove the name parameter for setAnalyzer()

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Offline calculation of total shard per node and caching it for weight calculation inside LocalShardBalancer (#14675)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [bug fix] validate lower bound for top n size (#14587)

Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Create SystemIndexRegistry with helper method matchesSystemIndex (#14415)

* Create new extension point in SystemIndexPlugin for a single plugin to get registered system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* WIP on system indices from IndexNameExpressionResolver

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add test in IndexNameExpressionResolverTests

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Remove changes in SystemIndexPlugin

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add method in IndexNameExpressionResolver to get matching system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Show how resolver can be chained to get system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Fix forbiddenApis check

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Update CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Make SystemIndices internal

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Remove unneeded changes

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Fix CI failures

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Fix precommit errors

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Use Regex instead of WildcardMatcher

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review feedback

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Allow caller to pass index expressions

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Create SystemIndexRegistry

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Update CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Remove singleton limitation

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add javadoc

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add @experimentalapi annotation

Signed-off-by: Craig Perkins <cwperx@amazon.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactor Grok validate pattern to iterative approach (#14206)

* grok validate patterns recusrion to iterative

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* Add max depth in resolving a pattern to avoid OOM

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* change path from deque to arraylist

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* rename queue to stack

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* Change max depth to 500

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* typo originPatternName fix

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* spotless

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

---------

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump opentelemetry from 1.39.0 to 1.40.0 (#14674)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump jackson from 2.17.1 to 2.17.2 (#14687)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add release notes for release 1.3.18 (#14699)

Signed-off-by: Zelin Hao <zelinhao@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump reactor from 3.5.19 to 3.5.20 (#14697)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add unit tests for read flow of RemoteClusterStateService and bug fix for transient settings (#14476)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Update version check for the bug fix of match_phrase_prefix_query not working on text field with multiple values and index_prefixes (#14703)

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Remove unnecessary cast to int from test (#14696)

Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* print reason why parent task was cancelled (#14604)

Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use set of shard routing for shard in unassigned shard batch check. (#14533)

Signed-off-by: Swetha Guptha <gupthasg@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add versioning for UploadedIndexMetadata (#14677)

* Add versioning for UploadedIndexMetadata
* Handle componentPrefix for backward compatibility

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix: update help output for _cat (#14722)

* fixed help output for _cat

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>

* updated changelog

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>

* updated changelog

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>

---------

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix hdfs-fixture kerb-admin & hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (#14729)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Update to Gradle 8.9 (#14574)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix hdfs-fixture hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (#14732)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add `strict_allow_templates` dynamic mapping option (#14555)

* The dynamic mapping parameter supports strict_allow_templates

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify skip version in yml test file

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Refactor some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Keep the old methods

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* change public to private

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Do not override toString method for Dynamic

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code and modify the changelog

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump net.minidev:json-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14748)

* Bump net.minidev:json-smart in /plugins/repository-azure

Bumps [net.minidev:json-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:json-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove query insights plugin from core (#14743)

Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add `strict_allow_templates` dynamic mapping option (#14555) (#14737) (#14742)

* The dynamic mapping parameter supports strict_allow_templates

* Modify change log

* Modify skip version in yml test file

* Refactor some code

* Keep the old methods

* change public to private

* Optimize some code

* Do not override toString method for Dynamic

* Optimize some code and modify the changelog

---------

(cherry picked from commit 6b8b3ef)

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix create or update alias API doesn't throw exception for unsupported parameters (#14719)

* Fix create or update alias API doesn't throw exception for unsupported parameters

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Update version check in yml test

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* modify change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Remove query categorization from core (#14759)

* Remove query categorization from core

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Trigger Build

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add changes to propagate queryGroupId across child requests and nodes (#14614)

* add query group header propagator

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* apply spotless check

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add new propagator in ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* spotlessApply

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Correct typo in method name (#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (#14200)

* Refactor FilterPath parse function (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add PR link changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix assertion failure while deleting remote backed index (#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (#14635)

* Allow system index warning

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review comments

Signed-off-by: Craig Perkins <cwperx@amazon.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Star tree codec changes (#14514)

---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (#14672)

* Bump com.github.spullara.mustache.java:compiler

Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14.
- [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14)

---
updated-dependencies:
- dependency-name: com.github.spullara.mustache.java:compiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14673)

* Bump net.minidev:accessors-smart in /plugins/repository-azure

Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:accessors-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* move query group thread context propagator out of ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add consumers to remote store based index settings (#14764)

Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add matchesPluginSystemIndexPattern to SystemIndexRegistry (#14750)

* Add matchesPluginSystemIndexPattern to SystemIndexRegistry

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Use single data structure to keep track of system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review comments

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add test for getAllDescriptors

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Update server/src/main/java/org/opensearch/indices/SystemIndexRegistry.java

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Craig Perkins <craig5008@gmail.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <craig5008@gmail.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* SPI for loading ABC templates (#14659)

* SPI for loading ABC templates

Signed-off-by: mgodwan <mgodwan@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (#12891)

* Fix bulk upsert ignores the default_pipeline and final_pipeline when auto-created index matches with the index template

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify changelog & comment

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Use new approach

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Fix test failure

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix flaky test due to node being used across all tests (#14787)

Signed-off-by: Mohit Godwani <mgodwan@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Star Tree Implementation [OnHeap] (#14512)

---------
Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add Gao Binlong as maintainer (#14796)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Clear ehcache disk cache files during initialization (#14738)

* Clear ehcache disk cache files during initialization

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding UT to fix line coverage

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comment

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding more Uts for better line coverage

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Throwing exception in case we fail to clear cache files during startup

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding more UTs

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding a UT for more coverage

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing gradle build

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Update ehcache disk cache close() logic

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactor remote-routing-table service inline with remote state interfaces (#14668)

---------
Signed-off-by: Arpit Bandejiya <abandeji@amazon.com>
Signed-off-by: Arpit-Bandejiya <abandeji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Set version to 2.15 for determining metadata during migration to remote store

Signed-off-by: Sandeep Kumawat <skumwt@amazon.com>
Co-authored-by: Sandeep Kumawat <skumwt@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (#14793)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix create or update alias API doesn't throw exception for unsupported parameters (#14769)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Change RCSS info logs to debug (#14814)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Bugfix] Fix NPE in ReplicaShardAllocator (#13993) (#14385)

* [Bugfix] Fix NPE in ReplicaShardAllocator (#13993)

Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com>

* Add fix info to CHANGELOG.md

Signed-off-by: Daniil Roman <danroman17397@gmail.com>

---------

Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com>
Signed-off-by: Daniil Roman <danroman17397@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Run performance benchmark on pull requests (#14760)

* add performance benchmark workflow for pull requests

Signed-off-by: Rishabh Singh <sngri@amazon.com>

* Update PERFORMANCE_BENCHMARKS.md

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update PERFORMANCE_BENCHMARKS.md

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

---------

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix constant_keyword field type (#14807)

Signed-off-by: kkewwei <kkewwei@163.com>

test

Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Remote Store Migration] Reconcile remote store based index settings during STRICT mode switch (#14792)

Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add prefix mode verification setting for repository verification (#14790)

* Add prefix mode verification setting for repository verification

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Add UTs and randomise prefix mode repository verification

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Incorporate PR review feedback

Signed-off-by: Ashish Singh <ssashish@amazon.com>

---------

Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add length check on comment body for benchmark workflow (#14834)

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add restore-from-snapshot test procedure for snapshot run benchmark config (#14842)

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix env variable name typo (#14843)

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use circuit breaker in InternalHistogram when adding empty buckets (#14754)

* introduce circuit breaker in InternalHistogram

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* use circuit breaker from reduce context

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* add test

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* revert use_real_memory change in OpenSearchNode

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* add change log

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Remote State] Create interface RemoteEntitiesManager (#14671)

* Create interface RemoteEntitiesManager

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Optimise TransportNodesAction to not send DiscoveryNodes for NodeStat… (#14749)

* Optimize TransportNodesAction to not send DiscoveryNodes for NodeStats, NodesInfo and ClusterStats call

Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Enabling term version check on local state for all ClusterManager Read Transport Actions (#14273)

* enabling term version check on local state for all admin read actions

Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Reduce logging in DEBUG for MasterService:run (#14795)

* Reduce logging in DEBUG for MasteService:run by introducing short and long summary in Taskbatcher

Signed-off-by: Sumit Bansal <sumitsb@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add SplitResponseProcessor to Search Pipelines (#14800)

* Add SplitResponseProcessor for search pipelines

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Register the split processor factory

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Address code review comments

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Avoid list copy by casting array

Signed-off-by: Daniel Widdis <widdis@gmail.com>

---------

Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add integration tests for RemoteRoutingTable Service. (#14631)

Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add SortResponseProcessor to Search Pipelines (#14785)

* Add SortResponseProcessor for search pipelines

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add stupid and unnecessary javadocs to satisfy overly strict CI

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Split casting and sorting methods for readability

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Register the sort processor factory

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Address code review comments

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Cast individual list elements to avoid creating two lists

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add yamlRestTests

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Clarify why there's unusual sorting

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Use instanceof instead of isAssignableFrom

Signed-off-by: Daniel Widdis <widdis@gmail.com>

---------

Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix allowUnmappedFields, mapUnmappedFieldAsString settings to be applied when parsing query string query (#13957)

* Modify to invoke QueryShardContext.fieldMapper() method to apply allowUnmappedFields and mapUnmappedFieldAsString settings

Signed-off-by: imyp92 <pyw5420@gmail.com>

* Add test cases to verify returning 400 responses if unmapped fields are included for some types of query

Signed-off-by: imyp92 <pyw5420@gmail.com>

* Add changelog

Signed-off-by: imyp92 <pyw5420@gmail.com>

---------

Signed-off-by: imyp92 <pyw5420@gmail.com>
Signed-off-by: gaobinlong <gbinlong@amazon.com>
Co-authored-by: gaobinlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.16.0 to 1.16.1 in /plugins/repository-azure (#14857)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.16.0 to 1.16.1.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.16.0...v1.16.1)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.gradle.develocity from 3.17.5 to 3.17.6 (#14856)

* Bump com.gradle.develocity from 3.17.5 to 3.17.6

Bumps com.gradle.develocity from 3.17.5 to 3.17.6.

---
updated-dependencies:
- dependency-name: com.gradle.develocity
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump org.jline:jline in /test/fixtures/hdfs-fixture (#14859)

Bumps [org.jline:jline](https://github.com/jline/jline3) from 3.26.2 to 3.26.3.
- [Release notes](https://github.com/jline/jline3/releases)
- [Changelog](https://github.com/jline/jline3/blob/master/changelog.md)
- [Commits](jline/jline3@jline-parent-3.26.2...jline-parent-3.26.3)

---
updated-dependencies:
- dependency-name: org.jline:jline
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use Lucene provided Persian stem (#14847)

Lucene provided Persian stem apparently isn't hooked yet and this change is
doing that based on what is done for Arabic stem support.

Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org>
Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump actions/checkout from 2 to 4 (#14858)

* Bump actions/checkout from 2 to 4

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v2...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Deprecate batch_size parameter on bulk API (#14725)

By default the full _bulk payload will be passed to ingest processors as a
batch, with any sub batching logic to be implemented by each processor if
necessary.

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add perms for remote snapshot cache eviction on scripted query (#14411)

Signed-off-by: Finn Carroll <carrofin@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add transport interceptor to populate queryGroupId in task headers

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add rest, transport layer changes for Hot to warm tiering - dedicated setup (#13980)

Signed-off-by: Neetika Singhal <neetiks@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Create listener to refresh search thread resource usage (#14832)

* [bug fix] fix incorrect coordinator node search resource usages

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* fix bug on serialization when passing task resource usage to coordinator

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* add more unit tests

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* remove query insights plugin related code

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* create per request listener to refresh task resource usage

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* Make new listener API public

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove wrong files added

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Address review comments

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Build fix

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Make singleton

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Address review comments

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Make sure listener runs before plugin listeners

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Minor fix

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: Jay Deng <jayd0104@gmail.com>
Co-authored-by: Chenyang Ji <cyji@amazon.com>
Co-authored-by: Jay Deng <jayd0104@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Caching avg total bytes and avg free bytes inside ClusterInfo (#14851)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use default value when index.number_of_replicas is null (#14812)

* Use default value when index.number_of_replicas is null

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

* Add integration test

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

* Add changelog

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

---------

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Remote Routing Table] Implement write and read flow for shard diff file. (#14684)

* Implement write and read flow to upload/download shard diff file.

Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Optimized ClusterStatsIndices to precomute shard stats (#14426)

* Optimize Cluster Stats Indices to precomute node level stats

Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix constraint bug which allows more primary shards than average primary shards per index (#14908)

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Optmising AwarenessAllocationDecider for hashmap.get call (#14761)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* update comment

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix IngestServiceTests.testBulkRequestExecutionWithFailures (#14918)

The test would previously fail if the randomness led to only a single
indexing request being included in the bulk payload. This change
guarantees multiple indexing requests in order to ensure the batch logic
kicks in.

Also replace some unneeded mocks with real classes.

Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add queryGroupTask

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove unnecessary imports

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add QueryGroupTask tests

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* rename WLM transport request handler

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add CHANGELOG entry

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix ut

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix UT to remove the verify for final method

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* apply spotless

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Zelin Hao <zelinhao@amazon.com>
Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Swetha Guptha <gupthasg@amazon.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Signed-off-by: Craig Perkins <craig5008@gmail.com>
Signed-off-by: mgodwan <mgodwan@amazon.com>
Signed-off-by: Mohit Godwani <mgodwan@amazon.com>
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sandeep Kumawat <skumwt@amazon.com>
Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com>
Signed-off-by: Daniil Roman <danroman17397@gmail.com>
Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>
Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com>
Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com>
Signed-off-by: Sumit Bansal <sumitsb@amazon.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
Signed-off-by: imyp92 <pyw5420@gmail.com>
Signed-off-by: gaobinlong <gbinlong@amazon.com>
Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org>
Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: Finn Carroll <carrofin@amazon.com>
Signed-off-by: Neetika Singhal <neetiks@amazon.com>
Signed-off-by: Jay Deng <jayd0104@gmail.com>
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com>
Co-authored-by: gaobinlong <gbinlong@amazon.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Chenyang Ji <cyji@amazon.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Sandesh Kumar <sandeshkr419@gmail.com>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: Zelin Hao <zelinhao@amazon.com>
Co-authored-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Co-authored-by: kkewwei <kkewwei@163.com>
Co-authored-by: SwethaGuptha <156877431+SwethaGuptha@users.noreply.github.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Ahmed Sobeh <ahmed.sobeh@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
Co-authored-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Co-authored-by: Craig Perkins <craig5008@gmail.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Co-authored-by: Mohit Godwani <81609427+mgodwan@users.noreply.github.com>
Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sandeep Kumawat <2025sandeepkumawat@gmail.com>
Co-authored-by: Sandeep Kumawat <skumwt@amazon.com>
Co-authored-by: Daniil Roman <danroman17397@gmail.com>
Co-authored-by: Rishabh Singh <rishabhksingh@gmail.com>
Co-authored-by: kkewwei <kewei.11@bytedance.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: Ashish Singh <ssashish@amazon.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Pranshu Shukla <55992439+Pranshu-S@users.noreply.github.com>
Co-authored-by: rajiv-kv <157019998+rajiv-kv@users.noreply.github.com>
Co-authored-by: Sumit Bansal <sumit.asr@gmail.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: shailendra0811 <167273922+shailendra0811@users.noreply.github.com>
Co-authored-by: Park, Yeongwu <pyw5420@gmail.com>
Co-authored-by: ebraminio <ebraminio@gmail.com>
Co-authored-by: Liyun Xiu <xiliyun@amazon.com>
Co-authored-by: Finn <carrofin@amazon.com>
Co-authored-by: Neetika Singhal <neetiks@amazon.com>
Co-authored-by: Jay Deng <jayd0104@gmail.com>
Co-authored-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>
opensearch-trigger-bot bot added a commit that referenced this issue Jul 30, 2024
@github-actions @shiv0408
@Arpit-Bandejiya @gaobinlong @RS146BIJAY @ansjcy @cwperks @sandeshkr419 @reta @zelinh @lukas-vlcek @kkewwei @SwethaGuptha @soosinha @ahmedsobeh @dependabot @opensearch-trigger-bot @deshsidd @peteralfonsi @imvtsl @robinf95 @sachinpkale @bharath-techie @shourya035 @mgodwan @sarthakaggarwal97 @sgup432 @skumawat2025 @DaniilRoman @rishabh6788 @dblock @ashking94 @bowenlan-amzn @Pranshu-S @rajiv-kv @sumitasr @dbwiddis @shailendra0811 @imyp92 @ebraminio @chishui @finnegancarroll @neetikasinghal @jed326 @gbbafna @andrross
Add queryGroupId to search workload tasks at co-ordinator and data no…
4b2dfe7 
…de level (#14708)

* add logic to add headers to Task

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add logic to add queryGroupId to task headers

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove redundant code

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add changelog entry

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix precommit

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add UTs for RemoteIndexMetadataManager (#14660)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes (#10959)

* Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes
Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Add more test

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* modify change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Fix test failure

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Change the indexAnalyzer used by prefix field

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Skip old version for yaml test

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Fix test failure

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify yaml test description

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Remove the name parameter for setAnalyzer()

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Offline calculation of total shard per node and caching it for weight calculation inside LocalShardBalancer (#14675)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [bug fix] validate lower bound for top n size (#14587)

Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Create SystemIndexRegistry with helper method matchesSystemIndex (#14415)

* Create new extension point in SystemIndexPlugin for a single plugin to get registered system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* WIP on system indices from IndexNameExpressionResolver

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add test in IndexNameExpressionResolverTests

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Remove changes in SystemIndexPlugin

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add method in IndexNameExpressionResolver to get matching system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Show how resolver can be chained to get system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Fix forbiddenApis check

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Update CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Make SystemIndices internal

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Remove unneeded changes

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Fix CI failures

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Fix precommit errors

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Use Regex instead of WildcardMatcher

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review feedback

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Allow caller to pass index expressions

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Create SystemIndexRegistry

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Update CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Remove singleton limitation

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add javadoc

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add @experimentalapi annotation

Signed-off-by: Craig Perkins <cwperx@amazon.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactor Grok validate pattern to iterative approach (#14206)

* grok validate patterns recusrion to iterative

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* Add max depth in resolving a pattern to avoid OOM

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* change path from deque to arraylist

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* rename queue to stack

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* Change max depth to 500

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* typo originPatternName fix

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* spotless

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

---------

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump opentelemetry from 1.39.0 to 1.40.0 (#14674)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump jackson from 2.17.1 to 2.17.2 (#14687)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add release notes for release 1.3.18 (#14699)

Signed-off-by: Zelin Hao <zelinhao@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump reactor from 3.5.19 to 3.5.20 (#14697)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add unit tests for read flow of RemoteClusterStateService and bug fix for transient settings (#14476)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Update version check for the bug fix of match_phrase_prefix_query not working on text field with multiple values and index_prefixes (#14703)

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Remove unnecessary cast to int from test (#14696)

Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* print reason why parent task was cancelled (#14604)

Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use set of shard routing for shard in unassigned shard batch check. (#14533)

Signed-off-by: Swetha Guptha <gupthasg@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add versioning for UploadedIndexMetadata (#14677)

* Add versioning for UploadedIndexMetadata
* Handle componentPrefix for backward compatibility

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix: update help output for _cat (#14722)

* fixed help output for _cat

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>

* updated changelog

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>

* updated changelog

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>

---------

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix hdfs-fixture kerb-admin & hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (#14729)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Update to Gradle 8.9 (#14574)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix hdfs-fixture hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (#14732)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add `strict_allow_templates` dynamic mapping option (#14555)

* The dynamic mapping parameter supports strict_allow_templates

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify skip version in yml test file

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Refactor some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Keep the old methods

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* change public to private

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Do not override toString method for Dynamic

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code and modify the changelog

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump net.minidev:json-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14748)

* Bump net.minidev:json-smart in /plugins/repository-azure

Bumps [net.minidev:json-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:json-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove query insights plugin from core (#14743)

Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add `strict_allow_templates` dynamic mapping option (#14555) (#14737) (#14742)

* The dynamic mapping parameter supports strict_allow_templates

* Modify change log

* Modify skip version in yml test file

* Refactor some code

* Keep the old methods

* change public to private

* Optimize some code

* Do not override toString method for Dynamic

* Optimize some code and modify the changelog

---------

(cherry picked from commit 6b8b3ef)

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix create or update alias API doesn't throw exception for unsupported parameters (#14719)

* Fix create or update alias API doesn't throw exception for unsupported parameters

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Update version check in yml test

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* modify change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Remove query categorization from core (#14759)

* Remove query categorization from core

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Trigger Build

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add changes to propagate queryGroupId across child requests and nodes (#14614)

* add query group header propagator

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* apply spotless check

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add new propagator in ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* spotlessApply

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Correct typo in method name (#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (#14200)

* Refactor FilterPath parse function (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add PR link changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix assertion failure while deleting remote backed index (#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (#14635)

* Allow system index warning

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review comments

Signed-off-by: Craig Perkins <cwperx@amazon.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Star tree codec changes (#14514)

---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (#14672)

* Bump com.github.spullara.mustache.java:compiler

Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14.
- [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14)

---
updated-dependencies:
- dependency-name: com.github.spullara.mustache.java:compiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14673)

* Bump net.minidev:accessors-smart in /plugins/repository-azure

Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:accessors-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* move query group thread context propagator out of ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add consumers to remote store based index settings (#14764)

Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add matchesPluginSystemIndexPattern to SystemIndexRegistry (#14750)

* Add matchesPluginSystemIndexPattern to SystemIndexRegistry

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Use single data structure to keep track of system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review comments

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add test for getAllDescriptors

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Update server/src/main/java/org/opensearch/indices/SystemIndexRegistry.java

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Craig Perkins <craig5008@gmail.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <craig5008@gmail.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* SPI for loading ABC templates (#14659)

* SPI for loading ABC templates

Signed-off-by: mgodwan <mgodwan@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (#12891)

* Fix bulk upsert ignores the default_pipeline and final_pipeline when auto-created index matches with the index template

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify changelog & comment

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Use new approach

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Fix test failure

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix flaky test due to node being used across all tests (#14787)

Signed-off-by: Mohit Godwani <mgodwan@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Star Tree Implementation [OnHeap] (#14512)

---------
Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add Gao Binlong as maintainer (#14796)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Clear ehcache disk cache files during initialization (#14738)

* Clear ehcache disk cache files during initialization

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding UT to fix line coverage

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comment

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding more Uts for better line coverage

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Throwing exception in case we fail to clear cache files during startup

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding more UTs

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding a UT for more coverage

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing gradle build

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Update ehcache disk cache close() logic

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactor remote-routing-table service inline with remote state interfaces (#14668)

---------
Signed-off-by: Arpit Bandejiya <abandeji@amazon.com>
Signed-off-by: Arpit-Bandejiya <abandeji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Set version to 2.15 for determining metadata during migration to remote store

Signed-off-by: Sandeep Kumawat <skumwt@amazon.com>
Co-authored-by: Sandeep Kumawat <skumwt@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (#14793)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix create or update alias API doesn't throw exception for unsupported parameters (#14769)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Change RCSS info logs to debug (#14814)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Bugfix] Fix NPE in ReplicaShardAllocator (#13993) (#14385)

* [Bugfix] Fix NPE in ReplicaShardAllocator (#13993)

Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com>

* Add fix info to CHANGELOG.md

Signed-off-by: Daniil Roman <danroman17397@gmail.com>

---------

Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com>
Signed-off-by: Daniil Roman <danroman17397@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Run performance benchmark on pull requests (#14760)

* add performance benchmark workflow for pull requests

Signed-off-by: Rishabh Singh <sngri@amazon.com>

* Update PERFORMANCE_BENCHMARKS.md

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update PERFORMANCE_BENCHMARKS.md

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

---------

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix constant_keyword field type (#14807)

Signed-off-by: kkewwei <kkewwei@163.com>

test

Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Remote Store Migration] Reconcile remote store based index settings during STRICT mode switch (#14792)

Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add prefix mode verification setting for repository verification (#14790)

* Add prefix mode verification setting for repository verification

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Add UTs and randomise prefix mode repository verification

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Incorporate PR review feedback

Signed-off-by: Ashish Singh <ssashish@amazon.com>

---------

Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add length check on comment body for benchmark workflow (#14834)

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add restore-from-snapshot test procedure for snapshot run benchmark config (#14842)

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix env variable name typo (#14843)

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use circuit breaker in InternalHistogram when adding empty buckets (#14754)

* introduce circuit breaker in InternalHistogram

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* use circuit breaker from reduce context

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* add test

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* revert use_real_memory change in OpenSearchNode

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* add change log

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Remote State] Create interface RemoteEntitiesManager (#14671)

* Create interface RemoteEntitiesManager

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Optimise TransportNodesAction to not send DiscoveryNodes for NodeStat… (#14749)

* Optimize TransportNodesAction to not send DiscoveryNodes for NodeStats, NodesInfo and ClusterStats call

Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Enabling term version check on local state for all ClusterManager Read Transport Actions (#14273)

* enabling term version check on local state for all admin read actions

Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Reduce logging in DEBUG for MasterService:run (#14795)

* Reduce logging in DEBUG for MasteService:run by introducing short and long summary in Taskbatcher

Signed-off-by: Sumit Bansal <sumitsb@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add SplitResponseProcessor to Search Pipelines (#14800)

* Add SplitResponseProcessor for search pipelines

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Register the split processor factory

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Address code review comments

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Avoid list copy by casting array

Signed-off-by: Daniel Widdis <widdis@gmail.com>

---------

Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add integration tests for RemoteRoutingTable Service. (#14631)

Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add SortResponseProcessor to Search Pipelines (#14785)

* Add SortResponseProcessor for search pipelines

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add stupid and unnecessary javadocs to satisfy overly strict CI

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Split casting and sorting methods for readability

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Register the sort processor factory

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Address code review comments

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Cast individual list elements to avoid creating two lists

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add yamlRestTests

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Clarify why there's unusual sorting

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Use instanceof instead of isAssignableFrom

Signed-off-by: Daniel Widdis <widdis@gmail.com>

---------

Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix allowUnmappedFields, mapUnmappedFieldAsString settings to be applied when parsing query string query (#13957)

* Modify to invoke QueryShardContext.fieldMapper() method to apply allowUnmappedFields and mapUnmappedFieldAsString settings

Signed-off-by: imyp92 <pyw5420@gmail.com>

* Add test cases to verify returning 400 responses if unmapped fields are included for some types of query

Signed-off-by: imyp92 <pyw5420@gmail.com>

* Add changelog

Signed-off-by: imyp92 <pyw5420@gmail.com>

---------

Signed-off-by: imyp92 <pyw5420@gmail.com>
Signed-off-by: gaobinlong <gbinlong@amazon.com>
Co-authored-by: gaobinlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.16.0 to 1.16.1 in /plugins/repository-azure (#14857)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.16.0 to 1.16.1.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.16.0...v1.16.1)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.gradle.develocity from 3.17.5 to 3.17.6 (#14856)

* Bump com.gradle.develocity from 3.17.5 to 3.17.6

Bumps com.gradle.develocity from 3.17.5 to 3.17.6.

---
updated-dependencies:
- dependency-name: com.gradle.develocity
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump org.jline:jline in /test/fixtures/hdfs-fixture (#14859)

Bumps [org.jline:jline](https://github.com/jline/jline3) from 3.26.2 to 3.26.3.
- [Release notes](https://github.com/jline/jline3/releases)
- [Changelog](https://github.com/jline/jline3/blob/master/changelog.md)
- [Commits](jline/jline3@jline-parent-3.26.2...jline-parent-3.26.3)

---
updated-dependencies:
- dependency-name: org.jline:jline
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use Lucene provided Persian stem (#14847)

Lucene provided Persian stem apparently isn't hooked yet and this change is
doing that based on what is done for Arabic stem support.

Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org>
Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump actions/checkout from 2 to 4 (#14858)

* Bump actions/checkout from 2 to 4

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v2...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Deprecate batch_size parameter on bulk API (#14725)

By default the full _bulk payload will be passed to ingest processors as a
batch, with any sub batching logic to be implemented by each processor if
necessary.

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add perms for remote snapshot cache eviction on scripted query (#14411)

Signed-off-by: Finn Carroll <carrofin@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add transport interceptor to populate queryGroupId in task headers

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add rest, transport layer changes for Hot to warm tiering - dedicated setup (#13980)

Signed-off-by: Neetika Singhal <neetiks@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Create listener to refresh search thread resource usage (#14832)

* [bug fix] fix incorrect coordinator node search resource usages

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* fix bug on serialization when passing task resource usage to coordinator

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* add more unit tests

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* remove query insights plugin related code

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* create per request listener to refresh task resource usage

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* Make new listener API public

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove wrong files added

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Address review comments

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Build fix

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Make singleton

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Address review comments

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Make sure listener runs before plugin listeners

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Minor fix

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: Jay Deng <jayd0104@gmail.com>
Co-authored-by: Chenyang Ji <cyji@amazon.com>
Co-authored-by: Jay Deng <jayd0104@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Caching avg total bytes and avg free bytes inside ClusterInfo (#14851)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use default value when index.number_of_replicas is null (#14812)

* Use default value when index.number_of_replicas is null

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

* Add integration test

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

* Add changelog

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

---------

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Remote Routing Table] Implement write and read flow for shard diff file. (#14684)

* Implement write and read flow to upload/download shard diff file.

Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Optimized ClusterStatsIndices to precomute shard stats (#14426)

* Optimize Cluster Stats Indices to precomute node level stats

Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix constraint bug which allows more primary shards than average primary shards per index (#14908)

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Optmising AwarenessAllocationDecider for hashmap.get call (#14761)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* update comment

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix IngestServiceTests.testBulkRequestExecutionWithFailures (#14918)

The test would previously fail if the randomness led to only a single
indexing request being included in the bulk payload. This change
guarantees multiple indexing requests in order to ensure the batch logic
kicks in.

Also replace some unneeded mocks with real classes.

Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add queryGroupTask

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove unnecessary imports

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add QueryGroupTask tests

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* rename WLM transport request handler

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add CHANGELOG entry

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix ut

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix UT to remove the verify for final method

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* apply spotless

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Zelin Hao <zelinhao@amazon.com>
Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Swetha Guptha <gupthasg@amazon.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Signed-off-by: Craig Perkins <craig5008@gmail.com>
Signed-off-by: mgodwan <mgodwan@amazon.com>
Signed-off-by: Mohit Godwani <mgodwan@amazon.com>
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sandeep Kumawat <skumwt@amazon.com>
Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com>
Signed-off-by: Daniil Roman <danroman17397@gmail.com>
Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>
Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com>
Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com>
Signed-off-by: Sumit Bansal <sumitsb@amazon.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
Signed-off-by: imyp92 <pyw5420@gmail.com>
Signed-off-by: gaobinlong <gbinlong@amazon.com>
Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org>
Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: Finn Carroll <carrofin@amazon.com>
Signed-off-by: Neetika Singhal <neetiks@amazon.com>
Signed-off-by: Jay Deng <jayd0104@gmail.com>
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com>
Co-authored-by: gaobinlong <gbinlong@amazon.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Chenyang Ji <cyji@amazon.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Sandesh Kumar <sandeshkr419@gmail.com>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: Zelin Hao <zelinhao@amazon.com>
Co-authored-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Co-authored-by: kkewwei <kkewwei@163.com>
Co-authored-by: SwethaGuptha <156877431+SwethaGuptha@users.noreply.github.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Ahmed Sobeh <ahmed.sobeh@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
Co-authored-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Co-authored-by: Craig Perkins <craig5008@gmail.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Co-authored-by: Mohit Godwani <81609427+mgodwan@users.noreply.github.com>
Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sandeep Kumawat <2025sandeepkumawat@gmail.com>
Co-authored-by: Sandeep Kumawat <skumwt@amazon.com>
Co-authored-by: Daniil Roman <danroman17397@gmail.com>
Co-authored-by: Rishabh Singh <rishabhksingh@gmail.com>
Co-authored-by: kkewwei <kewei.11@bytedance.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: Ashish Singh <ssashish@amazon.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Pranshu Shukla <55992439+Pranshu-S@users.noreply.github.com>
Co-authored-by: rajiv-kv <157019998+rajiv-kv@users.noreply.github.com>
Co-authored-by: Sumit Bansal <sumit.asr@gmail.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: shailendra0811 <167273922+shailendra0811@users.noreply.github.com>
Co-authored-by: Park, Yeongwu <pyw5420@gmail.com>
Co-authored-by: ebraminio <ebraminio@gmail.com>
Co-authored-by: Liyun Xiu <xiliyun@amazon.com>
Co-authored-by: Finn <carrofin@amazon.com>
Co-authored-by: Neetika Singhal <neetiks@amazon.com>
Co-authored-by: Jay Deng <jayd0104@gmail.com>
Co-authored-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>
(cherry picked from commit eb306d2)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
jed326 added a commit that referenced this issue Aug 7, 2024
@opensearch-trigger-bot @github-actions
@shiv0408 @Arpit-Bandejiya @gaobinlong @RS146BIJAY @ansjcy @cwperks @sandeshkr419 @reta @zelinh @lukas-vlcek @kkewwei @SwethaGuptha @soosinha @ahmedsobeh @dependabot @deshsidd @peteralfonsi @imvtsl @robinf95 @sachinpkale @bharath-techie @shourya035 @mgodwan @sarthakaggarwal97 @sgup432 @skumawat2025 @DaniilRoman @rishabh6788 @dblock @ashking94 @bowenlan-amzn @Pranshu-S @rajiv-kv @sumitasr @dbwiddis @shailendra0811 @imyp92 @ebraminio @chishui @finnegancarroll @neetikasinghal @jed326 @gbbafna @andrross
Add queryGroupId to search workload tasks at co-ordinator and data no…
d4bdc7a 
…de level (#14708) (#15029)

* add logic to add headers to Task



* add logic to add queryGroupId to task headers



* remove redundant code



* add changelog entry



* address comments



* fix precommit



* Add UTs for RemoteIndexMetadataManager (#14660)





* Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes (#10959)

* Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes


* Add more test



* modify change log



* Fix test failure



* Change the indexAnalyzer used by prefix field



* Skip old version for yaml test



* Optimize some code



* Fix test failure



* Modify yaml test description



* Remove the name parameter for setAnalyzer()



---------




* Offline calculation of total shard per node and caching it for weight calculation inside LocalShardBalancer (#14675)




* [bug fix] validate lower bound for top n size (#14587)




* Create SystemIndexRegistry with helper method matchesSystemIndex (#14415)

* Create new extension point in SystemIndexPlugin for a single plugin to get registered system indices



* Add to CHANGELOG



* WIP on system indices from IndexNameExpressionResolver



* Add test in IndexNameExpressionResolverTests



* Remove changes in SystemIndexPlugin



* Add method in IndexNameExpressionResolver to get matching system indices



* Show how resolver can be chained to get system indices



* Fix forbiddenApis check



* Update CHANGELOG



* Make SystemIndices internal



* Remove unneeded changes



* Fix CI failures



* Fix precommit errors



* Use Regex instead of WildcardMatcher



* Address code review feedback



* Allow caller to pass index expressions



* Create SystemIndexRegistry



* Update CHANGELOG



* Remove singleton limitation



* Add javadoc



* Add @experimentalapi annotation



---------




* Refactor Grok validate pattern to iterative approach (#14206)

* grok validate patterns recusrion to iterative



* Add max depth in resolving a pattern to avoid OOM



* change path from deque to arraylist



* rename queue to stack



* Change max depth to 500



* typo originPatternName fix



* spotless



---------




* Bump opentelemetry from 1.39.0 to 1.40.0 (#14674)




* Bump jackson from 2.17.1 to 2.17.2 (#14687)




* Add release notes for release 1.3.18 (#14699)




* Bump reactor from 3.5.19 to 3.5.20 (#14697)




* Add unit tests for read flow of RemoteClusterStateService and bug fix for transient settings (#14476)




* Update version check for the bug fix of match_phrase_prefix_query not working on text field with multiple values and index_prefixes (#14703)




* Remove unnecessary cast to int from test (#14696)




* print reason why parent task was cancelled (#14604)




* Use set of shard routing for shard in unassigned shard batch check. (#14533)




* Add versioning for UploadedIndexMetadata (#14677)

* Add versioning for UploadedIndexMetadata
* Handle componentPrefix for backward compatibility




* Fix: update help output for _cat (#14722)

* fixed help output for _cat



* updated changelog



* updated changelog



---------




* Fix hdfs-fixture kerb-admin & hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (#14729)




* Update to Gradle 8.9 (#14574)




* Fix hdfs-fixture hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (#14732)




* Add `strict_allow_templates` dynamic mapping option (#14555)

* The dynamic mapping parameter supports strict_allow_templates



* Modify change log



* Modify skip version in yml test file



* Refactor some code



* Keep the old methods



* change public to private



* Optimize some code



* Do not override toString method for Dynamic



* Optimize some code and modify the changelog



---------




* Bump net.minidev:json-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14748)

* Bump net.minidev:json-smart in /plugins/repository-azure

Bumps [net.minidev:json-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:json-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...



* Updating SHAs



* Update changelog



---------






* remove query insights plugin from core (#14743)




* Add `strict_allow_templates` dynamic mapping option (#14555) (#14737) (#14742)

* The dynamic mapping parameter supports strict_allow_templates

* Modify change log

* Modify skip version in yml test file

* Refactor some code

* Keep the old methods

* change public to private

* Optimize some code

* Do not override toString method for Dynamic

* Optimize some code and modify the changelog

---------

(cherry picked from commit 6b8b3ef)








* Fix create or update alias API doesn't throw exception for unsupported parameters (#14719)

* Fix create or update alias API doesn't throw exception for unsupported parameters



* Update version check in yml test



* modify change log



---------




* Remove query categorization from core (#14759)

* Remove query categorization from core



* Add changelog



* Trigger Build



---------




* Add changes to propagate queryGroupId across child requests and nodes (#14614)

* add query group header propagator



* apply spotless check



* add new propagator in ThreadContext



* spotlessApply



* address comments



* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...



* Updating SHAs



* Update changelog



---------






* [Bugfix] Fix ICacheKeySerializerTests flakiness (#14564)

* Fix testInvalidInput flakiness



* Addressed andrross's comment



* rerun security check



---------





* Correct typo in method name (#14621)




* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (#14200)

* Refactor FilterPath parse function (#12067)


* Implement unit tests for FilterPathTests (#12067)


* Write warn log if Filter is empty; Add comments (#12067)


* Add changelog



* Remove unnecessary log statement



* Remove unused logger



* Spotless apply



* Remove incorrect changelog



---------





* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(#14612)




* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings




* add changelog



* add PR link changelog



* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (#14613)




* Fix assertion failure while deleting remote backed index (#14601)




* Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (#14635)

* Allow system index warning



* Add to CHANGELOG



* Address code review comments



---------




* Star tree codec changes (#14514)

---------



* Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (#14672)

* Bump com.github.spullara.mustache.java:compiler

Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14.
- [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14)

---
updated-dependencies:
- dependency-name: com.github.spullara.mustache.java:compiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...



* Updating SHAs



* Update changelog



---------






* Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (#14673)

* Bump net.minidev:accessors-smart in /plugins/repository-azure

Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:accessors-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...



* Updating SHAs



* Update changelog



---------






* move query group thread context propagator out of ThreadContext



---------
























* Add consumers to remote store based index settings (#14764)




* Add matchesPluginSystemIndexPattern to SystemIndexRegistry (#14750)

* Add matchesPluginSystemIndexPattern to SystemIndexRegistry



* Add to CHANGELOG



* Use single data structure to keep track of system indices



* Address code review comments



* Add test for getAllDescriptors



* Update server/src/main/java/org/opensearch/indices/SystemIndexRegistry.java




---------






* SPI for loading ABC templates (#14659)

* SPI for loading ABC templates




* Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (#12891)

* Fix bulk upsert ignores the default_pipeline and final_pipeline when auto-created index matches with the index template



* Modify changelog & comment



* Use new approach



* Fix test failure



---------




* Fix flaky test due to node being used across all tests (#14787)




* Star Tree Implementation [OnHeap] (#14512)

---------



* Add Gao Binlong as maintainer (#14796)




* Clear ehcache disk cache files during initialization (#14738)

* Clear ehcache disk cache files during initialization



* Adding UT to fix line coverage



* Addressing comment



* Adding more Uts for better line coverage



* Throwing exception in case we fail to clear cache files during startup



* Adding more UTs



* Adding a UT for more coverage



* Fixing gradle build



* Update ehcache disk cache close() logic



---------




* Refactor remote-routing-table service inline with remote state interfaces (#14668)

---------




* Set version to 2.15 for determining metadata during migration to remote store





* Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (#14793)




* Fix create or update alias API doesn't throw exception for unsupported parameters (#14769)




* Change RCSS info logs to debug (#14814)




* [Bugfix] Fix NPE in ReplicaShardAllocator (#13993) (#14385)

* [Bugfix] Fix NPE in ReplicaShardAllocator (#13993)



* Add fix info to CHANGELOG.md



---------





* Run performance benchmark on pull requests (#14760)

* add performance benchmark workflow for pull requests



* Update PERFORMANCE_BENCHMARKS.md




* Update PERFORMANCE_BENCHMARKS.md




* Update .github/workflows/benchmark-pull-request.yml




* Update .github/workflows/benchmark-pull-request.yml




* Update .github/workflows/benchmark-pull-request.yml




* Update .github/workflows/benchmark-pull-request.yml




---------






* fix constant_keyword field type (#14807)



test





* [Remote Store Migration] Reconcile remote store based index settings during STRICT mode switch (#14792)




* Add prefix mode verification setting for repository verification (#14790)

* Add prefix mode verification setting for repository verification



* Add UTs and randomise prefix mode repository verification



* Incorporate PR review feedback



---------




* add length check on comment body for benchmark workflow (#14834)




* Add restore-from-snapshot test procedure for snapshot run benchmark config (#14842)




* Fix env variable name typo (#14843)




* Use circuit breaker in InternalHistogram when adding empty buckets (#14754)

* introduce circuit breaker in InternalHistogram



* use circuit breaker from reduce context



* add test



* revert use_real_memory change in OpenSearchNode



* add change log



---------




* [Remote State] Create interface RemoteEntitiesManager (#14671)

* Create interface RemoteEntitiesManager




* Optimise TransportNodesAction to not send DiscoveryNodes for NodeStat… (#14749)

* Optimize TransportNodesAction to not send DiscoveryNodes for NodeStats, NodesInfo and ClusterStats call




* Enabling term version check on local state for all ClusterManager Read Transport Actions (#14273)

* enabling term version check on local state for all admin read actions




* Reduce logging in DEBUG for MasterService:run (#14795)

* Reduce logging in DEBUG for MasteService:run by introducing short and long summary in Taskbatcher




* Add SplitResponseProcessor to Search Pipelines (#14800)

* Add SplitResponseProcessor for search pipelines



* Register the split processor factory



* Address code review comments



* Avoid list copy by casting array



---------




* Add integration tests for RemoteRoutingTable Service. (#14631)




* Add SortResponseProcessor to Search Pipelines (#14785)

* Add SortResponseProcessor for search pipelines



* Add stupid and unnecessary javadocs to satisfy overly strict CI



* Split casting and sorting methods for readability



* Register the sort processor factory



* Address code review comments



* Cast individual list elements to avoid creating two lists



* Add yamlRestTests



* Clarify why there's unusual sorting



* Use instanceof instead of isAssignableFrom



---------




* Fix allowUnmappedFields, mapUnmappedFieldAsString settings to be applied when parsing query string query (#13957)

* Modify to invoke QueryShardContext.fieldMapper() method to apply allowUnmappedFields and mapUnmappedFieldAsString settings



* Add test cases to verify returning 400 responses if unmapped fields are included for some types of query



* Add changelog



---------






* Bump com.microsoft.azure:msal4j from 1.16.0 to 1.16.1 in /plugins/repository-azure (#14857)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.16.0 to 1.16.1.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.16.0...v1.16.1)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-patch
...



* Updating SHAs



* Update changelog



---------






* Bump com.gradle.develocity from 3.17.5 to 3.17.6 (#14856)

* Bump com.gradle.develocity from 3.17.5 to 3.17.6

Bumps com.gradle.develocity from 3.17.5 to 3.17.6.

---
updated-dependencies:
- dependency-name: com.gradle.develocity
  dependency-type: direct:production
  update-type: version-update:semver-patch
...



* Update changelog



---------






* Bump org.jline:jline in /test/fixtures/hdfs-fixture (#14859)

Bumps [org.jline:jline](https://github.com/jline/jline3) from 3.26.2 to 3.26.3.
- [Release notes](https://github.com/jline/jline3/releases)
- [Changelog](https://github.com/jline/jline3/blob/master/changelog.md)
- [Commits](jline/jline3@jline-parent-3.26.2...jline-parent-3.26.3)

---
updated-dependencies:
- dependency-name: org.jline:jline
  dependency-type: direct:production
  update-type: version-update:semver-patch
...





* Use Lucene provided Persian stem (#14847)

Lucene provided Persian stem apparently isn't hooked yet and this change is
doing that based on what is done for Arabic stem support.






* Bump actions/checkout from 2 to 4 (#14858)

* Bump actions/checkout from 2 to 4

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v2...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...



* Update changelog



---------






* Deprecate batch_size parameter on bulk API (#14725)

By default the full _bulk payload will be passed to ingest processors as a
batch, with any sub batching logic to be implemented by each processor if
necessary.




* Add perms for remote snapshot cache eviction on scripted query (#14411)




* add transport interceptor to populate queryGroupId in task headers



* Add rest, transport layer changes for Hot to warm tiering - dedicated setup (#13980)




* Create listener to refresh search thread resource usage (#14832)

* [bug fix] fix incorrect coordinator node search resource usages



* fix bug on serialization when passing task resource usage to coordinator



* add more unit tests



* remove query insights plugin related code



* create per request listener to refresh task resource usage



* Make new listener API public



* Add changelog



* Remove wrong files added



* Address review comments



* Build fix



* Make singleton



* Address review comments



* Make sure listener runs before plugin listeners



* Spotless



* Minor fix



---------








* Caching avg total bytes and avg free bytes inside ClusterInfo (#14851)




* Use default value when index.number_of_replicas is null (#14812)

* Use default value when index.number_of_replicas is null



* Add integration test



* Add changelog



---------




* [Remote Routing Table] Implement write and read flow for shard diff file. (#14684)

* Implement write and read flow to upload/download shard diff file.




* Optimized ClusterStatsIndices to precomute shard stats (#14426)

* Optimize Cluster Stats Indices to precomute node level stats




* Fix constraint bug which allows more primary shards than average primary shards per index (#14908)




* Optmising AwarenessAllocationDecider for hashmap.get call (#14761)




* update comment



* Fix IngestServiceTests.testBulkRequestExecutionWithFailures (#14918)

The test would previously fail if the randomness led to only a single
indexing request being included in the bulk payload. This change
guarantees multiple indexing requests in order to ensure the batch logic
kicks in.

Also replace some unneeded mocks with real classes.




* add queryGroupTask



* remove unnecessary imports



* add QueryGroupTask tests



* rename WLM transport request handler



* add CHANGELOG entry



* fix ut



* address comments



* fix UT to remove the verify for final method



* apply spotless



---------




































































































(cherry picked from commit eb306d2)

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Zelin Hao <zelinhao@amazon.com>
Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Swetha Guptha <gupthasg@amazon.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Signed-off-by: Craig Perkins <craig5008@gmail.com>
Signed-off-by: mgodwan <mgodwan@amazon.com>
Signed-off-by: Mohit Godwani <mgodwan@amazon.com>
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sandeep Kumawat <skumwt@amazon.com>
Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com>
Signed-off-by: Daniil Roman <danroman17397@gmail.com>
Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>
Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com>
Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com>
Signed-off-by: Sumit Bansal <sumitsb@amazon.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
Signed-off-by: imyp92 <pyw5420@gmail.com>
Signed-off-by: gaobinlong <gbinlong@amazon.com>
Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org>
Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: Finn Carroll <carrofin@amazon.com>
Signed-off-by: Neetika Singhal <neetiks@amazon.com>
Signed-off-by: Jay Deng <jayd0104@gmail.com>
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com>
Co-authored-by: gaobinlong <gbinlong@amazon.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Chenyang Ji <cyji@amazon.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Sandesh Kumar <sandeshkr419@gmail.com>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: Zelin Hao <zelinhao@amazon.com>
Co-authored-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Co-authored-by: kkewwei <kkewwei@163.com>
Co-authored-by: SwethaGuptha <156877431+SwethaGuptha@users.noreply.github.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Ahmed Sobeh <ahmed.sobeh@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
Co-authored-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Co-authored-by: Craig Perkins <craig5008@gmail.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Co-authored-by: Mohit Godwani <81609427+mgodwan@users.noreply.github.com>
Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sandeep Kumawat <2025sandeepkumawat@gmail.com>
Co-authored-by: Sandeep Kumawat <skumwt@amazon.com>
Co-authored-by: Daniil Roman <danroman17397@gmail.com>
Co-authored-by: Rishabh Singh <rishabhksingh@gmail.com>
Co-authored-by: kkewwei <kewei.11@bytedance.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: Ashish Singh <ssashish@amazon.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Pranshu Shukla <55992439+Pranshu-S@users.noreply.github.com>
Co-authored-by: rajiv-kv <157019998+rajiv-kv@users.noreply.github.com>
Co-authored-by: Sumit Bansal <sumit.asr@gmail.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: shailendra0811 <167273922+shailendra0811@users.noreply.github.com>
Co-authored-by: Park, Yeongwu <pyw5420@gmail.com>
Co-authored-by: ebraminio <ebraminio@gmail.com>
Co-authored-by: Liyun Xiu <xiliyun@amazon.com>
Co-authored-by: Finn <carrofin@amazon.com>
Co-authored-by: Neetika Singhal <neetiks@amazon.com>
Co-authored-by: Jay Deng <jayd0104@gmail.com>
Co-authored-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>
harshavamsi pushed a commit to harshavamsi/OpenSearch that referenced this issue Aug 20, 2024
@kaushalmahi12 @shiv0408
@Arpit-Bandejiya @gaobinlong @RS146BIJAY @ansjcy @cwperks @sandeshkr419 @reta @zelinh @lukas-vlcek @kkewwei @SwethaGuptha @soosinha @ahmedsobeh @dependabot @opensearch-trigger-bot @github-actions @deshsidd @peteralfonsi @imvtsl @robinf95 @sachinpkale @bharath-techie @shourya035 @mgodwan @sarthakaggarwal97 @sgup432 @skumawat2025 @DaniilRoman @rishabh6788 @dblock @ashking94 @bowenlan-amzn @Pranshu-S @rajiv-kv @sumitasr @dbwiddis @shailendra0811 @imyp92 @ebraminio @chishui @finnegancarroll @neetikasinghal @jed326 @gbbafna @andrross @harshavamsi
Add queryGroupId to search workload tasks at co-ordinator and data no…
3d631f2 
…de level (opensearch-project#14708)

* add logic to add headers to Task

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add logic to add queryGroupId to task headers

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove redundant code

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add changelog entry

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix precommit

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add UTs for RemoteIndexMetadataManager (opensearch-project#14660)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes (opensearch-project#10959)

* Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes
Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Add more test

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* modify change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Fix test failure

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Change the indexAnalyzer used by prefix field

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Skip old version for yaml test

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Fix test failure

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify yaml test description

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Remove the name parameter for setAnalyzer()

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Offline calculation of total shard per node and caching it for weight calculation inside LocalShardBalancer (opensearch-project#14675)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [bug fix] validate lower bound for top n size (opensearch-project#14587)

Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Create SystemIndexRegistry with helper method matchesSystemIndex (opensearch-project#14415)

* Create new extension point in SystemIndexPlugin for a single plugin to get registered system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* WIP on system indices from IndexNameExpressionResolver

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add test in IndexNameExpressionResolverTests

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Remove changes in SystemIndexPlugin

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add method in IndexNameExpressionResolver to get matching system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Show how resolver can be chained to get system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Fix forbiddenApis check

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Update CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Make SystemIndices internal

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Remove unneeded changes

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Fix CI failures

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Fix precommit errors

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Use Regex instead of WildcardMatcher

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review feedback

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Allow caller to pass index expressions

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Create SystemIndexRegistry

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Update CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Remove singleton limitation

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add javadoc

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add @experimentalapi annotation

Signed-off-by: Craig Perkins <cwperx@amazon.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactor Grok validate pattern to iterative approach (opensearch-project#14206)

* grok validate patterns recusrion to iterative

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* Add max depth in resolving a pattern to avoid OOM

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* change path from deque to arraylist

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* rename queue to stack

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* Change max depth to 500

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* typo originPatternName fix

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* spotless

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

---------

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump opentelemetry from 1.39.0 to 1.40.0 (opensearch-project#14674)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump jackson from 2.17.1 to 2.17.2 (opensearch-project#14687)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add release notes for release 1.3.18 (opensearch-project#14699)

Signed-off-by: Zelin Hao <zelinhao@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump reactor from 3.5.19 to 3.5.20 (opensearch-project#14697)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add unit tests for read flow of RemoteClusterStateService and bug fix for transient settings (opensearch-project#14476)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Update version check for the bug fix of match_phrase_prefix_query not working on text field with multiple values and index_prefixes (opensearch-project#14703)

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Remove unnecessary cast to int from test (opensearch-project#14696)

Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* print reason why parent task was cancelled (opensearch-project#14604)

Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use set of shard routing for shard in unassigned shard batch check. (opensearch-project#14533)

Signed-off-by: Swetha Guptha <gupthasg@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add versioning for UploadedIndexMetadata (opensearch-project#14677)

* Add versioning for UploadedIndexMetadata
* Handle componentPrefix for backward compatibility

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix: update help output for _cat (opensearch-project#14722)

* fixed help output for _cat

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>

* updated changelog

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>

* updated changelog

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>

---------

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix hdfs-fixture kerb-admin & hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (opensearch-project#14729)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Update to Gradle 8.9 (opensearch-project#14574)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix hdfs-fixture hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (opensearch-project#14732)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add `strict_allow_templates` dynamic mapping option (opensearch-project#14555)

* The dynamic mapping parameter supports strict_allow_templates

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify skip version in yml test file

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Refactor some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Keep the old methods

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* change public to private

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Do not override toString method for Dynamic

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code and modify the changelog

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump net.minidev:json-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (opensearch-project#14748)

* Bump net.minidev:json-smart in /plugins/repository-azure

Bumps [net.minidev:json-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:json-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove query insights plugin from core (opensearch-project#14743)

Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add `strict_allow_templates` dynamic mapping option (opensearch-project#14555) (opensearch-project#14737) (opensearch-project#14742)

* The dynamic mapping parameter supports strict_allow_templates

* Modify change log

* Modify skip version in yml test file

* Refactor some code

* Keep the old methods

* change public to private

* Optimize some code

* Do not override toString method for Dynamic

* Optimize some code and modify the changelog

---------

(cherry picked from commit 6b8b3ef)

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix create or update alias API doesn't throw exception for unsupported parameters (opensearch-project#14719)

* Fix create or update alias API doesn't throw exception for unsupported parameters

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Update version check in yml test

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* modify change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Remove query categorization from core (opensearch-project#14759)

* Remove query categorization from core

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Trigger Build

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add changes to propagate queryGroupId across child requests and nodes (opensearch-project#14614)

* add query group header propagator

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* apply spotless check

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add new propagator in ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* spotlessApply

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add PR link changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (opensearch-project#14635)

* Allow system index warning

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review comments

Signed-off-by: Craig Perkins <cwperx@amazon.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Star tree codec changes (opensearch-project#14514)

---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (opensearch-project#14672)

* Bump com.github.spullara.mustache.java:compiler

Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14.
- [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14)

---
updated-dependencies:
- dependency-name: com.github.spullara.mustache.java:compiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (opensearch-project#14673)

* Bump net.minidev:accessors-smart in /plugins/repository-azure

Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:accessors-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* move query group thread context propagator out of ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add consumers to remote store based index settings (opensearch-project#14764)

Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add matchesPluginSystemIndexPattern to SystemIndexRegistry (opensearch-project#14750)

* Add matchesPluginSystemIndexPattern to SystemIndexRegistry

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Use single data structure to keep track of system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review comments

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add test for getAllDescriptors

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Update server/src/main/java/org/opensearch/indices/SystemIndexRegistry.java

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Craig Perkins <craig5008@gmail.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <craig5008@gmail.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* SPI for loading ABC templates (opensearch-project#14659)

* SPI for loading ABC templates

Signed-off-by: mgodwan <mgodwan@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (opensearch-project#12891)

* Fix bulk upsert ignores the default_pipeline and final_pipeline when auto-created index matches with the index template

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify changelog & comment

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Use new approach

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Fix test failure

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix flaky test due to node being used across all tests (opensearch-project#14787)

Signed-off-by: Mohit Godwani <mgodwan@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Star Tree Implementation [OnHeap] (opensearch-project#14512)

---------
Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add Gao Binlong as maintainer (opensearch-project#14796)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Clear ehcache disk cache files during initialization (opensearch-project#14738)

* Clear ehcache disk cache files during initialization

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding UT to fix line coverage

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comment

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding more Uts for better line coverage

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Throwing exception in case we fail to clear cache files during startup

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding more UTs

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding a UT for more coverage

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing gradle build

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Update ehcache disk cache close() logic

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactor remote-routing-table service inline with remote state interfaces (opensearch-project#14668)

---------
Signed-off-by: Arpit Bandejiya <abandeji@amazon.com>
Signed-off-by: Arpit-Bandejiya <abandeji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Set version to 2.15 for determining metadata during migration to remote store

Signed-off-by: Sandeep Kumawat <skumwt@amazon.com>
Co-authored-by: Sandeep Kumawat <skumwt@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (opensearch-project#14793)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix create or update alias API doesn't throw exception for unsupported parameters (opensearch-project#14769)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Change RCSS info logs to debug (opensearch-project#14814)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Bugfix] Fix NPE in ReplicaShardAllocator (opensearch-project#13993) (opensearch-project#14385)

* [Bugfix] Fix NPE in ReplicaShardAllocator (opensearch-project#13993)

Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com>

* Add fix info to CHANGELOG.md

Signed-off-by: Daniil Roman <danroman17397@gmail.com>

---------

Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com>
Signed-off-by: Daniil Roman <danroman17397@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Run performance benchmark on pull requests (opensearch-project#14760)

* add performance benchmark workflow for pull requests

Signed-off-by: Rishabh Singh <sngri@amazon.com>

* Update PERFORMANCE_BENCHMARKS.md

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update PERFORMANCE_BENCHMARKS.md

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

---------

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix constant_keyword field type (opensearch-project#14807)

Signed-off-by: kkewwei <kkewwei@163.com>

test

Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Remote Store Migration] Reconcile remote store based index settings during STRICT mode switch (opensearch-project#14792)

Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add prefix mode verification setting for repository verification (opensearch-project#14790)

* Add prefix mode verification setting for repository verification

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Add UTs and randomise prefix mode repository verification

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Incorporate PR review feedback

Signed-off-by: Ashish Singh <ssashish@amazon.com>

---------

Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add length check on comment body for benchmark workflow (opensearch-project#14834)

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add restore-from-snapshot test procedure for snapshot run benchmark config (opensearch-project#14842)

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix env variable name typo (opensearch-project#14843)

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use circuit breaker in InternalHistogram when adding empty buckets (opensearch-project#14754)

* introduce circuit breaker in InternalHistogram

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* use circuit breaker from reduce context

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* add test

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* revert use_real_memory change in OpenSearchNode

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* add change log

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Remote State] Create interface RemoteEntitiesManager (opensearch-project#14671)

* Create interface RemoteEntitiesManager

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Optimise TransportNodesAction to not send DiscoveryNodes for NodeStat… (opensearch-project#14749)

* Optimize TransportNodesAction to not send DiscoveryNodes for NodeStats, NodesInfo and ClusterStats call

Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Enabling term version check on local state for all ClusterManager Read Transport Actions (opensearch-project#14273)

* enabling term version check on local state for all admin read actions

Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Reduce logging in DEBUG for MasterService:run (opensearch-project#14795)

* Reduce logging in DEBUG for MasteService:run by introducing short and long summary in Taskbatcher

Signed-off-by: Sumit Bansal <sumitsb@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add SplitResponseProcessor to Search Pipelines (opensearch-project#14800)

* Add SplitResponseProcessor for search pipelines

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Register the split processor factory

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Address code review comments

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Avoid list copy by casting array

Signed-off-by: Daniel Widdis <widdis@gmail.com>

---------

Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add integration tests for RemoteRoutingTable Service. (opensearch-project#14631)

Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add SortResponseProcessor to Search Pipelines (opensearch-project#14785)

* Add SortResponseProcessor for search pipelines

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add stupid and unnecessary javadocs to satisfy overly strict CI

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Split casting and sorting methods for readability

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Register the sort processor factory

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Address code review comments

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Cast individual list elements to avoid creating two lists

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add yamlRestTests

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Clarify why there's unusual sorting

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Use instanceof instead of isAssignableFrom

Signed-off-by: Daniel Widdis <widdis@gmail.com>

---------

Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix allowUnmappedFields, mapUnmappedFieldAsString settings to be applied when parsing query string query (opensearch-project#13957)

* Modify to invoke QueryShardContext.fieldMapper() method to apply allowUnmappedFields and mapUnmappedFieldAsString settings

Signed-off-by: imyp92 <pyw5420@gmail.com>

* Add test cases to verify returning 400 responses if unmapped fields are included for some types of query

Signed-off-by: imyp92 <pyw5420@gmail.com>

* Add changelog

Signed-off-by: imyp92 <pyw5420@gmail.com>

---------

Signed-off-by: imyp92 <pyw5420@gmail.com>
Signed-off-by: gaobinlong <gbinlong@amazon.com>
Co-authored-by: gaobinlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.16.0 to 1.16.1 in /plugins/repository-azure (opensearch-project#14857)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.16.0 to 1.16.1.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.16.0...v1.16.1)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.gradle.develocity from 3.17.5 to 3.17.6 (opensearch-project#14856)

* Bump com.gradle.develocity from 3.17.5 to 3.17.6

Bumps com.gradle.develocity from 3.17.5 to 3.17.6.

---
updated-dependencies:
- dependency-name: com.gradle.develocity
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump org.jline:jline in /test/fixtures/hdfs-fixture (opensearch-project#14859)

Bumps [org.jline:jline](https://github.com/jline/jline3) from 3.26.2 to 3.26.3.
- [Release notes](https://github.com/jline/jline3/releases)
- [Changelog](https://github.com/jline/jline3/blob/master/changelog.md)
- [Commits](jline/jline3@jline-parent-3.26.2...jline-parent-3.26.3)

---
updated-dependencies:
- dependency-name: org.jline:jline
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use Lucene provided Persian stem (opensearch-project#14847)

Lucene provided Persian stem apparently isn't hooked yet and this change is
doing that based on what is done for Arabic stem support.

Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org>
Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump actions/checkout from 2 to 4 (opensearch-project#14858)

* Bump actions/checkout from 2 to 4

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v2...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Deprecate batch_size parameter on bulk API (opensearch-project#14725)

By default the full _bulk payload will be passed to ingest processors as a
batch, with any sub batching logic to be implemented by each processor if
necessary.

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add perms for remote snapshot cache eviction on scripted query (opensearch-project#14411)

Signed-off-by: Finn Carroll <carrofin@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add transport interceptor to populate queryGroupId in task headers

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add rest, transport layer changes for Hot to warm tiering - dedicated setup (opensearch-project#13980)

Signed-off-by: Neetika Singhal <neetiks@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Create listener to refresh search thread resource usage (opensearch-project#14832)

* [bug fix] fix incorrect coordinator node search resource usages

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* fix bug on serialization when passing task resource usage to coordinator

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* add more unit tests

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* remove query insights plugin related code

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* create per request listener to refresh task resource usage

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* Make new listener API public

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove wrong files added

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Address review comments

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Build fix

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Make singleton

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Address review comments

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Make sure listener runs before plugin listeners

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Minor fix

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: Jay Deng <jayd0104@gmail.com>
Co-authored-by: Chenyang Ji <cyji@amazon.com>
Co-authored-by: Jay Deng <jayd0104@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Caching avg total bytes and avg free bytes inside ClusterInfo (opensearch-project#14851)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use default value when index.number_of_replicas is null (opensearch-project#14812)

* Use default value when index.number_of_replicas is null

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

* Add integration test

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

* Add changelog

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

---------

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Remote Routing Table] Implement write and read flow for shard diff file. (opensearch-project#14684)

* Implement write and read flow to upload/download shard diff file.

Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Optimized ClusterStatsIndices to precomute shard stats (opensearch-project#14426)

* Optimize Cluster Stats Indices to precomute node level stats

Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix constraint bug which allows more primary shards than average primary shards per index (opensearch-project#14908)

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Optmising AwarenessAllocationDecider for hashmap.get call (opensearch-project#14761)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* update comment

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix IngestServiceTests.testBulkRequestExecutionWithFailures (opensearch-project#14918)

The test would previously fail if the randomness led to only a single
indexing request being included in the bulk payload. This change
guarantees multiple indexing requests in order to ensure the batch logic
kicks in.

Also replace some unneeded mocks with real classes.

Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add queryGroupTask

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove unnecessary imports

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add QueryGroupTask tests

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* rename WLM transport request handler

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add CHANGELOG entry

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix ut

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix UT to remove the verify for final method

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* apply spotless

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Zelin Hao <zelinhao@amazon.com>
Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Swetha Guptha <gupthasg@amazon.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Signed-off-by: Craig Perkins <craig5008@gmail.com>
Signed-off-by: mgodwan <mgodwan@amazon.com>
Signed-off-by: Mohit Godwani <mgodwan@amazon.com>
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sandeep Kumawat <skumwt@amazon.com>
Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com>
Signed-off-by: Daniil Roman <danroman17397@gmail.com>
Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>
Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com>
Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com>
Signed-off-by: Sumit Bansal <sumitsb@amazon.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
Signed-off-by: imyp92 <pyw5420@gmail.com>
Signed-off-by: gaobinlong <gbinlong@amazon.com>
Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org>
Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: Finn Carroll <carrofin@amazon.com>
Signed-off-by: Neetika Singhal <neetiks@amazon.com>
Signed-off-by: Jay Deng <jayd0104@gmail.com>
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com>
Co-authored-by: gaobinlong <gbinlong@amazon.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Chenyang Ji <cyji@amazon.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Sandesh Kumar <sandeshkr419@gmail.com>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: Zelin Hao <zelinhao@amazon.com>
Co-authored-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Co-authored-by: kkewwei <kkewwei@163.com>
Co-authored-by: SwethaGuptha <156877431+SwethaGuptha@users.noreply.github.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Ahmed Sobeh <ahmed.sobeh@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
Co-authored-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Co-authored-by: Craig Perkins <craig5008@gmail.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Co-authored-by: Mohit Godwani <81609427+mgodwan@users.noreply.github.com>
Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sandeep Kumawat <2025sandeepkumawat@gmail.com>
Co-authored-by: Sandeep Kumawat <skumwt@amazon.com>
Co-authored-by: Daniil Roman <danroman17397@gmail.com>
Co-authored-by: Rishabh Singh <rishabhksingh@gmail.com>
Co-authored-by: kkewwei <kewei.11@bytedance.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: Ashish Singh <ssashish@amazon.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Pranshu Shukla <55992439+Pranshu-S@users.noreply.github.com>
Co-authored-by: rajiv-kv <157019998+rajiv-kv@users.noreply.github.com>
Co-authored-by: Sumit Bansal <sumit.asr@gmail.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: shailendra0811 <167273922+shailendra0811@users.noreply.github.com>
Co-authored-by: Park, Yeongwu <pyw5420@gmail.com>
Co-authored-by: ebraminio <ebraminio@gmail.com>
Co-authored-by: Liyun Xiu <xiliyun@amazon.com>
Co-authored-by: Finn <carrofin@amazon.com>
Co-authored-by: Neetika Singhal <neetiks@amazon.com>
Co-authored-by: Jay Deng <jayd0104@gmail.com>
Co-authored-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>
wdongyu pushed a commit to wdongyu/OpenSearch that referenced this issue Aug 22, 2024
@deshsidd @robinf95
Refactoring FilterPath.parse by using an iterative approach instead o…
9f33371 
…f recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
wdongyu pushed a commit to wdongyu/OpenSearch that referenced this issue Aug 22, 2024
@kaushalmahi12 @dependabot
@peteralfonsi @imvtsl @deshsidd @robinf95 @RS146BIJAY @soosinha @sachinpkale @cwperks @bharath-techie
Add changes to propagate queryGroupId across child requests and nodes (…
130d4b8 
…opensearch-project#14614)

* add query group header propagator

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* apply spotless check

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add new propagator in ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* spotlessApply

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add PR link changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (opensearch-project#14635)

* Allow system index warning

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review comments

Signed-off-by: Craig Perkins <cwperx@amazon.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Star tree codec changes (opensearch-project#14514)

---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (opensearch-project#14672)

* Bump com.github.spullara.mustache.java:compiler

Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14.
- [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14)

---
updated-dependencies:
- dependency-name: com.github.spullara.mustache.java:compiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (opensearch-project#14673)

* Bump net.minidev:accessors-smart in /plugins/repository-azure

Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:accessors-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* move query group thread context propagator out of ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
wdongyu pushed a commit to wdongyu/OpenSearch that referenced this issue Aug 22, 2024
@kaushalmahi12 @shiv0408
@Arpit-Bandejiya @gaobinlong @RS146BIJAY @ansjcy @cwperks @sandeshkr419 @reta @zelinh @lukas-vlcek @kkewwei @SwethaGuptha @soosinha @ahmedsobeh @dependabot @opensearch-trigger-bot @github-actions @deshsidd @peteralfonsi @imvtsl @robinf95 @sachinpkale @bharath-techie @shourya035 @mgodwan @sarthakaggarwal97 @sgup432 @skumawat2025 @DaniilRoman @rishabh6788 @dblock @ashking94 @bowenlan-amzn @Pranshu-S @rajiv-kv @sumitasr @dbwiddis @shailendra0811 @imyp92 @ebraminio @chishui @finnegancarroll @neetikasinghal @jed326 @gbbafna @andrross
Add queryGroupId to search workload tasks at co-ordinator and data no…
c0e1d49 
…de level (opensearch-project#14708)

* add logic to add headers to Task

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add logic to add queryGroupId to task headers

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove redundant code

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add changelog entry

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix precommit

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add UTs for RemoteIndexMetadataManager (opensearch-project#14660)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes (opensearch-project#10959)

* Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes
Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Add more test

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* modify change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Fix test failure

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Change the indexAnalyzer used by prefix field

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Skip old version for yaml test

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Fix test failure

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify yaml test description

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Remove the name parameter for setAnalyzer()

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Offline calculation of total shard per node and caching it for weight calculation inside LocalShardBalancer (opensearch-project#14675)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [bug fix] validate lower bound for top n size (opensearch-project#14587)

Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Create SystemIndexRegistry with helper method matchesSystemIndex (opensearch-project#14415)

* Create new extension point in SystemIndexPlugin for a single plugin to get registered system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* WIP on system indices from IndexNameExpressionResolver

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add test in IndexNameExpressionResolverTests

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Remove changes in SystemIndexPlugin

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add method in IndexNameExpressionResolver to get matching system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Show how resolver can be chained to get system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Fix forbiddenApis check

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Update CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Make SystemIndices internal

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Remove unneeded changes

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Fix CI failures

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Fix precommit errors

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Use Regex instead of WildcardMatcher

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review feedback

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Allow caller to pass index expressions

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Create SystemIndexRegistry

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Update CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Remove singleton limitation

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add javadoc

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add @experimentalapi annotation

Signed-off-by: Craig Perkins <cwperx@amazon.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactor Grok validate pattern to iterative approach (opensearch-project#14206)

* grok validate patterns recusrion to iterative

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* Add max depth in resolving a pattern to avoid OOM

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* change path from deque to arraylist

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* rename queue to stack

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* Change max depth to 500

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* typo originPatternName fix

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

* spotless

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>

---------

Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump opentelemetry from 1.39.0 to 1.40.0 (opensearch-project#14674)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump jackson from 2.17.1 to 2.17.2 (opensearch-project#14687)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add release notes for release 1.3.18 (opensearch-project#14699)

Signed-off-by: Zelin Hao <zelinhao@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump reactor from 3.5.19 to 3.5.20 (opensearch-project#14697)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add unit tests for read flow of RemoteClusterStateService and bug fix for transient settings (opensearch-project#14476)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Update version check for the bug fix of match_phrase_prefix_query not working on text field with multiple values and index_prefixes (opensearch-project#14703)

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Remove unnecessary cast to int from test (opensearch-project#14696)

Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* print reason why parent task was cancelled (opensearch-project#14604)

Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use set of shard routing for shard in unassigned shard batch check. (opensearch-project#14533)

Signed-off-by: Swetha Guptha <gupthasg@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add versioning for UploadedIndexMetadata (opensearch-project#14677)

* Add versioning for UploadedIndexMetadata
* Handle componentPrefix for backward compatibility

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix: update help output for _cat (opensearch-project#14722)

* fixed help output for _cat

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>

* updated changelog

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>

* updated changelog

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>

---------

Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix hdfs-fixture kerb-admin & hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (opensearch-project#14729)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Update to Gradle 8.9 (opensearch-project#14574)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix hdfs-fixture hadoop-minicluster dependencies are not being updated / false positive reports on CVEs (opensearch-project#14732)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add `strict_allow_templates` dynamic mapping option (opensearch-project#14555)

* The dynamic mapping parameter supports strict_allow_templates

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify skip version in yml test file

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Refactor some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Keep the old methods

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* change public to private

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Do not override toString method for Dynamic

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Optimize some code and modify the changelog

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump net.minidev:json-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (opensearch-project#14748)

* Bump net.minidev:json-smart in /plugins/repository-azure

Bumps [net.minidev:json-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:json-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove query insights plugin from core (opensearch-project#14743)

Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add `strict_allow_templates` dynamic mapping option (opensearch-project#14555) (opensearch-project#14737) (opensearch-project#14742)

* The dynamic mapping parameter supports strict_allow_templates

* Modify change log

* Modify skip version in yml test file

* Refactor some code

* Keep the old methods

* change public to private

* Optimize some code

* Do not override toString method for Dynamic

* Optimize some code and modify the changelog

---------

(cherry picked from commit 6b8b3ef)

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix create or update alias API doesn't throw exception for unsupported parameters (opensearch-project#14719)

* Fix create or update alias API doesn't throw exception for unsupported parameters

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Update version check in yml test

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* modify change log

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Remove query categorization from core (opensearch-project#14759)

* Remove query categorization from core

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Trigger Build

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add changes to propagate queryGroupId across child requests and nodes (opensearch-project#14614)

* add query group header propagator

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* apply spotless check

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add new propagator in ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* spotlessApply

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 in /plugins/repository-azure (opensearch-project#14610)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.15.1 to 1.16.0.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.15.1...v1.16.0)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Bugfix] Fix ICacheKeySerializerTests flakiness (opensearch-project#14564)

* Fix testInvalidInput flakiness

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* Addressed andrross's comment

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

* rerun security check

Signed-off-by: Peter Alfonsi <petealft@amazon.com>

---------

Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Correct typo in method name (opensearch-project#14621)

Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactoring FilterPath.parse by using an iterative approach instead of recursion. (opensearch-project#14200)

* Refactor FilterPath parse function (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Implement unit tests for FilterPathTests (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Write warn log if Filter is empty; Add comments (opensearch-project#12067)
Signed-off-by: Robin Friedmann <robinfriedmann.rf@gmail.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unnecessary log statement

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove unused logger

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless apply

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove incorrect changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Removing String format in RemoteStoreMigrationAllocationDecider to optimise performance(opensearch-project#14612)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata; Correct the check for deciding upload of HashesOfConsistentSettings (opensearch-project#14513)

* Clear templates before Adding; Use NamedWriteableAwareStreamInput for RemoteCustomMetadata
* Correct the check for deciding upload of hashes of consistent settings

Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add PR link changelog

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Improve reroute performance by optimising List.removeAll in LocalShardsBalancer to filter remote search shard from relocation decision (opensearch-project#14613)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix assertion failure while deleting remote backed index (opensearch-project#14601)

Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (opensearch-project#14635)

* Allow system index warning

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review comments

Signed-off-by: Craig Perkins <cwperx@amazon.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Star tree codec changes (opensearch-project#14514)

---------
Signed-off-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 in /modules/lang-mustache (opensearch-project#14672)

* Bump com.github.spullara.mustache.java:compiler

Bumps [com.github.spullara.mustache.java:compiler](https://github.com/spullara/mustache.java) from 0.9.13 to 0.9.14.
- [Commits](spullara/mustache.java@mustache.java-0.9.13...mustache.java-0.9.14)

---
updated-dependencies:
- dependency-name: com.github.spullara.mustache.java:compiler
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 in /plugins/repository-azure (opensearch-project#14673)

* Bump net.minidev:accessors-smart in /plugins/repository-azure

Bumps [net.minidev:accessors-smart](https://github.com/netplex/json-smart-v2) from 2.5.0 to 2.5.1.
- [Release notes](https://github.com/netplex/json-smart-v2/releases)
- [Commits](netplex/json-smart-v2@2.5.0...2.5.1)

---
updated-dependencies:
- dependency-name: net.minidev:accessors-smart
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* move query group thread context propagator out of ThreadContext

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add consumers to remote store based index settings (opensearch-project#14764)

Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add matchesPluginSystemIndexPattern to SystemIndexRegistry (opensearch-project#14750)

* Add matchesPluginSystemIndexPattern to SystemIndexRegistry

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add to CHANGELOG

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Use single data structure to keep track of system indices

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Address code review comments

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Add test for getAllDescriptors

Signed-off-by: Craig Perkins <cwperx@amazon.com>

* Update server/src/main/java/org/opensearch/indices/SystemIndexRegistry.java

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Craig Perkins <craig5008@gmail.com>

---------

Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Craig Perkins <craig5008@gmail.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* SPI for loading ABC templates (opensearch-project#14659)

* SPI for loading ABC templates

Signed-off-by: mgodwan <mgodwan@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (opensearch-project#12891)

* Fix bulk upsert ignores the default_pipeline and final_pipeline when auto-created index matches with the index template

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Modify changelog & comment

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Use new approach

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

* Fix test failure

Signed-off-by: Gao Binlong <gbinlong@amazon.com>

---------

Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix flaky test due to node being used across all tests (opensearch-project#14787)

Signed-off-by: Mohit Godwani <mgodwan@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Star Tree Implementation [OnHeap] (opensearch-project#14512)

---------
Signed-off-by: Sarthak Aggarwal <sarthagg@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add Gao Binlong as maintainer (opensearch-project#14796)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Clear ehcache disk cache files during initialization (opensearch-project#14738)

* Clear ehcache disk cache files during initialization

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding UT to fix line coverage

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Addressing comment

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding more Uts for better line coverage

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Throwing exception in case we fail to clear cache files during startup

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding more UTs

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Adding a UT for more coverage

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Fixing gradle build

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

* Update ehcache disk cache close() logic

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>

---------

Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Refactor remote-routing-table service inline with remote state interfaces (opensearch-project#14668)

---------
Signed-off-by: Arpit Bandejiya <abandeji@amazon.com>
Signed-off-by: Arpit-Bandejiya <abandeji@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Set version to 2.15 for determining metadata during migration to remote store

Signed-off-by: Sandeep Kumawat <skumwt@amazon.com>
Co-authored-by: Sandeep Kumawat <skumwt@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix bulk upsert ignores the default_pipeline and final_pipeline when the auto-created index matches the index template (opensearch-project#14793)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix create or update alias API doesn't throw exception for unsupported parameters (opensearch-project#14769)

Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Change RCSS info logs to debug (opensearch-project#14814)

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Bugfix] Fix NPE in ReplicaShardAllocator (opensearch-project#13993) (opensearch-project#14385)

* [Bugfix] Fix NPE in ReplicaShardAllocator (opensearch-project#13993)

Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com>

* Add fix info to CHANGELOG.md

Signed-off-by: Daniil Roman <danroman17397@gmail.com>

---------

Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com>
Signed-off-by: Daniil Roman <danroman17397@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Run performance benchmark on pull requests (opensearch-project#14760)

* add performance benchmark workflow for pull requests

Signed-off-by: Rishabh Singh <sngri@amazon.com>

* Update PERFORMANCE_BENCHMARKS.md

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update PERFORMANCE_BENCHMARKS.md

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

* Update .github/workflows/benchmark-pull-request.yml

Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>

---------

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix constant_keyword field type (opensearch-project#14807)

Signed-off-by: kkewwei <kkewwei@163.com>

test

Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Remote Store Migration] Reconcile remote store based index settings during STRICT mode switch (opensearch-project#14792)

Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add prefix mode verification setting for repository verification (opensearch-project#14790)

* Add prefix mode verification setting for repository verification

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Add UTs and randomise prefix mode repository verification

Signed-off-by: Ashish Singh <ssashish@amazon.com>

* Incorporate PR review feedback

Signed-off-by: Ashish Singh <ssashish@amazon.com>

---------

Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add length check on comment body for benchmark workflow (opensearch-project#14834)

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add restore-from-snapshot test procedure for snapshot run benchmark config (opensearch-project#14842)

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix env variable name typo (opensearch-project#14843)

Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use circuit breaker in InternalHistogram when adding empty buckets (opensearch-project#14754)

* introduce circuit breaker in InternalHistogram

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* use circuit breaker from reduce context

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* add test

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* revert use_real_memory change in OpenSearchNode

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

* add change log

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>

---------

Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Remote State] Create interface RemoteEntitiesManager (opensearch-project#14671)

* Create interface RemoteEntitiesManager

Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Optimise TransportNodesAction to not send DiscoveryNodes for NodeStat… (opensearch-project#14749)

* Optimize TransportNodesAction to not send DiscoveryNodes for NodeStats, NodesInfo and ClusterStats call

Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Enabling term version check on local state for all ClusterManager Read Transport Actions (opensearch-project#14273)

* enabling term version check on local state for all admin read actions

Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Reduce logging in DEBUG for MasterService:run (opensearch-project#14795)

* Reduce logging in DEBUG for MasteService:run by introducing short and long summary in Taskbatcher

Signed-off-by: Sumit Bansal <sumitsb@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add SplitResponseProcessor to Search Pipelines (opensearch-project#14800)

* Add SplitResponseProcessor for search pipelines

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Register the split processor factory

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Address code review comments

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Avoid list copy by casting array

Signed-off-by: Daniel Widdis <widdis@gmail.com>

---------

Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add integration tests for RemoteRoutingTable Service. (opensearch-project#14631)

Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add SortResponseProcessor to Search Pipelines (opensearch-project#14785)

* Add SortResponseProcessor for search pipelines

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add stupid and unnecessary javadocs to satisfy overly strict CI

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Split casting and sorting methods for readability

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Register the sort processor factory

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Address code review comments

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Cast individual list elements to avoid creating two lists

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Add yamlRestTests

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Clarify why there's unusual sorting

Signed-off-by: Daniel Widdis <widdis@gmail.com>

* Use instanceof instead of isAssignableFrom

Signed-off-by: Daniel Widdis <widdis@gmail.com>

---------

Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix allowUnmappedFields, mapUnmappedFieldAsString settings to be applied when parsing query string query (opensearch-project#13957)

* Modify to invoke QueryShardContext.fieldMapper() method to apply allowUnmappedFields and mapUnmappedFieldAsString settings

Signed-off-by: imyp92 <pyw5420@gmail.com>

* Add test cases to verify returning 400 responses if unmapped fields are included for some types of query

Signed-off-by: imyp92 <pyw5420@gmail.com>

* Add changelog

Signed-off-by: imyp92 <pyw5420@gmail.com>

---------

Signed-off-by: imyp92 <pyw5420@gmail.com>
Signed-off-by: gaobinlong <gbinlong@amazon.com>
Co-authored-by: gaobinlong <gbinlong@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.microsoft.azure:msal4j from 1.16.0 to 1.16.1 in /plugins/repository-azure (opensearch-project#14857)

* Bump com.microsoft.azure:msal4j in /plugins/repository-azure

Bumps [com.microsoft.azure:msal4j](https://github.com/AzureAD/microsoft-authentication-library-for-java) from 1.16.0 to 1.16.1.
- [Release notes](https://github.com/AzureAD/microsoft-authentication-library-for-java/releases)
- [Changelog](https://github.com/AzureAD/microsoft-authentication-library-for-java/blob/dev/changelog.txt)
- [Commits](AzureAD/microsoft-authentication-library-for-java@v1.16.0...v1.16.1)

---
updated-dependencies:
- dependency-name: com.microsoft.azure:msal4j
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Updating SHAs

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump com.gradle.develocity from 3.17.5 to 3.17.6 (opensearch-project#14856)

* Bump com.gradle.develocity from 3.17.5 to 3.17.6

Bumps com.gradle.develocity from 3.17.5 to 3.17.6.

---
updated-dependencies:
- dependency-name: com.gradle.develocity
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump org.jline:jline in /test/fixtures/hdfs-fixture (opensearch-project#14859)

Bumps [org.jline:jline](https://github.com/jline/jline3) from 3.26.2 to 3.26.3.
- [Release notes](https://github.com/jline/jline3/releases)
- [Changelog](https://github.com/jline/jline3/blob/master/changelog.md)
- [Commits](jline/jline3@jline-parent-3.26.2...jline-parent-3.26.3)

---
updated-dependencies:
- dependency-name: org.jline:jline
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use Lucene provided Persian stem (opensearch-project#14847)

Lucene provided Persian stem apparently isn't hooked yet and this change is
doing that based on what is done for Arabic stem support.

Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org>
Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Bump actions/checkout from 2 to 4 (opensearch-project#14858)

* Bump actions/checkout from 2 to 4

Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v2...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update changelog

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Deprecate batch_size parameter on bulk API (opensearch-project#14725)

By default the full _bulk payload will be passed to ingest processors as a
batch, with any sub batching logic to be implemented by each processor if
necessary.

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add perms for remote snapshot cache eviction on scripted query (opensearch-project#14411)

Signed-off-by: Finn Carroll <carrofin@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add transport interceptor to populate queryGroupId in task headers

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Add rest, transport layer changes for Hot to warm tiering - dedicated setup (opensearch-project#13980)

Signed-off-by: Neetika Singhal <neetiks@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Create listener to refresh search thread resource usage (opensearch-project#14832)

* [bug fix] fix incorrect coordinator node search resource usages

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* fix bug on serialization when passing task resource usage to coordinator

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* add more unit tests

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* remove query insights plugin related code

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* create per request listener to refresh task resource usage

Signed-off-by: Chenyang Ji <cyji@amazon.com>

* Make new listener API public

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Add changelog

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Remove wrong files added

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Address review comments

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Build fix

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Make singleton

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Address review comments

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Make sure listener runs before plugin listeners

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Spotless

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

* Minor fix

Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>

---------

Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: Jay Deng <jayd0104@gmail.com>
Co-authored-by: Chenyang Ji <cyji@amazon.com>
Co-authored-by: Jay Deng <jayd0104@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Caching avg total bytes and avg free bytes inside ClusterInfo (opensearch-project#14851)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Use default value when index.number_of_replicas is null (opensearch-project#14812)

* Use default value when index.number_of_replicas is null

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

* Add integration test

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

* Add changelog

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>

---------

Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* [Remote Routing Table] Implement write and read flow for shard diff file. (opensearch-project#14684)

* Implement write and read flow to upload/download shard diff file.

Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Optimized ClusterStatsIndices to precomute shard stats (opensearch-project#14426)

* Optimize Cluster Stats Indices to precomute node level stats

Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix constraint bug which allows more primary shards than average primary shards per index (opensearch-project#14908)

Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Optmising AwarenessAllocationDecider for hashmap.get call (opensearch-project#14761)

Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* update comment

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* Fix IngestServiceTests.testBulkRequestExecutionWithFailures (opensearch-project#14918)

The test would previously fail if the randomness led to only a single
indexing request being included in the bulk payload. This change
guarantees multiple indexing requests in order to ensure the batch logic
kicks in.

Also replace some unneeded mocks with real classes.

Signed-off-by: Andrew Ross <andrross@amazon.com>
Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add queryGroupTask

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* remove unnecessary imports

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add QueryGroupTask tests

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* rename WLM transport request handler

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* add CHANGELOG entry

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix ut

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* address comments

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* fix UT to remove the verify for final method

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

* apply spotless

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

---------

Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Signed-off-by: Gao Binlong <gbinlong@amazon.com>
Signed-off-by: RS146BIJAY <rishavsagar4b1@gmail.com>
Signed-off-by: Chenyang Ji <cyji@amazon.com>
Signed-off-by: Craig Perkins <cwperx@amazon.com>
Signed-off-by: Sandesh Kumar <sandeshkr419@gmail.com>
Signed-off-by: Andriy Redko <andriy.redko@aiven.io>
Signed-off-by: Zelin Hao <zelinhao@amazon.com>
Signed-off-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Signed-off-by: kkewwei <kkewwei@163.com>
Signed-off-by: Swetha Guptha <gupthasg@amazon.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: ahmedsobeh <ahmed.sobeh@aiven.io>
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Siddhant Deshmukh <deshsid@amazon.com>
Signed-off-by: Peter Alfonsi <petealft@amazon.com>
Signed-off-by: vatsal <vatsal.v.anand@gmail.com>
Signed-off-by: Sachin Kale <kalsac@amazon.com>
Signed-off-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Signed-off-by: Craig Perkins <craig5008@gmail.com>
Signed-off-by: mgodwan <mgodwan@amazon.com>
Signed-off-by: Mohit Godwani <mgodwan@amazon.com>
Signed-off-by: Sagar Upadhyaya <sagar.upadhyaya.121@gmail.com>
Signed-off-by: Sandeep Kumawat <skumwt@amazon.com>
Signed-off-by: Daniil Roman <daniilroman.cv@gmail.com>
Signed-off-by: Daniil Roman <danroman17397@gmail.com>
Signed-off-by: Rishabh Singh <sngri@amazon.com>
Signed-off-by: Rishabh Singh <rishabhksingh@gmail.com>
Signed-off-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com>
Signed-off-by: Pranshu Shukla <pranshushukla06@gmail.com>
Signed-off-by: Rajiv Kumar Vaidyanathan <rajivkv@amazon.com>
Signed-off-by: Sumit Bansal <sumitsb@amazon.com>
Signed-off-by: Daniel Widdis <widdis@gmail.com>
Signed-off-by: Shailendra Singh <singhlhs@amazon.com>
Signed-off-by: imyp92 <pyw5420@gmail.com>
Signed-off-by: gaobinlong <gbinlong@amazon.com>
Signed-off-by: Ebrahim Byagowi <ebrahim@gnu.org>
Signed-off-by: Liyun Xiu <xiliyun@amazon.com>
Signed-off-by: Finn Carroll <carrofin@amazon.com>
Signed-off-by: Neetika Singhal <neetiks@amazon.com>
Signed-off-by: Jay Deng <jayd0104@gmail.com>
Signed-off-by: Gaurav Bafna <gbbafna@amazon.com>
Signed-off-by: Andrew Ross <andrross@amazon.com>
Co-authored-by: Shivansh Arora <hishiv@amazon.com>
Co-authored-by: Arpit-Bandejiya <abandeji@amazon.com>
Co-authored-by: gaobinlong <gbinlong@amazon.com>
Co-authored-by: rishavz_sagar <rishavsagar4b1@gmail.com>
Co-authored-by: Chenyang Ji <cyji@amazon.com>
Co-authored-by: Craig Perkins <cwperx@amazon.com>
Co-authored-by: Sandesh Kumar <sandeshkr419@gmail.com>
Co-authored-by: Andriy Redko <andriy.redko@aiven.io>
Co-authored-by: Zelin Hao <zelinhao@amazon.com>
Co-authored-by: Lukáš Vlček <lukas.vlcek@aiven.io>
Co-authored-by: kkewwei <kkewwei@163.com>
Co-authored-by: SwethaGuptha <156877431+SwethaGuptha@users.noreply.github.com>
Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com>
Co-authored-by: Ahmed Sobeh <ahmed.sobeh@aiven.io>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Siddhant Deshmukh <deshsid@amazon.com>
Co-authored-by: Peter Alfonsi <peter.alfonsi@gmail.com>
Co-authored-by: Peter Alfonsi <petealft@amazon.com>
Co-authored-by: Vatsal <36672090+imvtsl@users.noreply.github.com>
Co-authored-by: Robin Friedmann <robinfriedmann.rf@gmail.com>
Co-authored-by: Sachin Kale <sachinpkale@gmail.com>
Co-authored-by: Bharathwaj G <bharath78910@gmail.com>
Co-authored-by: Shourya Dutta Biswas <114977491+shourya035@users.noreply.github.com>
Co-authored-by: Craig Perkins <craig5008@gmail.com>
Co-authored-by: Andriy Redko <drreta@gmail.com>
Co-authored-by: Mohit Godwani <81609427+mgodwan@users.noreply.github.com>
Co-authored-by: Sarthak Aggarwal <sarthagg@amazon.com>
Co-authored-by: Sagar <99425694+sgup432@users.noreply.github.com>
Co-authored-by: Sandeep Kumawat <2025sandeepkumawat@gmail.com>
Co-authored-by: Sandeep Kumawat <skumwt@amazon.com>
Co-authored-by: Daniil Roman <danroman17397@gmail.com>
Co-authored-by: Rishabh Singh <rishabhksingh@gmail.com>
Co-authored-by: kkewwei <kewei.11@bytedance.com>
Co-authored-by: Daniel (dB.) Doubrovkine <dblock@amazon.com>
Co-authored-by: Ashish Singh <ssashish@amazon.com>
Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com>
Co-authored-by: Pranshu Shukla <55992439+Pranshu-S@users.noreply.github.com>
Co-authored-by: rajiv-kv <157019998+rajiv-kv@users.noreply.github.com>
Co-authored-by: Sumit Bansal <sumit.asr@gmail.com>
Co-authored-by: Daniel Widdis <widdis@gmail.com>
Co-authored-by: shailendra0811 <167273922+shailendra0811@users.noreply.github.com>
Co-authored-by: Park, Yeongwu <pyw5420@gmail.com>
Co-authored-by: ebraminio <ebraminio@gmail.com>
Co-authored-by: Liyun Xiu <xiliyun@amazon.com>
Co-authored-by: Finn <carrofin@amazon.com>
Co-authored-by: Neetika Singhal <neetiks@amazon.com>
Co-authored-by: Jay Deng <jayd0104@gmail.com>
Co-authored-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com>
Co-authored-by: Andrew Ross <andrross@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@deshsidd deshsidd

Labels
bug Something isn't working good first issue Good for newcomers Search Search query, autocomplete ...etc
Projects
Search Project Board
Archived in project
Milestone
No milestone
5 participants
@peternied @jainankitk @deshsidd @getsaurabh02 @cwperks