Backport/backport 8377 to 2.x

[indrajohn7]

Description

This PR is to discuss the optimisation changes for listener based get_snapshot calls handled through async based model.
This is a backport commit of the PR: #8377

Related Issues

Resolves #1788

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[opensearch-trigger-bot]

Compatibility status:

Checking compatibility for: https://github.com/opensearch-project/ml-commons.git with ref: 2.x
Checking compatibility for: https://github.com/opensearch-project/common-utils.git with ref: 2.x
Checking compatibility for: https://github.com/opensearch-project/reporting.git with ref: 2.x
Incompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/ml-commons.git]
Components skipped due to git failures: [https://github.com/opensearch-project/opensearch-oci-object-storage.git]
Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]

BUILD SUCCESSFUL in 26m 55s

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❕
• TEST FAILURES:

      2 org.opensearch.search.SearchWeightedRoutingIT.testStrictWeightedRoutingWithCustomString

• URL: https://build.ci.opensearch.org/job/gradle-check/22094/
• CommitID: c94f06e
  Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

Merging #9167 (c94f06e) into 2.x (e59b21a) will decrease coverage by 0.06%.
Report is 7 commits behind head on 2.x.
The diff coverage is 71.61%.

@@             Coverage Diff              @@
##                2.x    #9167      +/-   ##
============================================
- Coverage     70.81%   70.76%   -0.06%     
- Complexity    57425    57477      +52     
============================================
  Files          4754     4754              
  Lines        271688   271756      +68     
  Branches      40070    40082      +12     
============================================
- Hits         192408   192319      -89     
- Misses        62717    62979     +262     
+ Partials      16563    16458     -105     

Files Changed	Coverage Δ
.../opensearch/client/indices/CreateIndexRequest.java	75.55% <0.00%> (-2.23%)	⬇️
...opensearch/client/slm/SnapshotLifecyclePolicy.java	0.00% <ø> (ø)
...ch/client/slm/SnapshotLifecyclePolicyMetadata.java	0.00% <ø> (ø)
.../opensearch/client/slm/SnapshotLifecycleStats.java	0.00% <ø> (ø)
...rch/client/slm/SnapshotRetentionConfiguration.java	0.00% <ø> (ø)
...h/script/mustache/MultiSearchTemplateResponse.java	56.25% <ø> (ø)
...rg/opensearch/index/rankeval/RankEvalResponse.java	98.57% <ø> (+10.00%)	⬆️
...va/org/opensearch/index/rankeval/RankEvalSpec.java	88.59% <ø> (+0.87%)	⬆️
...a/org/opensearch/index/rankeval/RatedDocument.java	81.03% <0.00%> (ø)
...va/org/opensearch/index/rankeval/RatedRequest.java	91.79% <ø> (+6.71%)	⬆️
... and 145 more

... and 431 files with indirect coverage changes

[Poojita-Raj]

Gradle Check (Jenkins) Run Completed with:

* **RESULT:** UNSTABLE ❕

* **TEST FAILURES:**

      2 org.opensearch.search.SearchWeightedRoutingIT.testStrictWeightedRoutingWithCustomString

* **URL:** https://build.ci.opensearch.org/job/gradle-check/22094/

* **CommitID:** [c94f06e](https://github.com/opensearch-project/OpenSearch/commit/c94f06e920a375aeb350431f574217791179d8b5)
  Please review all [flaky tests](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#flaky-tests) that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Flaky - #8030