Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Weighted Shard Routing] API versioning #5255

Merged
merged 28 commits into from
Jan 9, 2023

Conversation

anshu1106
Copy link
Contributor

@anshu1106 anshu1106 commented Nov 15, 2022

Signed-off-by: Anshu Agarwal anshukag@amazon.com

Description

This PR adds support for versioning weighted routing apis. PUT and GET is versioned in this PR.

Issues Resolved

#5038

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
@anshu1106 anshu1106 requested review from a team and reta as code owners November 15, 2022 10:35
@anshu1106 anshu1106 marked this pull request as draft November 15, 2022 10:35
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

Anshu Agarwal added 2 commits December 13, 2022 20:42
Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
@anshu1106 anshu1106 force-pushed the feature/api-versioning branch from 1e115f0 to ffc638b Compare January 5, 2023 13:51
@anshu1106
Copy link
Contributor Author

Gradle Check (Jenkins) Run Completed with:

Unrelated test failures, building again

@github-actions
Copy link
Contributor

github-actions bot commented Jan 5, 2023

Gradle Check (Jenkins) Run Completed with:

@anshu1106 anshu1106 force-pushed the feature/api-versioning branch from ffc638b to 141298e Compare January 5, 2023 14:30
@github-actions
Copy link
Contributor

github-actions bot commented Jan 5, 2023

Gradle Check (Jenkins) Run Completed with:

@anshu1106
Copy link
Contributor Author

anshu1106 commented Jan 6, 2023

Gradle Check (Jenkins) Run Completed with:

Don't see any issue opened for PIT test failures. I tried running the tests in my laptop, failures are happening in main branch as well, but its flaky because I could see a lot of builds passing, but somehow consistent failure in this PR

Tests with failures:
 - org.opensearch.search.DeletePitMultiNodeTests.testDeletePitWithValidAndDeletedIds
 - org.opensearch.search.DeletePitMultiNodeTests.testDeleteWhileSearch
 - org.opensearch.search.DeletePitMultiNodeTests.testDeletePitWhileNodeDrop
 - org.opensearch.search.DeletePitMultiNodeTests.testDeleteAllPits
 - org.opensearch.search.DeletePitMultiNodeTests.testDeletePit
 - org.opensearch.search.DeletePitMultiNodeTests.testtConcurrentDeletes
 - org.opensearch.search.DeletePitMultiNodeTests.testDeleteAllPitsWhileNodeDrop
 - org.opensearch.search.DeletePitMultiNodeTests.testDeletePitWithValidAndInvalidIds
 - org.opensearch.search.DeletePitMultiNodeTests.classMethod

11500 tests completed, 9 failed, 28 skipped

> Task :server:test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':server:test'.
> There were failing tests. See the report at: file:///Users/anshukag/work/OpenSearch5/OpenSearch/server/build/reports/tests/test/index.html

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.

* Get more help at https://help.gradle.org

BUILD FAILED in 28m 52s
42 actionable tasks: 4 executed, 38 up-to-date
anshukag@88665a35ea6c OpenSearch % git branch
  feature/api-versioning
  feature/poc-fail-open
  feature/wrr-deprecate-get-local
* main
  test
  wrr-handle-multiple-awareness-attribute
anshukag@88665a35ea6c OpenSearch % git status
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean

@anshu1106 anshu1106 force-pushed the feature/api-versioning branch from 141298e to a7cb7eb Compare January 6, 2023 06:39
@github-actions
Copy link
Contributor

github-actions bot commented Jan 6, 2023

Gradle Check (Jenkins) Run Completed with:

@rramachand21 rramachand21 added v2.5.0 'Issues and PRs related to version v2.5.0' roadmap labels Jan 6, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Jan 8, 2023

Gradle Check (Jenkins) Run Completed with:

Anshu Agarwal added 2 commits January 9, 2023 11:12
Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
@github-actions
Copy link
Contributor

github-actions bot commented Jan 9, 2023

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      2 org.opensearch.common.util.concurrent.QueueResizableOpenSearchThreadPoolExecutorTests.classMethod
      1 org.opensearch.common.util.concurrent.QueueResizableOpenSearchThreadPoolExecutorTests.testResizeQueueDown

@codecov-commenter
Copy link

Codecov Report

Merging #5255 (9a9e35f) into main (2504be2) will decrease coverage by 0.44%.
The diff coverage is 58.16%.

@@             Coverage Diff              @@
##               main    #5255      +/-   ##
============================================
- Coverage     70.93%   70.48%   -0.45%     
+ Complexity    58713    58410     -303     
============================================
  Files          4767     4767              
  Lines        280162   280308     +146     
  Branches      40441    40473      +32     
============================================
- Hits         198744   197587    -1157     
- Misses        65199    66641    +1442     
+ Partials      16219    16080     -139     
Impacted Files Coverage Δ
...te/ClusterDeleteWeightedRoutingRequestBuilder.java 0.00% <0.00%> (ø)
...d/put/ClusterPutWeightedRoutingRequestBuilder.java 66.66% <0.00%> (-33.34%) ⬇️
...g/opensearch/cluster/routing/OperationRouting.java 86.46% <0.00%> (ø)
...eighted/get/TransportGetWeightedRoutingAction.java 75.00% <50.00%> (+1.31%) ⬆️
...weighted/put/ClusterPutWeightedRoutingRequest.java 63.54% <51.61%> (-4.07%) ⬇️
...ed/delete/ClusterDeleteWeightedRoutingRequest.java 51.92% <55.31%> (+37.63%) ⬆️
...eighted/get/ClusterGetWeightedRoutingResponse.java 57.69% <55.55%> (-4.58%) ⬇️
...search/cluster/routing/WeightedRoutingService.java 75.39% <60.00%> (-4.05%) ⬇️
...arch/cluster/metadata/WeightedRoutingMetadata.java 59.72% <72.41%> (-1.69%) ⬇️
...rg/opensearch/cluster/routing/WeightedRouting.java 65.51% <80.00%> (+19.68%) ⬆️
... and 501 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@Bukhtawar Bukhtawar merged commit e0ddf52 into opensearch-project:main Jan 9, 2023
@Bukhtawar Bukhtawar added the backport 2.x Backport to 2.x branch label Jan 9, 2023
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-5255-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 e0ddf524e705097a5e52ddbc9f7045680a38ffb2
# Push it to GitHub
git push --set-upstream origin backport/backport-5255-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-5255-to-2.x.

anshu1106 added a commit to anshu1106/OpenSearch that referenced this pull request Jan 10, 2023
* Support API versioning for weighted shard routing

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
anshu1106 added a commit to anshu1106/OpenSearch that referenced this pull request Jan 10, 2023
* Support API versioning for weighted shard routing

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
anshu1106 added a commit to anshu1106/OpenSearch that referenced this pull request Jan 10, 2023
* Support API versioning for weighted shard routing

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
gbbafna pushed a commit that referenced this pull request Jan 10, 2023
#5784)

* [Weighted Routing] Add support for discovered master and remove local weights in the response (#5680)

* Add support for discovered master and remove local weights in the weighted routing API response

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* [Weighted Shard Routing] API versioning (#5255)

* Support API versioning for weighted shard routing

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* [Weighted Shard Routing] Fail open requests on search shard failures (#5072)

* Fail open requests on search shard failures (

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* Address fail open comments (#5778)

[Weighted Shard Routing] Refactor and fix singleton in FailAwareWeightedRouting

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* remove unintended changes in changelog

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

* remove unintended changes from changelog

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
Co-authored-by: Anshu Agarwal <anshukag@amazon.com>
Bukhtawar pushed a commit that referenced this pull request Jan 10, 2023
…ed shard routing (#5781)

* [Backport 2.x] 
[Weighted Shard Routing] Add support for discovered master and remove local weights in the response #5680
[Weighted Shard Routing] API versioning #5255
[Weighted Shard Routing] Fail open requests on search shard failures #5072
[Weighted Shard Routing] Refactor and fix singleton in FailAwareWeightedRouting #5778

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
kotwanikunal pushed a commit that referenced this pull request Jan 25, 2023
…ed shard routing (#5781)

* [Backport 2.x] 
[Weighted Shard Routing] Add support for discovered master and remove local weights in the response #5680
[Weighted Shard Routing] API versioning #5255
[Weighted Shard Routing] Fail open requests on search shard failures #5072
[Weighted Shard Routing] Refactor and fix singleton in FailAwareWeightedRouting #5778

Signed-off-by: Anshu Agarwal <anshukag@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch v2.5.0 'Issues and PRs related to version v2.5.0'
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants