-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add transport action for primary term validation for remote-backed indices #5616
Conversation
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
Codecov Report
@@ Coverage Diff @@
## main #5616 +/- ##
============================================
+ Coverage 70.84% 70.93% +0.09%
- Complexity 58296 58339 +43
============================================
Files 4741 4740 -1
Lines 278541 278635 +94
Branches 40268 40273 +5
============================================
+ Hits 197322 197654 +332
+ Misses 65068 64839 -229
+ Partials 16151 16142 -9
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
But no-op replication can be considered as a big feature, right? |
No-op itself is not a feature, but a part of "Request level durability for remote backed indices". Refer this comment - #4954 (comment) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we need to validate if clusterService.state().getNodes().getMinNodeVersion()
of all nodes is greater than this version to ensure remote translog works. Since this can break during mixed versions rolling upgrades
Had discussed this offline with @gbbafna and @sachinpkale. Major point around this was - If there is a mixed cluster with let's say older version and 2.5, then a primary on 2.5 will send requests to replicas on older version which can fail to understand this. However, since this is an experimental feature, We can handle this when removing the experimental feature flag. This also seems to be the status quo as the segment replication feature which is also behind the experimental feature flag has PublishCheckpointAction which older version replicas would not understand when a index is created in a mixed cluster setup. |
Can you please open up a tracking issue for the same? |
Created #5635 for the same. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we add tests to ensure that primary term and no replication modes are used correctly across TransportActions
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Added |
Gradle Check (Jenkins) Run Completed with:
|
Signed-off-by: Ashish Singh <ssashish@amazon.com>
Gradle Check (Jenkins) Run Completed with:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the UTs can we open a follow up for an Integ test for the transport calls and validating primary term failure cases during isolated writes
Tracking issue - #5636 |
…dices (opensearch-project#5616) Add transport action for primary term validation for remote-backed indices Signed-off-by: Ashish Singh <ssashish@amazon.com>
…dices (opensearch-project#5616) Add transport action for primary term validation for remote-backed indices Signed-off-by: Ashish Singh <ssashish@amazon.com>
…dices (opensearch-project#5616) Add transport action for primary term validation for remote-backed indices Signed-off-by: Ashish Singh <ssashish@amazon.com>
…5731) * Remove PRRL creation/deletion in peer recovery of remote store enabled replica (#4954) * Add RecoverySourceHandlerFactory for extensibility Signed-off-by: Ashish Singh <ssashish@amazon.com> * recoverToTarget made extensible to allow multiple implementations Signed-off-by: Ashish Singh <ssashish@amazon.com> * Remove PRRL after SendFileStep in Peer Recovery Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> * Empty-Commit Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> * Empty-Commit Signed-off-by: Ashish Singh <ssashish@amazon.com> * Empty-Commit Signed-off-by: Ashish Singh <ssashish@amazon.com> * Remove CHANGELOG entry as this is incremental PR Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> * Enhance CheckpointState to support no-op replication (#5282) * CheckpointState enhanced to support no-op replication Signed-off-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: Bukhtawar Khan<bukhtawa@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> * Add transport action for primary term validation for remote-backed indices (#5616) Add transport action for primary term validation for remote-backed indices Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com>
…pensearch-project#5731) * Remove PRRL creation/deletion in peer recovery of remote store enabled replica (opensearch-project#4954) * Add RecoverySourceHandlerFactory for extensibility Signed-off-by: Ashish Singh <ssashish@amazon.com> * recoverToTarget made extensible to allow multiple implementations Signed-off-by: Ashish Singh <ssashish@amazon.com> * Remove PRRL after SendFileStep in Peer Recovery Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> * Empty-Commit Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> * Empty-Commit Signed-off-by: Ashish Singh <ssashish@amazon.com> * Empty-Commit Signed-off-by: Ashish Singh <ssashish@amazon.com> * Remove CHANGELOG entry as this is incremental PR Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> * Enhance CheckpointState to support no-op replication (opensearch-project#5282) * CheckpointState enhanced to support no-op replication Signed-off-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: Bukhtawar Khan<bukhtawa@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> * Add transport action for primary term validation for remote-backed indices (opensearch-project#5616) Add transport action for primary term validation for remote-backed indices Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com>
* Remove PRRL creation/deletion in peer recovery of remote store enabled replica (#4954) * Add RecoverySourceHandlerFactory for extensibility Signed-off-by: Ashish Singh <ssashish@amazon.com> * recoverToTarget made extensible to allow multiple implementations Signed-off-by: Ashish Singh <ssashish@amazon.com> * Remove PRRL after SendFileStep in Peer Recovery Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> * Empty-Commit Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> * Empty-Commit Signed-off-by: Ashish Singh <ssashish@amazon.com> * Empty-Commit Signed-off-by: Ashish Singh <ssashish@amazon.com> * Remove CHANGELOG entry as this is incremental PR Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> * Enhance CheckpointState to support no-op replication (#5282) * CheckpointState enhanced to support no-op replication Signed-off-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: Bukhtawar Khan<bukhtawa@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> * Add transport action for primary term validation for remote-backed indices (#5616) Add transport action for primary term validation for remote-backed indices Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: Ashish <ssashish@amazon.com>
…5731) * Remove PRRL creation/deletion in peer recovery of remote store enabled replica (#4954) * Add RecoverySourceHandlerFactory for extensibility Signed-off-by: Ashish Singh <ssashish@amazon.com> * recoverToTarget made extensible to allow multiple implementations Signed-off-by: Ashish Singh <ssashish@amazon.com> * Remove PRRL after SendFileStep in Peer Recovery Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> * Empty-Commit Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> * Empty-Commit Signed-off-by: Ashish Singh <ssashish@amazon.com> * Empty-Commit Signed-off-by: Ashish Singh <ssashish@amazon.com> * Remove CHANGELOG entry as this is incremental PR Signed-off-by: Ashish Singh <ssashish@amazon.com> * Incorporate PR review feedback Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> * Enhance CheckpointState to support no-op replication (#5282) * CheckpointState enhanced to support no-op replication Signed-off-by: Ashish Singh <ssashish@amazon.com> Co-authored-by: Bukhtawar Khan<bukhtawa@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com> * Add transport action for primary term validation for remote-backed indices (#5616) Add transport action for primary term validation for remote-backed indices Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Ashish Singh <ssashish@amazon.com>
Signed-off-by: Ashish Singh ssashish@amazon.com
Description
Solves #5464
Issues Resolved
Solves #5464
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.