-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fetch previous cluster UUID #9746
Fetch previous cluster UUID #9746
Conversation
Gradle Check (Jenkins) Run Completed with:
|
Compatibility status:Checks if related components are compatible with change 56a2c81 Incompatible componentsSkipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git] |
server/src/main/java/org/opensearch/gateway/remote/ClusterMetadataManifest.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Outdated
Show resolved
Hide resolved
56a2c81
to
f2cc7ab
Compare
Compatibility status:Checks if related components are compatible with change 660675c Incompatible componentsIncompatible components: [https://github.com/opensearch-project/cross-cluster-replication.git] Skipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/reporting.git] |
Flaky test: #8980 |
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Show resolved
Hide resolved
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Show resolved
Hide resolved
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
660675c
to
3ba0108
Compare
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Show resolved
Hide resolved
Compatibility status:Checks if related components are compatible with change 3ba0108 Incompatible componentsSkipped componentsCompatible componentsCompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git] |
Gradle Check (Jenkins) Run Completed with:
|
The backport to
To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-9746-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 bdf6f1d09702aebd8b1e94a2cb722c14a6825297
# Push it to GitHub
git push --set-upstream origin backport/backport-9746-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x Then, create a pull request where the |
The backport to
To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.10 2.10
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.10
# Create a new branch
git switch --create backport/backport-9746-to-2.10
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 bdf6f1d09702aebd8b1e94a2cb722c14a6825297
# Push it to GitHub
git push --set-upstream origin backport/backport-9746-to-2.10
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.10 Then, create a pull request where the |
server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java
Show resolved
Hide resolved
final Map<String, String> clusterUUIDGraph = manifestsByClusterUUID.values() | ||
.stream() | ||
.collect(Collectors.toMap(ClusterMetadataManifest::getClusterUUID, ClusterMetadataManifest::getPreviousClusterUUID)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can there be cases of isolated masters where more than 1 master can be publishing the metadata? how are we handling such cases?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
multiple cluster managers can only exist intermittently and one of the cluster managers will fail when they try to validate the quorum since another cluster manager is holding the quorum now. during publish and commit phase, quorum of cluster manager nodes need to approve of the operation.
such cases are handed by ensuring we always pick a committed uuid from remote(isInvalidClusterUUID
) and trim any duplicate UUIDs using our trimming logic
Signed-off-by: Sooraj Sinha <soosinha@amazon.com> (cherry picked from commit bdf6f1d)
Signed-off-by: Sooraj Sinha <soosinha@amazon.com> (cherry picked from commit bdf6f1d)
Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Ivan Brusic <ivan.brusic@flocksafety.com>
Signed-off-by: Sooraj Sinha <soosinha@amazon.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Description
This PR has the logic to get the previous cluster UUID. The previous cluster UUID will be used to restore the metadata from remote store while bootstrap.
Related Issues
#9821
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.