fix stale remote cluster uuid state not purged from remote #10016

linuxpi · 2023-09-13T11:35:48Z

Description

[Describe what this change achieves]

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2023-09-13T12:06:32Z

Compatibility status:

Checks if related components are compatible with change 4a64bb9

Incompatible components

Incompatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]

github-actions · 2023-09-13T12:12:13Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25480/
CommitID: 378dcec
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-09-13T13:16:03Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25485/
CommitID: eb7600f
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-09-13T18:01:32Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.remotestore.RemoteStoreStatsIT.testStatsResponseFromLocalNode

URL: https://build.ci.opensearch.org/job/gradle-check/25497/
CommitID: a0f4a86
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

codecov · 2023-09-13T18:02:19Z

Codecov Report

Merging #10016 (4a64bb9) into main (2965e69) will increase coverage by 0.12%.
Report is 7 commits behind head on main.
The diff coverage is 79.49%.

@@             Coverage Diff              @@
##               main   #10016      +/-   ##
============================================
+ Coverage     71.16%   71.28%   +0.12%     
- Complexity    58297    58439     +142     
============================================
  Files          4830     4843      +13     
  Lines        274903   275205     +302     
  Branches      40059    40068       +9     
============================================
+ Hits         195637   196192     +555     
+ Misses        62882    62607     -275     
- Partials      16384    16406      +22

Files	Coverage Δ
...arch/telemetry/metrics/DefaultMetricsRegistry.java	`100.00% <100.00%> (ø)`
...ch/telemetry/metrics/noop/NoopMetricsRegistry.java	`100.00% <100.00%> (ø)`
...va/org/opensearch/telemetry/metrics/tags/Tags.java	`100.00% <100.00%> (ø)`
...org/opensearch/repositories/url/URLRepository.java	`72.00% <100.00%> (ø)`
...g/opensearch/transport/netty4/Netty4Transport.java	`73.65% <100.00%> (ø)`
...opensearch/repositories/azure/AzureRepository.java	`68.88% <100.00%> (-1.33%)`	⬇️
...g/opensearch/repositories/hdfs/HdfsRepository.java	`61.90% <100.00%> (ø)`
...java/org/opensearch/repositories/s3/S3Service.java	`74.47% <100.00%> (-0.53%)`	⬇️
.../org/opensearch/telemetry/OTelTelemetryPlugin.java	`100.00% <100.00%> (ø)`
...rg/opensearch/telemetry/OTelTelemetrySettings.java	`100.00% <100.00%> (ø)`
... and 39 more

... and 445 files with indirect coverage changes

github-actions · 2023-09-14T06:07:38Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/25561/
CommitID: 210ae69
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java

github-actions · 2023-10-04T13:55:36Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/26898/
CommitID: b23f8c3
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: bansvaru <bansvaru@amazon.com>

github-actions · 2023-10-05T10:59:33Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/27109/
CommitID: b23f8c3
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-10-05T11:40:35Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimarySingleReplica

URL: https://build.ci.opensearch.org/job/gradle-check/27110/
CommitID: 4a64bb9
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

* fix stale remote cluster uuid state not purged from remote Signed-off-by: bansvaru <bansvaru@amazon.com> * fix tests Signed-off-by: bansvaru <bansvaru@amazon.com> * use new limit parameter Signed-off-by: bansvaru <bansvaru@amazon.com> * minor refactoring Signed-off-by: bansvaru <bansvaru@amazon.com> * delete index metadata files before manifest file Signed-off-by: bansvaru <bansvaru@amazon.com> * add basic UT Signed-off-by: bansvaru <bansvaru@amazon.com> * delete all data related to a cluster uuid in a single call Signed-off-by: bansvaru <bansvaru@amazon.com> * fix git diff Signed-off-by: bansvaru <bansvaru@amazon.com> * remove unreferenced code Signed-off-by: bansvaru <bansvaru@amazon.com> * fix spa Signed-off-by: bansvaru <bansvaru@amazon.com> --------- Signed-off-by: bansvaru <bansvaru@amazon.com> (cherry picked from commit 1eec0f1) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…10403) --------- (cherry picked from commit 1eec0f1) Signed-off-by: bansvaru <bansvaru@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* fix stale remote cluster uuid state not purged from remote Signed-off-by: bansvaru <bansvaru@amazon.com> * fix tests Signed-off-by: bansvaru <bansvaru@amazon.com> * use new limit parameter Signed-off-by: bansvaru <bansvaru@amazon.com> * minor refactoring Signed-off-by: bansvaru <bansvaru@amazon.com> * delete index metadata files before manifest file Signed-off-by: bansvaru <bansvaru@amazon.com> * add basic UT Signed-off-by: bansvaru <bansvaru@amazon.com> * delete all data related to a cluster uuid in a single call Signed-off-by: bansvaru <bansvaru@amazon.com> * fix git diff Signed-off-by: bansvaru <bansvaru@amazon.com> * remove unreferenced code Signed-off-by: bansvaru <bansvaru@amazon.com> * fix spa Signed-off-by: bansvaru <bansvaru@amazon.com> --------- Signed-off-by: bansvaru <bansvaru@amazon.com> (cherry picked from commit 1eec0f1) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…10446) * fix stale remote cluster uuid state not purged from remote * fix tests * use new limit parameter * minor refactoring * delete index metadata files before manifest file * add basic UT * delete all data related to a cluster uuid in a single call * fix git diff * remove unreferenced code * fix spa --------- (cherry picked from commit 1eec0f1) Signed-off-by: bansvaru <bansvaru@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…h-project#10016) * fix stale remote cluster uuid state not purged from remote Signed-off-by: bansvaru <bansvaru@amazon.com> * fix tests Signed-off-by: bansvaru <bansvaru@amazon.com> * use new limit parameter Signed-off-by: bansvaru <bansvaru@amazon.com> * minor refactoring Signed-off-by: bansvaru <bansvaru@amazon.com> * delete index metadata files before manifest file Signed-off-by: bansvaru <bansvaru@amazon.com> * add basic UT Signed-off-by: bansvaru <bansvaru@amazon.com> * delete all data related to a cluster uuid in a single call Signed-off-by: bansvaru <bansvaru@amazon.com> * fix git diff Signed-off-by: bansvaru <bansvaru@amazon.com> * remove unreferenced code Signed-off-by: bansvaru <bansvaru@amazon.com> * fix spa Signed-off-by: bansvaru <bansvaru@amazon.com> --------- Signed-off-by: bansvaru <bansvaru@amazon.com>

…h-project#10016) * fix stale remote cluster uuid state not purged from remote Signed-off-by: bansvaru <bansvaru@amazon.com> * fix tests Signed-off-by: bansvaru <bansvaru@amazon.com> * use new limit parameter Signed-off-by: bansvaru <bansvaru@amazon.com> * minor refactoring Signed-off-by: bansvaru <bansvaru@amazon.com> * delete index metadata files before manifest file Signed-off-by: bansvaru <bansvaru@amazon.com> * add basic UT Signed-off-by: bansvaru <bansvaru@amazon.com> * delete all data related to a cluster uuid in a single call Signed-off-by: bansvaru <bansvaru@amazon.com> * fix git diff Signed-off-by: bansvaru <bansvaru@amazon.com> * remove unreferenced code Signed-off-by: bansvaru <bansvaru@amazon.com> * fix spa Signed-off-by: bansvaru <bansvaru@amazon.com> --------- Signed-off-by: bansvaru <bansvaru@amazon.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com>

linuxpi force-pushed the delete-stale-remote-cluster-uuids branch from 378dcec to eb7600f Compare September 13, 2023 12:29

linuxpi force-pushed the delete-stale-remote-cluster-uuids branch from eb7600f to a0f4a86 Compare September 13, 2023 17:18

linuxpi force-pushed the delete-stale-remote-cluster-uuids branch from a0f4a86 to 210ae69 Compare September 14, 2023 05:38

soosinha reviewed Sep 14, 2023

View reviewed changes

server/src/main/java/org/opensearch/gateway/remote/RemoteClusterStateService.java Show resolved Hide resolved

linuxpi marked this pull request as ready for review September 15, 2023 04:39

linuxpi requested review from reta, anasalkouz, andrross, Bukhtawar, CEHENKLE, dblock, gbbafna, setiah, kartg, kotwanikunal, mch2, nknize, owaiskazi19, peternied, Rishikesh1159, ryanbogan, saratvemulapalli, shwetathareja and dreamer-89 as code owners September 15, 2023 04:39

sachinpkale approved these changes Oct 5, 2023

View reviewed changes

sachinpkale added the skip-changelog label Oct 5, 2023

linuxpi added 10 commits October 5, 2023 16:26

fix stale remote cluster uuid state not purged from remote

c25116c

Signed-off-by: bansvaru <bansvaru@amazon.com>

fix tests

e62a341

Signed-off-by: bansvaru <bansvaru@amazon.com>

use new limit parameter

0a6c838

Signed-off-by: bansvaru <bansvaru@amazon.com>

minor refactoring

4bdd7b7

Signed-off-by: bansvaru <bansvaru@amazon.com>

delete index metadata files before manifest file

31b9d9d

Signed-off-by: bansvaru <bansvaru@amazon.com>

add basic UT

2fbc281

Signed-off-by: bansvaru <bansvaru@amazon.com>

delete all data related to a cluster uuid in a single call

913250d

Signed-off-by: bansvaru <bansvaru@amazon.com>

fix git diff

abb9e5c

Signed-off-by: bansvaru <bansvaru@amazon.com>

remove unreferenced code

ef761cf

Signed-off-by: bansvaru <bansvaru@amazon.com>

fix spa

4a64bb9

Signed-off-by: bansvaru <bansvaru@amazon.com>

linuxpi force-pushed the delete-stale-remote-cluster-uuids branch from b23f8c3 to 4a64bb9 Compare October 5, 2023 10:57

sachinpkale merged commit 1eec0f1 into opensearch-project:main Oct 5, 2023
12 of 13 checks passed

sachinpkale added the backport 2.x Backport to 2.x branch label Oct 5, 2023

opensearch-trigger-bot bot mentioned this pull request Oct 5, 2023

[Backport 2.x] fix stale remote cluster uuid state not purged from remote #10403

Merged

linuxpi deleted the delete-stale-remote-cluster-uuids branch October 5, 2023 16:48

sachinpkale added the backport 2.11 label Oct 6, 2023

opensearch-trigger-bot bot mentioned this pull request Oct 6, 2023

[Backport 2.11] fix stale remote cluster uuid state not purged from remote #10446

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix stale remote cluster uuid state not purged from remote #10016

fix stale remote cluster uuid state not purged from remote #10016

linuxpi commented Sep 13, 2023

github-actions bot commented Sep 13, 2023 •

edited

Loading

github-actions bot commented Sep 13, 2023

github-actions bot commented Sep 13, 2023

github-actions bot commented Sep 13, 2023

codecov bot commented Sep 13, 2023 •

edited

Loading

github-actions bot commented Sep 14, 2023

github-actions bot commented Oct 4, 2023

github-actions bot commented Oct 5, 2023

github-actions bot commented Oct 5, 2023

fix stale remote cluster uuid state not purged from remote #10016

fix stale remote cluster uuid state not purged from remote #10016

Conversation

linuxpi commented Sep 13, 2023

Description

Related Issues

Check List

github-actions bot commented Sep 13, 2023 • edited Loading

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Sep 13, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 13, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 13, 2023

Gradle Check (Jenkins) Run Completed with:

codecov bot commented Sep 13, 2023 • edited Loading

Codecov Report

github-actions bot commented Sep 14, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 4, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 5, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 5, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 13, 2023 •

edited

Loading

codecov bot commented Sep 13, 2023 •

edited

Loading