Update Shallow Snapshot flows to support remote path type & hash algo #12988

ashking94 · 2024-03-31T07:30:36Z

Description

This PR is a sub task of feature request #12567 and being tracked in meta issue - #12589.

In this PR, we have wired the RemoteStorePathStrategy to Shallow Snapshots. We have written new tests and modified the existing tests as required. We handle following senarios -

Snapshots and restores
Snapshot clone
Lock file deletion during snapshot deletion
Remote store cleanup after every snapshot deletion

Related Issues

Resolves #12987

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
Commits are signed per the DCO using --signoff
~~[ ] Commit changes are listed out in CHANGELOG.md file (See: Changelog)~~
~~[ ] Public documentation issue/PR created~~

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2024-03-31T07:52:18Z

Compatibility status:

Checks if related components are compatible with change 9f15bbf

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/sql.git]

github-actions · 2024-03-31T08:18:26Z

✅ Gradle check result for 81e3a03: SUCCESS

codecov · 2024-03-31T08:19:13Z

Codecov Report

Attention: Patch coverage is 66.15385% with 44 lines in your changes are missing coverage. Please review.

Project coverage is 71.54%. Comparing base (b15cb0c) to head (c2bb40e).
Report is 134 commits behind head on main.

Files	Patch %	Lines
...blobstore/RemoteStoreShardShallowCopySnapshot.java	57.14%	12 Missing and 12 partials ⚠️
.../org/opensearch/index/remote/RemoteStoreEnums.java	79.41%	4 Missing and 3 partials ⚠️
...ensearch/index/remote/RemoteStorePathStrategy.java	50.00%	2 Missing and 2 partials ⚠️
.../main/java/org/opensearch/index/IndexSettings.java	57.14%	0 Missing and 3 partials ⚠️
...ch/repositories/blobstore/BlobStoreRepository.java	50.00%	3 Missing ⚠️
...h/cluster/metadata/MetadataCreateIndexService.java	92.30%	0 Missing and 1 partial ⚠️
.../index/remote/RemoteStorePathStrategyResolver.java	80.00%	0 Missing and 1 partial ⚠️
...java/org/opensearch/index/shard/StoreRecovery.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #12988      +/-   ##
============================================
+ Coverage     71.42%   71.54%   +0.12%     
- Complexity    59978    60495     +517     
============================================
  Files          4985     5026      +41     
  Lines        282275   284522    +2247     
  Branches      40946    41213     +267     
============================================
+ Hits         201603   203562    +1959     
- Misses        63999    64122     +123     
- Partials      16673    16838     +165

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

.../main/java/org/opensearch/index/snapshots/blobstore/RemoteStoreShardShallowCopySnapshot.java

github-actions · 2024-04-03T09:59:41Z

❕ Gradle check result for 2ef85af: UNSTABLE

TEST FAILURES:

      3 org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
      1 org.opensearch.index.IndexServiceTests.testAsyncTranslogTrimTaskOnClosedIndex
      1 org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationDuringQueryPhase

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

ashking94 · 2024-04-03T10:24:07Z

❕ Gradle check result for 2ef85af: UNSTABLE

TEST FAILURES:
      3 org.opensearch.cluster.MinimumClusterManagerNodesIT.testThreeNodesNoClusterManagerBlock
      1 org.opensearch.index.IndexServiceTests.testAsyncTranslogTrimTaskOnClosedIndex
      1 org.opensearch.http.SearchRestCancellationIT.testAutomaticCancellationDuringQueryPhase
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Flaky tests - #10006, #11547 & #5426

Signed-off-by: Ashish Singh <ssashish@amazon.com>

github-actions · 2024-04-03T13:21:17Z

✅ Gradle check result for 6b53314: SUCCESS

server/src/internalClusterTest/java/org/opensearch/remotestore/RemoteRestoreSnapshotIT.java

server/src/main/java/org/opensearch/cluster/metadata/MetadataCreateIndexService.java

server/src/main/java/org/opensearch/index/remote/RemoteStorePathStrategyResolver.java

server/src/main/java/org/opensearch/cluster/metadata/MetadataCreateIndexService.java

server/src/main/java/org/opensearch/index/IndexSettings.java

server/src/main/java/org/opensearch/index/remote/RemoteStoreEnums.java

.../main/java/org/opensearch/index/snapshots/blobstore/RemoteStoreShardShallowCopySnapshot.java

Signed-off-by: Ashish Singh <ssashish@amazon.com>

github-actions · 2024-04-04T15:43:34Z

❕ Gradle check result for 86a77a3: UNSTABLE

TEST FAILURES:

      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testIndexReopenClose

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Signed-off-by: Ashish Singh <ssashish@amazon.com>

ashking94 · 2024-04-04T18:53:07Z

org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testIndexReopenClose

Flaky test - #13084

github-actions · 2024-04-04T19:13:59Z

✅ Gradle check result for dcccd05: SUCCESS

harishbhakuni

changes lgtm

.../main/java/org/opensearch/index/snapshots/blobstore/RemoteStoreShardShallowCopySnapshot.java

server/src/main/java/org/opensearch/index/IndexSettings.java

server/src/main/java/org/opensearch/index/remote/RemoteStoreEnums.java

.../main/java/org/opensearch/index/snapshots/blobstore/RemoteStoreShardShallowCopySnapshot.java

.../java/org/opensearch/index/snapshots/blobstore/RemoteStoreShardShallowCopySnapshotTests.java

Signed-off-by: Ashish Singh <ssashish@amazon.com>

github-actions · 2024-04-05T08:28:57Z

❕ Gradle check result for 9f15bbf: UNSTABLE

TEST FAILURES:

      1 org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimaryMultipleReplicaShards

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

github-actions · 2024-04-05T08:46:25Z

❌ Gradle check result for c2bb40e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

gbbafna · 2024-04-05T09:17:03Z

org.opensearch.discovery.ClusterDisruptionIT.testAckedIndexing failed . Retrying gradle build .

github-actions · 2024-04-05T10:08:10Z

❕ Gradle check result for c2bb40e: UNSTABLE

TEST FAILURES:

      1 org.opensearch.cluster.coordination.AwarenessAttributeDecommissionIT.testConcurrentDecommissionAction

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

ashking94 · 2024-04-05T10:15:12Z

testConcurrentDecommissionAction

Flaky test - #12197

…opensearch-project#12988) Signed-off-by: Ashish Singh <ssashish@amazon.com>

…opensearch-project#12988) Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com>

…#12988) Signed-off-by: Ashish Singh <ssashish@amazon.com> (cherry picked from commit e713175) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…#12988) (#13410) (cherry picked from commit e713175) Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

github-actions bot added enhancement Enhancement or improvement to existing feature or request Storage:Performance Storage:Resiliency Issues and PRs related to the storage resiliency v2.14.0 labels Mar 31, 2024

ashking94 added the skip-changelog label Apr 1, 2024

harishbhakuni reviewed Apr 1, 2024

View reviewed changes

.../main/java/org/opensearch/index/snapshots/blobstore/RemoteStoreShardShallowCopySnapshot.java Show resolved Hide resolved

ashking94 force-pushed the 12987 branch from 81e3a03 to 2ef85af Compare April 3, 2024 09:09

Update Shallow Snapshot flows to support remote path type & hash algo

6b53314

Signed-off-by: Ashish Singh <ssashish@amazon.com>

ashking94 force-pushed the 12987 branch from 2ef85af to 6b53314 Compare April 3, 2024 12:34

ashking94 requested a review from harishbhakuni April 3, 2024 12:41

ashking94 marked this pull request as ready for review April 3, 2024 13:22

ashking94 requested review from abbashus, adnapibar, anasalkouz, andrross, Bukhtawar, CEHENKLE, dblock, dbwiddis, dreamer-89, gbbafna, kotwanikunal, mch2 and msfroh as code owners April 3, 2024 13:22

harishbhakuni reviewed Apr 4, 2024

View reviewed changes

server/src/internalClusterTest/java/org/opensearch/remotestore/RemoteRestoreSnapshotIT.java Show resolved Hide resolved

gbbafna reviewed Apr 4, 2024

View reviewed changes

Incorporate PR review feedback

86a77a3

Signed-off-by: Ashish Singh <ssashish@amazon.com>

ashking94 requested review from harishbhakuni and gbbafna April 4, 2024 15:51

Add more test cases in IT

dcccd05

Signed-off-by: Ashish Singh <ssashish@amazon.com>

harishbhakuni approved these changes Apr 5, 2024

View reviewed changes

gbbafna reviewed Apr 5, 2024

View reviewed changes

Incoporate PR review feedback

9f15bbf

Signed-off-by: Ashish Singh <ssashish@amazon.com>

ashking94 requested a review from gbbafna April 5, 2024 07:35

Removed unused ctor

c2bb40e

Signed-off-by: Ashish Singh <ssashish@amazon.com>

gbbafna approved these changes Apr 5, 2024

View reviewed changes

gbbafna merged commit e713175 into opensearch-project:main Apr 5, 2024
28 of 31 checks passed

ashking94 added a commit to ashking94/OpenSearch that referenced this pull request Apr 23, 2024

Update Shallow Snapshot flows to support remote path type & hash algo (…

f139387

…opensearch-project#12988) Signed-off-by: Ashish Singh <ssashish@amazon.com>

ashking94 added the backport 2.x Backport to 2.x branch label Apr 26, 2024

opensearch-trigger-bot bot mentioned this pull request Apr 26, 2024

[Backport 2.x] Update Shallow Snapshot flows to support remote path type & hash algo #13410

Merged

This was referenced Apr 29, 2024

[META] Optimised prefix pattern per shard for remote store data and metadata files for higher throughput #12589

Closed

[Remote Store] Update version to 2.14 in path type min node version check #13436

Merged

Bukhtawar mentioned this pull request Jun 27, 2024

Add Ashish Singh as maintainer #14567

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Shallow Snapshot flows to support remote path type & hash algo #12988

Update Shallow Snapshot flows to support remote path type & hash algo #12988

ashking94 commented Mar 31, 2024 •

edited

Loading

github-actions bot commented Mar 31, 2024 •

edited

Loading

github-actions bot commented Mar 31, 2024

codecov bot commented Mar 31, 2024 •

edited

Loading

github-actions bot commented Apr 3, 2024

ashking94 commented Apr 3, 2024

github-actions bot commented Apr 3, 2024

github-actions bot commented Apr 4, 2024

ashking94 commented Apr 4, 2024

github-actions bot commented Apr 4, 2024

harishbhakuni left a comment

github-actions bot commented Apr 5, 2024

github-actions bot commented Apr 5, 2024

gbbafna commented Apr 5, 2024

github-actions bot commented Apr 5, 2024

ashking94 commented Apr 5, 2024

Update Shallow Snapshot flows to support remote path type & hash algo #12988

Update Shallow Snapshot flows to support remote path type & hash algo #12988

Conversation

ashking94 commented Mar 31, 2024 • edited Loading

Description

Related Issues

Check List

github-actions bot commented Mar 31, 2024 • edited Loading

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Mar 31, 2024

codecov bot commented Mar 31, 2024 • edited Loading

Codecov Report

github-actions bot commented Apr 3, 2024

ashking94 commented Apr 3, 2024

github-actions bot commented Apr 3, 2024

github-actions bot commented Apr 4, 2024

ashking94 commented Apr 4, 2024

github-actions bot commented Apr 4, 2024

harishbhakuni left a comment

Choose a reason for hiding this comment

github-actions bot commented Apr 5, 2024

github-actions bot commented Apr 5, 2024

gbbafna commented Apr 5, 2024

github-actions bot commented Apr 5, 2024

ashking94 commented Apr 5, 2024

ashking94 commented Mar 31, 2024 •

edited

Loading

github-actions bot commented Mar 31, 2024 •

edited

Loading

codecov bot commented Mar 31, 2024 •

edited

Loading