Introduce remote store path type in customData in IndexMetadata

[ashking94]

Description

In this PR, we introduce remote store path information which we will store in the IndexMatadata class and specifically customData field. This information will be useful in determining the path for a shard. User also gets the option to select the optimised approach versus the existing approach. We also get the extensibility option to further optimise it and change it for better resiliency, performance and throughput.

With this PR, we achieve the following things -

1. Ability to create an index with information of the blob store path type in custom data in index metadata.
2. When snapshots are taken, we store the index metadata as well for the index. During the snapshot restore, the same is restored. We are able to keep the information of the blob store path type in custom data in index metadata.

However, we need to handle couple of more things which I am adding to the meta issue :

1. During snapshot restore of indexes not having remote custom data, we use the cluster default blob store path type.
2. During migration of indexes from document to remote store, we need to use the cluster default blob store path type.

Related Issues

Resolves #12608

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)
• Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 5e915e1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

Compatibility status:

Checks if related components are compatible with change fab0366

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/performance-analyzer.git]

[ashking94]

Created opensearch-project/documentation-website#6675 for documentation

[github-actions]

❕ Gradle check result for d256b66: UNSTABLE

• TEST FAILURES:

      1 org.opensearch.search.SearchWeightedRoutingIT.testMultiGetWithNetworkDisruption_FailOpenEnabled
      1 org.opensearch.cluster.coordination.AwarenessAttributeDecommissionIT.testConcurrentDecommissionAction

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[ashking94]

❌ Gradle check result for 48edb4e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Flaky test - #12338

[github-actions]

❌ Gradle check result for 25e06e9: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[ashking94]

❌ Gradle check result for 25e06e9: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Flaky test - #8928

[github-actions]

❌ Gradle check result for 25e06e9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[ashking94]

❌ Gradle check result for 25e06e9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Flaky test - #12338

[github-actions]

❌ Gradle check result for 25e06e9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[ashking94]

❌ Gradle check result for 25e06e9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Flaky test - #12717. The fix has been merged an hour ago. Rebased from main branch.

[github-actions]

❕ Gradle check result for fab0366: UNSTABLE

• TEST FAILURES:

      2 org.opensearch.index.shard.RemoteStoreRefreshListenerTests.classMethod
      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testIndexReopenClose
      1 org.opensearch.index.shard.RemoteStoreRefreshListenerTests.testRefreshSuccessAfterFailureInFirstAttemptAfterSnapshotAndMetadataUpload

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[ashking94]

❕ Gradle check result for fab0366: UNSTABLE

• TEST FAILURES:

      2 org.opensearch.index.shard.RemoteStoreRefreshListenerTests.classMethod
      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testIndexReopenClose
      1 org.opensearch.index.shard.RemoteStoreRefreshListenerTests.testRefreshSuccessAfterFailureInFirstAttemptAfterSnapshotAndMetadataUpload

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Flaky test - #10987 and #8131.

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-12607-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 91582e7c25a6f76eeebfa3ab74287178e1986491
# Push it to GitHub
git push --set-upstream origin backport/backport-12607-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-12607-to-2.x.

[ashking94]

Auto backport failed. Raising manual backport.