Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating the separator for RemoteStoreLockManager since underscore is allowed in base64UUID url charset #10379

Merged
merged 6 commits into from
Oct 5, 2023

Conversation

harishbhakuni
Copy link
Contributor

@harishbhakuni harishbhakuni commented Oct 5, 2023

Description

We are using URL Safe Base64UUID in most of the places in code which can have '__' as one of the characters. For Remote Store Metadata Lock files, Keeping '___' (triple underscores) as separator to separate out snapshot UUID and remote store metadata name would not work because if either the Metadata or the Snapshot UUID starts or ends will '', we cannot use our separator to separate out the fields.

To Fix this, as part of this PR I am updating the separator to "..." for new lock files. We will still support reading older lock files for backward compatibility, but we will do best effort by pulling out the snapshotID from the end first and then getting the md file.

Related Issues

Resolves #9115

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • GitHub issue/PR created in OpenSearch documentation repo for the required public documentation changes (#[Issue/PR number])

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Contributor

github-actions bot commented Oct 5, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 5, 2023

Compatibility status:

Checks if related components are compatible with change b8adac0

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git]

Harish Bhakuni added 3 commits October 5, 2023 11:05
Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>
Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>
Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>
@github-actions
Copy link
Contributor

github-actions bot commented Oct 5, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 5, 2023

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.snapshots.DedicatedClusterSnapshotRestoreIT.testIndexDeletionDuringSnapshotCreationInQueue
      1 org.opensearch.index.reindex.UpdateByQueryBasicTests.classMethod

@sohami sohami added the v2.11.0 Issues and PRs related to version 2.11.0 label Oct 5, 2023
CHANGELOG.md Outdated Show resolved Hide resolved
@harishbhakuni harishbhakuni changed the title Refactor Remote Store Metadata Lock Manager Utils Updating the separator for RemoteLockManager since underscore is allowed in base64UUID url charset Oct 5, 2023
Harish Bhakuni added 2 commits October 5, 2023 12:31
Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>
Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>
@harishbhakuni harishbhakuni changed the title Updating the separator for RemoteLockManager since underscore is allowed in base64UUID url charset Updating the separator for RemoteStoreLockManager since underscore is allowed in base64UUID url charset Oct 5, 2023
@github-actions
Copy link
Contributor

github-actions bot commented Oct 5, 2023

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

github-actions bot commented Oct 5, 2023

Gradle Check (Jenkins) Run Completed with:

@sohami
Copy link
Collaborator

sohami commented Oct 5, 2023

Merging as gradle check passed and pre-commit was successful locally as well.

@sohami sohami merged commit 66aef13 into opensearch-project:main Oct 5, 2023
12 of 15 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 5, 2023
… allowed in base64UUID url charset (#10379)

* Refactor Remote Store Metadata Lock Manager Utils

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Address PR Comments

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Address PR Comments

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Update Changelog entry

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Update Changelog entry

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Unmute testDeleteShallowCopySnapshot test

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

---------

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>
Co-authored-by: Harish Bhakuni <hbhakuni@amazon.com>
(cherry picked from commit 66aef13)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@harishbhakuni harishbhakuni added the backport 2.x Backport to 2.x branch label Oct 5, 2023
sohami pushed a commit that referenced this pull request Oct 5, 2023
… allowed in base64UUID url charset (#10379) (#10421)

* Refactor Remote Store Metadata Lock Manager Utils



* Address PR Comments



* Address PR Comments



* Update Changelog entry



* Update Changelog entry



* Unmute testDeleteShallowCopySnapshot test



---------



(cherry picked from commit 66aef13)

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Harish Bhakuni <hbhakuni@amazon.com>
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 5, 2023
… allowed in base64UUID url charset (#10379)

* Refactor Remote Store Metadata Lock Manager Utils

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Address PR Comments

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Address PR Comments

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Update Changelog entry

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Update Changelog entry

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Unmute testDeleteShallowCopySnapshot test

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

---------

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>
Co-authored-by: Harish Bhakuni <hbhakuni@amazon.com>
(cherry picked from commit 66aef13)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
sohami pushed a commit that referenced this pull request Oct 5, 2023
… allowed in base64UUID url charset (#10379) (#10428)

* Refactor Remote Store Metadata Lock Manager Utils



* Address PR Comments



* Address PR Comments



* Update Changelog entry



* Update Changelog entry



* Unmute testDeleteShallowCopySnapshot test



---------



(cherry picked from commit 66aef13)

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Harish Bhakuni <hbhakuni@amazon.com>
kaushalmahi12 pushed a commit to kaushalmahi12/OpenSearch that referenced this pull request Oct 6, 2023
… allowed in base64UUID url charset (opensearch-project#10379)

* Refactor Remote Store Metadata Lock Manager Utils

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Address PR Comments

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Address PR Comments

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Update Changelog entry

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Update Changelog entry

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Unmute testDeleteShallowCopySnapshot test

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

---------

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>
Co-authored-by: Harish Bhakuni <hbhakuni@amazon.com>
deshsidd pushed a commit to deshsidd/OpenSearch that referenced this pull request Oct 9, 2023
… allowed in base64UUID url charset (opensearch-project#10379)

* Refactor Remote Store Metadata Lock Manager Utils

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Address PR Comments

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Address PR Comments

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Update Changelog entry

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Update Changelog entry

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Unmute testDeleteShallowCopySnapshot test

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

---------

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>
Co-authored-by: Harish Bhakuni <hbhakuni@amazon.com>
austintlee pushed a commit to austintlee/OpenSearch that referenced this pull request Oct 23, 2023
… allowed in base64UUID url charset (opensearch-project#10379)

* Refactor Remote Store Metadata Lock Manager Utils

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Address PR Comments

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Address PR Comments

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Update Changelog entry

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Update Changelog entry

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Unmute testDeleteShallowCopySnapshot test

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

---------

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>
Co-authored-by: Harish Bhakuni <hbhakuni@amazon.com>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
… allowed in base64UUID url charset (opensearch-project#10379)

* Refactor Remote Store Metadata Lock Manager Utils

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Address PR Comments

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Address PR Comments

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Update Changelog entry

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Update Changelog entry

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

* Unmute testDeleteShallowCopySnapshot test

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>

---------

Signed-off-by: Harish Bhakuni <hbhakuni@amazon.com>
Co-authored-by: Harish Bhakuni <hbhakuni@amazon.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch backport 2.11 bug Something isn't working flaky-test Random test failure that succeeds on second run v2.11.0 Issues and PRs related to version 2.11.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] org.opensearch.snapshots.DeleteSnapshotIT.testDeleteShallowCopySnapshot is flaky
3 participants