Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add segment replication + remote store GA information #5037

Merged
merged 11 commits into from
Sep 19, 2023
Merged

Add segment replication + remote store GA information #5037

merged 11 commits into from
Sep 19, 2023
+21 −15

Conversation

Naarcha-AWS
Copy link
Collaborator

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@Naarcha-AWS
Add segment replication + remote store GA information
96dc030 
Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
@Naarcha-AWS
Add introduction labels
779369c 
Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
@Naarcha-AWS
Copy link
Collaborator Author

@dreamer-89: This PR is ready for your review. Let me know if there is anything I missed.

@hdhalter hdhalter added v2.10.0 release-notes PR: Include this PR in the automated release notes 3 - Tech review PR: Tech review in progress labels Sep 15, 2023
@Naarcha-AWS
Apply suggestions from code review
591ba8f 
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
@Naarcha-AWS
Apply suggestions from code review
b34a484 
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
@Naarcha-AWS @dreamer-89
Update _tuning-your-cluster/availability-and-recovery/segment-replica…
9b668fb 
…tion/index.md

Co-authored-by: Suraj Singh <surajrider@gmail.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
@Naarcha-AWS @Bukhtawar
Apply suggestions from code review
c5291f7 
Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
@Naarcha-AWS
Apply suggestions from code review
92133af 
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
@Naarcha-AWS
Update index.md
a2e3323 
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
kolchfa-aws
kolchfa-aws reviewed Sep 19, 2023
View reviewed changes
Copy link
Collaborator

@kolchfa-aws kolchfa-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good but we need to reference a consistent name (either remote-backed storage or remote store).

_tuning-your-cluster/availability-and-recovery/remote-store/remote-segment-backpressure.md Show resolved Hide resolved
_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved
_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved
_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved
_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved
_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved
@Naarcha-AWS @kolchfa-aws
Apply suggestions from code review
0fc62f0 
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
@Naarcha-AWS Naarcha-AWS added 4 - Doc review PR: Doc review in progress and removed 3 - Tech review PR: Tech review in progress labels Sep 19, 2023
@Naarcha-AWS
Replace Remote Store with Remote Backed Storage
f4c8df8 
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
kolchfa-aws
kolchfa-aws approved these changes Sep 19, 2023
View reviewed changes
Copy link
Collaborator

@kolchfa-aws kolchfa-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Naarcha-AWS Naarcha-AWS added 5 - Editorial review PR: Editorial review in progress and removed 4 - Doc review PR: Doc review in progress labels Sep 19, 2023
natebower
natebower reviewed Sep 19, 2023
View reviewed changes
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Naarcha-AWS Please see my comments and changes and let me know if you have any questions. Thanks!

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved
_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved
_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated
As of OpenSearch 2.10, you can use two methods for segment replication:

- With **remote-backed storage**, a persistent storage solution: The primary shard sends segment files to the remote-backed storage and the replica shards source the copy from the same store. For more information about using a remote-backed storage, see [Remote-backed storage]({{site.url}}{{site.baseurl}}/tuning-your-cluster/availability-and-recovery/remote-store/index/#segment-replication-and-remote-backed-storage).
- Without remote-backed storage: The primary shard sends segment files directly to the replica shards using node-to-node communication.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we call the second method what it is, as opposed to what it isn't? Node-to-node replication?

_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved
_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved
_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved
_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved
_tuning-your-cluster/availability-and-recovery/segment-replication/index.md Outdated Show resolved Hide resolved
@Naarcha-AWS @natebower
Apply suggestions from code review
1b215d2 
Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
@Naarcha-AWS Naarcha-AWS merged commit 64e120a into main Sep 19, 2023
vagimeli pushed a commit that referenced this pull request Sep 19, 2023
@Naarcha-AWS @dreamer-89
@Bukhtawar @kolchfa-aws @natebower @vagimeli
Add segment replication + remote store GA information (#5037)
8d6bd01 
* Add segment replication + remote store GA information

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Add introduction labels

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _tuning-your-cluster/availability-and-recovery/segment-replication/index.md

Co-authored-by: Suraj Singh <surajrider@gmail.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update index.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Replace Remote Store with Remote Backed Storage

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Suraj Singh <surajrider@gmail.com>
Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
vagimeli added a commit that referenced this pull request Sep 19, 2023
@vagimeli
Revert "Add segment replication + remote store GA information (#5037)"
c71fb7a 
This reverts commit 8d6bd01.
@dreamer-89 dreamer-89 mentioned this pull request Sep 25, 2023
Closed
4 tasks
harshavamsi pushed a commit to harshavamsi/documentation-website that referenced this pull request Oct 31, 2023
@Naarcha-AWS @dreamer-89
@Bukhtawar @kolchfa-aws @natebower @harshavamsi
Add segment replication + remote store GA information (opensearch-pro…
55b27a3 
…ject#5037)

* Add segment replication + remote store GA information

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Add introduction labels

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _tuning-your-cluster/availability-and-recovery/segment-replication/index.md

Co-authored-by: Suraj Singh <surajrider@gmail.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update index.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Replace Remote Store with Remote Backed Storage

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Suraj Singh <surajrider@gmail.com>
Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
vagimeli pushed a commit that referenced this pull request Dec 21, 2023
@Naarcha-AWS @dreamer-89
@Bukhtawar @kolchfa-aws @natebower @vagimeli
Add segment replication + remote store GA information (#5037)
e0c021c 
* Add segment replication + remote store GA information

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Add introduction labels

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update _tuning-your-cluster/availability-and-recovery/segment-replication/index.md

Co-authored-by: Suraj Singh <surajrider@gmail.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Update index.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Replace Remote Store with Remote Backed Storage

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Nathan Bower <nbower@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Naarcha-AWS <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Co-authored-by: Suraj Singh <surajrider@gmail.com>
Co-authored-by: Bukhtawar Khan <bukhtawar7152@gmail.com>
Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>
@Naarcha-AWS Naarcha-AWS deleted the segment-replication-ga branch March 28, 2024 23:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@natebower natebower natebower left review comments

@kolchfa-aws kolchfa-aws kolchfa-aws approved these changes

@cwillum cwillum Awaiting requested review from cwillum

@hdhalter hdhalter Awaiting requested review from hdhalter

@vagimeli vagimeli Awaiting requested review from vagimeli

@ananzh ananzh Awaiting requested review from ananzh

@seanneumann seanneumann Awaiting requested review from seanneumann

@AMoo-Miki AMoo-Miki Awaiting requested review from AMoo-Miki AMoo-Miki is a code owner

@Bukhtawar Bukhtawar Awaiting requested review from Bukhtawar

@dreamer-89 dreamer-89 Awaiting requested review from dreamer-89

@anasalkouz anasalkouz Awaiting requested review from anasalkouz

Assignees

@Naarcha-AWS Naarcha-AWS

Labels
5 - Editorial review PR: Editorial review in progress release-notes PR: Include this PR in the automated release notes v2.10.0
Projects
None yet
Milestone
No milestone
7 participants
@Naarcha-AWS @Bukhtawar @anasalkouz @dreamer-89 @natebower @kolchfa-aws @hdhalter