Refactor Translog metadata upload/download to write/read header and footer via VersionedCodecStreamWrapper #7953

BhumikaSaini-Amazon · 2023-06-07T15:35:12Z

Description

Refactor Translog metadata upload/download to write/read header and footer via VersionedCodecStreamWrapper introduced in #5917

Related Issues

Resolves #7228

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
~~Commit changes are listed out in CHANGELOG.md file (See: Changelog)~~

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2023-06-07T16:05:13Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/17043/
CommitID: fadea6f
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-06-08T06:22:09Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/17104/
CommitID: 9850c65
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/index/translog/transfer/TranslogTransferManager.java

linuxpi · 2023-06-08T06:56:36Z

Changes look good @BhumikaSaini-Amazon. Please check why the tests are failing

ashking94 · 2023-06-08T10:12:56Z

@BhumikaSaini-Amazon Lets update the PR title with an imperative style message.

github-actions · 2023-06-08T10:18:15Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/17112/
CommitID: d34ef74
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/index/translog/transfer/TranslogTransferMetadata.java

gbbafna

LGTM. 1 minor comment

github-actions · 2023-06-12T10:40:36Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/17302/
CommitID: e7eed22
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-06-12T11:18:25Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/17305/
CommitID: 245a98a
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

github-actions · 2023-06-12T12:07:00Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/17306/
CommitID: 519785e
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

BhumikaSaini-Amazon · 2023-06-12T12:10:30Z

Failing tests in build:

org.opensearch.cluster.shards.ClusterShardLimitIT.testCreateIndexWithMaxClusterShardSetting

Flaky test #6287

Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

github-actions · 2023-06-12T12:42:11Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.search.backpressure.SearchBackpressureIT.testSearchTaskCancellationWithHighCpu
      1 org.opensearch.cluster.allocation.AwarenessAllocationIT.testThreeZoneOneReplicaWithForceZoneValueAndLoadAwareness

URL: https://build.ci.opensearch.org/job/gradle-check/17308/
CommitID: 1c3c959
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

github-actions · 2023-06-12T14:59:06Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/17326/
CommitID: 42d8bda
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-06-13T05:16:20Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.remotestore.RemoteStoreRefreshListenerIT.testRemoteRefreshRetryOnFailure

URL: https://build.ci.opensearch.org/job/gradle-check/17416/
CommitID: 42d8bda
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

…ooter via VersionedCodecStreamWrapper (#7953) Signed-off-by: Bhumika Saini <sabhumik@amazon.com> (cherry picked from commit 4b4d84e) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…ooter via VersionedCodecStreamWrapper (#7953) (#8036) (cherry picked from commit 4b4d84e) Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

…ooter via VersionedCodecStreamWrapper (opensearch-project#7953) Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

…ooter via VersionedCodecStreamWrapper (opensearch-project#7953) (opensearch-project#8036) (cherry picked from commit 4b4d84e) Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

…ooter via VersionedCodecStreamWrapper (opensearch-project#7953) Signed-off-by: Bhumika Saini <sabhumik@amazon.com> Signed-off-by: Rishab Nahata <rnnahata@amazon.com>

…ooter via VersionedCodecStreamWrapper (opensearch-project#7953) Signed-off-by: Bhumika Saini <sabhumik@amazon.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com>

BhumikaSaini-Amazon marked this pull request as ready for review June 8, 2023 06:11

BhumikaSaini-Amazon requested review from reta, anasalkouz, andrross, Bukhtawar, CEHENKLE, dblock, gbbafna, setiah, kartg, kotwanikunal, mch2, nknize, owaiskazi19, Rishikesh1159, ryanbogan, saratvemulapalli, shwetathareja, dreamer-89, tlfeng, VachaShah and dbwiddis as code owners June 8, 2023 06:11

ashking94 added the skip-changelog label Jun 8, 2023

linuxpi reviewed Jun 8, 2023

View reviewed changes

server/src/main/java/org/opensearch/index/translog/transfer/TranslogTransferManager.java Show resolved Hide resolved

BhumikaSaini-Amazon changed the title ~~Resolve #7228~~ Refactor Translog metadata upload/download to write/read header and footer via VersionedCodecStreamWrapper Jun 8, 2023

gbbafna reviewed Jun 12, 2023

View reviewed changes

server/src/main/java/org/opensearch/index/translog/transfer/TranslogTransferMetadata.java Outdated Show resolved Hide resolved

gbbafna reviewed Jun 12, 2023

View reviewed changes

Bhumika Saini added 8 commits June 12, 2023 17:02

Resolve opensearch-project#7228

1c7eb83

Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

Move VersionedCodecStreamWrapper to TranslogTransferManager

17ee559

Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

Fix failing tests

664af67

Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

Pass TranslogTransferMetadataHandlerTests#testWriteContent

86f6279

Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

Empty commit

24ee5ca

Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

Revert obsolete TranslogTransferMetadata attribute updates

60c36e1

Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

Make BUFFER_SIZE, CURRENT_VERSION, METADATA_CODEC package-private

bcbebaa

Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

Empty commit

519785e

Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

BhumikaSaini-Amazon force-pushed the refactor-md-ul-dl branch from 245a98a to 519785e Compare June 12, 2023 11:35

Empty commit

1c3c959

Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

gbbafna approved these changes Jun 12, 2023

View reviewed changes

Fix javadoc

42d8bda

Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

gbbafna merged commit 4b4d84e into opensearch-project:main Jun 13, 2023

gbbafna added the backport 2.x Backport to 2.x branch label Jun 13, 2023

opensearch-trigger-bot bot mentioned this pull request Jun 13, 2023

[Backport 2.x] Refactor Translog metadata upload/download to write/read header and footer via VersionedCodecStreamWrapper #8036

Merged

gaiksaya pushed a commit to gaiksaya/OpenSearch that referenced this pull request Jun 15, 2023

Refactor Translog metadata upload/download to write/read header and f…

236e617

…ooter via VersionedCodecStreamWrapper (opensearch-project#7953) Signed-off-by: Bhumika Saini <sabhumik@amazon.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor Translog metadata upload/download to write/read header and footer via VersionedCodecStreamWrapper #7953

Refactor Translog metadata upload/download to write/read header and footer via VersionedCodecStreamWrapper #7953

BhumikaSaini-Amazon commented Jun 7, 2023 •

edited

Loading

github-actions bot commented Jun 7, 2023

github-actions bot commented Jun 8, 2023

linuxpi commented Jun 8, 2023

ashking94 commented Jun 8, 2023

github-actions bot commented Jun 8, 2023

gbbafna left a comment

github-actions bot commented Jun 12, 2023

github-actions bot commented Jun 12, 2023

github-actions bot commented Jun 12, 2023

BhumikaSaini-Amazon commented Jun 12, 2023

github-actions bot commented Jun 12, 2023

github-actions bot commented Jun 12, 2023

github-actions bot commented Jun 13, 2023

Refactor Translog metadata upload/download to write/read header and footer via VersionedCodecStreamWrapper #7953

Refactor Translog metadata upload/download to write/read header and footer via VersionedCodecStreamWrapper #7953

Conversation

BhumikaSaini-Amazon commented Jun 7, 2023 • edited Loading

Description

Related Issues

Check List

github-actions bot commented Jun 7, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jun 8, 2023

Gradle Check (Jenkins) Run Completed with:

linuxpi commented Jun 8, 2023

ashking94 commented Jun 8, 2023

github-actions bot commented Jun 8, 2023

Gradle Check (Jenkins) Run Completed with:

gbbafna left a comment

Choose a reason for hiding this comment

github-actions bot commented Jun 12, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jun 12, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jun 12, 2023

Gradle Check (Jenkins) Run Completed with:

BhumikaSaini-Amazon commented Jun 12, 2023

github-actions bot commented Jun 12, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jun 12, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jun 13, 2023

Gradle Check (Jenkins) Run Completed with:

BhumikaSaini-Amazon commented Jun 7, 2023 •

edited

Loading