Fix flaky pit/scroll tests in SegmentReplicationIT #10770

mch2 · 2023-10-20T03:32:40Z

Description

This change adds an assertBusy to wait until files are cleared from disk as it is not synchronous with the scroll/pit removal.

Related Issues

Resolves #10769

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)
Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2023-10-20T04:06:18Z

Compatibility status:

Checks if related components are compatible with change 5e32d8b

Incompatible components

Incompatible components: [https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/neural-search.git]

github-actions · 2023-10-20T04:13:41Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28520/
CommitID: 6925e19
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-10-20T05:05:48Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28525/
CommitID: 6925e19
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-10-20T16:32:38Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/28601/
CommitID: 6925e19
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

dreamer-89

Thanks for fixing this.

dreamer-89 · 2023-11-03T00:20:57Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌

URL: https://build.ci.opensearch.org/job/gradle-check/28601/

CommitID: 6925e19
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

#7934
#11079

github-actions · 2023-11-03T01:09:13Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/29507/
CommitID: 6925e19
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

dreamer-89 · 2023-11-03T01:56:58Z

@mch2 : There are bunch of flaky tests failing in last few gradle checks. Can you rebase your branch with latest main?

This change adds an assertBusy to wait until files are cleared from disk as it is not synchronous with the scroll/pit removal. Signed-off-by: Marc Handalian <handalm@amazon.com>

github-actions · 2023-11-05T22:45:21Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/29556/
CommitID: 5e32d8b
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-11-05T23:58:41Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/29557/
CommitID: 5e32d8b
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-11-06T01:37:10Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.repositories.azure.AzureBlobContainerRetriesTests.testWriteLargeBlob

URL: https://build.ci.opensearch.org/job/gradle-check/29558/
CommitID: 5e32d8b
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

codecov · 2023-11-06T01:38:30Z

Codecov Report

Merging #10770 (5e32d8b) into main (747f7d1) will increase coverage by 0.04%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main   #10770      +/-   ##
============================================
+ Coverage     71.20%   71.25%   +0.04%     
- Complexity    58716    58774      +58     
============================================
  Files          4872     4872              
  Lines        276773   276773              
  Branches      40240    40240              
============================================
+ Hits         197078   197212     +134     
+ Misses        63238    63137     -101     
+ Partials      16457    16424      -33

see 450 files with indirect coverage changes

mch2 · 2023-11-06T04:46:53Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌

URL: https://build.ci.opensearch.org/job/gradle-check/29556/

CommitID: 5e32d8b
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

#10006

mch2 · 2023-11-06T04:47:39Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌

URL: https://build.ci.opensearch.org/job/gradle-check/29557/

CommitID: 5e32d8b
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

#11074

This change adds an assertBusy to wait until files are cleared from disk as it is not synchronous with the scroll/pit removal. Signed-off-by: Marc Handalian <handalm@amazon.com> (cherry picked from commit c1962cc) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

This change adds an assertBusy to wait until files are cleared from disk as it is not synchronous with the scroll/pit removal. (cherry picked from commit c1962cc) Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…t#10770) This change adds an assertBusy to wait until files are cleared from disk as it is not synchronous with the scroll/pit removal. Signed-off-by: Marc Handalian <handalm@amazon.com>

…t#10770) This change adds an assertBusy to wait until files are cleared from disk as it is not synchronous with the scroll/pit removal. Signed-off-by: Marc Handalian <handalm@amazon.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com>

mch2 added the skip-changelog label Oct 20, 2023

github-actions bot added the bug Something isn't working label Oct 20, 2023

dreamer-89 approved these changes Nov 3, 2023

View reviewed changes

dreamer-89 added the backport 2.x Backport to 2.x branch label Nov 3, 2023

Fix flaky pit/scroll tests in SegmentReplicationIT

5e32d8b

This change adds an assertBusy to wait until files are cleared from disk as it is not synchronous with the scroll/pit removal. Signed-off-by: Marc Handalian <handalm@amazon.com>

mch2 force-pushed the pitscrollflaky branch from 6925e19 to 5e32d8b Compare November 5, 2023 22:03

mch2 requested a review from adnapibar as a code owner November 5, 2023 22:03

mch2 merged commit c1962cc into opensearch-project:main Nov 6, 2023
46 of 53 checks passed

mch2 deleted the pitscrollflaky branch November 6, 2023 04:47

opensearch-trigger-bot bot mentioned this pull request Nov 6, 2023

[Backport 2.x] Fix flaky pit/scroll tests in SegmentReplicationIT #11098

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix flaky pit/scroll tests in SegmentReplicationIT #10770

Fix flaky pit/scroll tests in SegmentReplicationIT #10770

mch2 commented Oct 20, 2023 •

edited

Loading

github-actions bot commented Oct 20, 2023 •

edited

Loading

github-actions bot commented Oct 20, 2023

github-actions bot commented Oct 20, 2023

github-actions bot commented Oct 20, 2023

dreamer-89 left a comment

dreamer-89 commented Nov 3, 2023 •

edited

Loading

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Nov 3, 2023

dreamer-89 commented Nov 3, 2023

github-actions bot commented Nov 5, 2023

github-actions bot commented Nov 5, 2023

github-actions bot commented Nov 6, 2023

codecov bot commented Nov 6, 2023

mch2 commented Nov 6, 2023

Gradle Check (Jenkins) Run Completed with:

mch2 commented Nov 6, 2023

Gradle Check (Jenkins) Run Completed with:

Fix flaky pit/scroll tests in SegmentReplicationIT #10770

Fix flaky pit/scroll tests in SegmentReplicationIT #10770

Conversation

mch2 commented Oct 20, 2023 • edited Loading

Description

Related Issues

Check List

github-actions bot commented Oct 20, 2023 • edited Loading

Compatibility status:

Incompatible components

Skipped components

Compatible components

github-actions bot commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 20, 2023

Gradle Check (Jenkins) Run Completed with:

dreamer-89 left a comment

Choose a reason for hiding this comment

dreamer-89 commented Nov 3, 2023 • edited Loading

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Nov 3, 2023

Gradle Check (Jenkins) Run Completed with:

dreamer-89 commented Nov 3, 2023

github-actions bot commented Nov 5, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Nov 5, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Nov 6, 2023

Gradle Check (Jenkins) Run Completed with:

codecov bot commented Nov 6, 2023

Codecov Report

mch2 commented Nov 6, 2023

Gradle Check (Jenkins) Run Completed with:

mch2 commented Nov 6, 2023

Gradle Check (Jenkins) Run Completed with:

mch2 commented Oct 20, 2023 •

edited

Loading

github-actions bot commented Oct 20, 2023 •

edited

Loading

dreamer-89 commented Nov 3, 2023 •

edited

Loading