[BUG] org.opensearch.remotestore.RemoteStoreRefreshListenerIT.testRemoteRefreshRetryOnFailure is flaky #7703

reta · 2023-05-23T18:27:50Z

Describe the bug
org.opensearch.remotestore.RemoteStoreRefreshListenerIT.testRemoteRefreshRetryOnFailure is flaky

java.lang.AssertionError
	at org.junit.Assert.fail(Assert.java:87)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at org.junit.Assert.assertTrue(Assert.java:53)
	at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
	at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1084)
	at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.testRemoteRefreshRetryOnFailure(RemoteStoreRefreshListenerIT.java:47)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:1589)
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more
	Suppressed: java.lang.AssertionError
		at org.junit.Assert.fail(Assert.java:87)
		at org.junit.Assert.assertTrue(Assert.java:42)
		at org.junit.Assert.assertTrue(Assert.java:53)
		at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
		at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1072)
		... 39 more

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
The test should pass reliably

Plugins
Standard

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

CI

Additional context
https://build.ci.opensearch.org/job/gradle-check/16080/testReport/junit/org.opensearch.remotestore/RemoteStoreRefreshListenerIT/testRemoteRefreshRetryOnFailure/

The text was updated successfully, but these errors were encountered:

Signed-off-by: Ashish Singh <ssashish@amazon.com>

ashking94 · 2023-06-02T06:42:46Z

There are other exception stack traces as well that I can see -

  2> REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.remotestore.RemoteStoreRefreshListenerIT.testRemoteRefreshRetryOnFailure {seed=[B4035D2FFC2EAD3C:6BBEFB8675A3EC01]}" -Dtests.seed=B4035D2FFC2EAD3C -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=es-VE -Dtests.timezone=Asia/Kuwait -Druntime.java=19
  2> REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.remotestore.RemoteStoreRefreshListenerIT.testRemoteRefreshRetryOnFailure {seed=[B4035D2FFC2EAD3C:6BBEFB8675A3EC01]}" -Dtests.seed=B4035D2FFC2EAD3C -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=es-VE -Dtests.timezone=Asia/Kuwait -Druntime.java=19
  2> java.lang.AssertionError
        at org.junit.Assert.fail(Assert.java:87)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.junit.Assert.assertTrue(Assert.java:53)
        at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.lambda$testRemoteRefreshRetryOnFailure$0(RemoteStoreRefreshListenerIT.java:50)
        at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1084)
        at org.opensearch.remotestore.RemoteStoreRefreshListenerIT.testRemoteRefreshRetryOnFailure(RemoteStoreRefreshListenerIT.java:47)

    java.lang.AssertionError: java.lang.AssertionError: Could not find index.latest blob for repo [BlobStoreRepository[[my-segment-repo-1], [org.opensearch.snapshots.mockstore.MockRepository$MockBlobStore@1e3751bf]]]
        at org.opensearch.repositories.blobstore.BlobStoreTestUtil.assertConsistency(BlobStoreTestUtil.java:154)
        at org.opensearch.repositories.blobstore.BlobStoreTestUtil.assertRepoConsistency(BlobStoreTestUtil.java:113)
        at org.opensearch.snapshots.AbstractSnapshotIntegTestCase.lambda$assertRepoConsistency$1(AbstractSnapshotIntegTestCase.java:157)
        at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
        at org.opensearch.snapshots.AbstractSnapshotIntegTestCase.assertRepoConsistency(AbstractSnapshotIntegTestCase.java:151)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
        at java.base/java.lang.reflect.Method.invoke(Method.java:578)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
        at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
        at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
        at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
        at org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
        at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
        at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
        at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
        at org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:1589)

        Caused by:
        java.lang.AssertionError: Could not find index.latest blob for repo [BlobStoreRepository[[my-segment-repo-1], [org.opensearch.snapshots.mockstore.MockRepository$MockBlobStore@1e3751bf]]]
            at org.opensearch.repositories.blobstore.BlobStoreTestUtil.lambda$assertConsistency$0(BlobStoreTestUtil.java:133)
            at org.opensearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:73)
            at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:88)
            at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:806)
            at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
            at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
            at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
            ... 1 more

Signed-off-by: Ashish Singh <ssashish@amazon.com>

Signed-off-by: Ashish Singh <ssashish@amazon.com> (cherry picked from commit 5d5e8ad) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

(cherry picked from commit 5d5e8ad) Signed-off-by: Ashish Singh <ssashish@amazon.com>

dblock · 2023-06-08T17:27:14Z

Still flaky in #7969 (comment)

…earch-project#7879) (opensearch-project#7881) (cherry picked from commit 5d5e8ad) Signed-off-by: Ashish Singh <ssashish@amazon.com>

ashking94 · 2023-07-07T05:39:38Z

I will be prioritising this issue, will try to get this closed before 2.9 release.

kotwanikunal · 2023-07-07T17:25:21Z

@ashking94 I think the only issue is that the IOException from MockRepository marks the segment file as failed and it is not uploaded until another refresh kicks in.

I tried adding a force refresh operation here and the test seemed to succeed for 500 times without any issues.

ashking94 · 2023-07-08T03:21:39Z

@kotwanikunal Thanks for the investigation. The tests are failing due to segments have not been uploaded to remote store completely yet. In this case, we are intentionally failing the uploads and have retry mechanism which will ultimately get the remote store in sync with local segments. Doing a force refresh is not recommended as it triggers refresh internally and hence it would make the upload go through ultimately. For now, I am muting the test. I should be able to get the fix real soon.

…earch-project#7879) Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com>

reta added bug Something isn't working untriaged flaky-test Random test failure that succeeds on second run labels May 23, 2023

This was referenced May 23, 2023

Bump com.google.api:gax from 2.17.0 to 2.27.0 in /plugins/repository-gcs #7697

Merged

[Meta] Fix random test failures #1715

Closed

sachinpkale added Storage:Durability Issues and PRs related to the durability framework v2.8.0 'Issues and PRs related to version v2.8.0' and removed untriaged labels May 24, 2023

gbbafna assigned ashking94 May 24, 2023

raghuvanshraj mentioned this issue May 25, 2023

[Remote Store] Support to emit multiple streams for a file content each responsible for processing a specific part of the file #7000

Merged

6 tasks

ashking94 added a commit to ashking94/OpenSearch that referenced this issue Jun 2, 2023

Fix flaky test - closes opensearch-project#7703

6ccff27

Signed-off-by: Ashish Singh <ssashish@amazon.com>

ashking94 mentioned this issue Jun 2, 2023

[Remote Store] Fix flaky test - closes #7703 #7879

Merged

6 tasks

gbbafna closed this as completed in #7879 Jun 2, 2023

gbbafna pushed a commit that referenced this issue Jun 2, 2023

[Remote Store] Fix flaky test - closes #7703 (#7879)

5d5e8ad

Signed-off-by: Ashish Singh <ssashish@amazon.com>

gbbafna pushed a commit that referenced this issue Jun 2, 2023

[Remote Store] Fix flaky test - closes #7703 (#7879) (#7881)

35ba672

(cherry picked from commit 5d5e8ad) Signed-off-by: Ashish Singh <ssashish@amazon.com>

BhumikaSaini-Amazon mentioned this issue Jun 8, 2023

Refactor Translog metadata upload/download to write/read header and footer via VersionedCodecStreamWrapper #7953

Merged

6 tasks

dblock reopened this Jun 8, 2023

github-actions bot added the untriaged label Jun 8, 2023

BhumikaSaini-Amazon mentioned this issue Jun 16, 2023

[Remote Segment Store] Add Lucene major version to UploadedSegmentMetadata #8088

Merged

6 tasks

This was referenced Jun 16, 2023

Manually backport #8107 bump guava to 32.0.1-jre #8114

Merged

[AUTOCUT] Gradle Check Failure on push to 2.x #8126

Closed

reta mentioned this issue Jun 21, 2023

[AUTOCUT] Gradle Check Failure on push to 2.x #8194

Closed

kotwanikunal mentioned this issue Jun 26, 2023

[Backport] [2.x] Bump org.apache.hadoop:hadoop-minicluster from 3.3.5 to 3.3.6 in /test/fixtures/hdfs-fixture (#8257) #8268

Merged

harishbhakuni mentioned this issue Jun 27, 2023

[Snapshot Interop] Add Changes in Restore Snapshot Flow for remote store Interoperability #6788

Merged

6 tasks

mch2 added v2.9.0 'Issues and PRs related to version v2.9.0' and removed v2.8.0 'Issues and PRs related to version v2.8.0' labels Jun 29, 2023

anasalkouz added the Severity-Blocker label Jun 29, 2023

heemin32 mentioned this issue Jun 29, 2023

[Backport 2.x] GeoTile, and GeoGrid aggregation #8295

Merged

6 tasks

kotwanikunal mentioned this issue Jul 6, 2023

[BUG] org.opensearch.remotestore.RemoteStoreRefreshListenerIT.testRemoteRefreshRetryOnFailure is flaky #8506

Closed

ashking94 mentioned this issue Jul 7, 2023

Extend existing IndexRecoveryIT for remote indexes #8505

Merged

6 tasks

kotwanikunal assigned kotwanikunal and ashking94 and unassigned ashking94 and kotwanikunal Jul 7, 2023

anasalkouz removed the untriaged label Jul 7, 2023

ashking94 mentioned this issue Jul 8, 2023

[Remote Store] Mute testRemoteRefreshRetryOnFailure test case #8542

Merged

6 tasks

ashking94 mentioned this issue Jul 8, 2023

Fix flakyness in RemoteStoreRefreshListenerIT #8547

Merged

6 tasks

sachinpkale closed this as completed in #8547 Jul 9, 2023

shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this issue Apr 25, 2024

[Remote Store] Fix flaky test - closes opensearch-project#7703 (opens…

f2d4c95

…earch-project#7879) Signed-off-by: Ashish Singh <ssashish@amazon.com> Signed-off-by: Shivansh Arora <hishiv@amazon.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] org.opensearch.remotestore.RemoteStoreRefreshListenerIT.testRemoteRefreshRetryOnFailure is flaky #7703

[BUG] org.opensearch.remotestore.RemoteStoreRefreshListenerIT.testRemoteRefreshRetryOnFailure is flaky #7703

reta commented May 23, 2023

ashking94 commented Jun 2, 2023

dblock commented Jun 8, 2023

ashking94 commented Jul 7, 2023

kotwanikunal commented Jul 7, 2023

ashking94 commented Jul 8, 2023 •

edited

Loading

[BUG] org.opensearch.remotestore.RemoteStoreRefreshListenerIT.testRemoteRefreshRetryOnFailure is flaky #7703

[BUG] org.opensearch.remotestore.RemoteStoreRefreshListenerIT.testRemoteRefreshRetryOnFailure is flaky #7703

Comments

reta commented May 23, 2023

ashking94 commented Jun 2, 2023

dblock commented Jun 8, 2023

ashking94 commented Jul 7, 2023

kotwanikunal commented Jul 7, 2023

ashking94 commented Jul 8, 2023 • edited Loading

ashking94 commented Jul 8, 2023 •

edited

Loading