Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TimeSeriesLifecycleActionsIT muliple tests fail #53738

Closed
jkakavas opened this issue Mar 18, 2020 · 5 comments · Fixed by #54809
Closed

TimeSeriesLifecycleActionsIT muliple tests fail #53738

jkakavas opened this issue Mar 18, 2020 · 5 comments · Fixed by #54809
Assignees
Labels
:Data Management/ILM+SLM Index and Snapshot lifecycle management >test-failure Triaged test failures from CI

Comments

@jkakavas
Copy link
Member

we have 3 open issues https://github.com/elastic/elasticsearch/search?q=TimeSeriesLifecycleActionsIT&type=Issues so I thnk we can consolidate all these here and I'll mute the whole class

https://gradle-enterprise.elastic.co/s/p6yto4rhlkr2a

org.elasticsearch.xpack.ilm.TimeSeriesLifecycleActionsIT > testRetryFailedDeleteAction FAILED
    java.lang.AssertionError: 
    Expected: "delete"
         but: was null
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
        at org.junit.Assert.assertThat(Assert.java:956)
        at org.junit.Assert.assertThat(Assert.java:923)
        at org.elasticsearch.xpack.ilm.TimeSeriesLifecycleActionsIT.lambda$testRetryFailedDeleteAction$8(TimeSeriesLifecycleActionsIT.java:222)
        at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:887)
        at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:860)
        at org.elasticsearch.xpack.ilm.TimeSeriesLifecycleActionsIT.testRetryFailedDeleteAction(TimeSeriesLifecycleActionsIT.java:222)

    org.elasticsearch.client.ResponseException: method [DELETE], host [http://127.0.0.1:51980], URI [*?expand_wildcards=open%2Cclosed%2Chidden], status line [HTTP/1.1 403 Forbidden]
    {"error":{"root_cause":[{"type":"cluster_block_exception","reason":"index [index-wdjrnbsiia] blocked by: [FORBIDDEN/5/index read-only (api)];"}],"type":"cluster_block_exception","reason":"index [index-wdjrnbsiia] blocked by: [FORBIDDEN/5/index read-only (api)];"},"status":403}
        at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:283)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:261)
        at org.elasticsearch.client.RestClient.performRequest(RestClient.java:235)
        at org.elasticsearch.test.rest.ESRestTestCase.wipeAllIndices(ESRestTestCase.java:577)
        at org.elasticsearch.test.rest.ESRestTestCase.wipeCluster(ESRestTestCase.java:525)
        at org.elasticsearch.test.rest.ESRestTestCase.cleanUpCluster(ESRestTestCase.java:293)
        at jdk.internal.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.base/java.lang.Thread.run(Thread.java:834)
REPRODUCE WITH: ./gradlew ':x-pack:plugin:ilm:qa:multi-node:integTestRunner' --tests "org.elasticsearch.xpack.ilm.TimeSeriesLifecycleActionsIT.testRetryFailedDeleteAction" -Dtests.seed=CC45FDA1AE932803 -Dtests.security.manager=true -Dtests.locale=mk-MK -Dtests.timezone=Mexico/BajaNorte -Dcompiler.java=13

as a note testRetryFailedDeleteAction seems to started failing when #52494 was merged ( see here )

@jkakavas jkakavas added >test-failure Triaged test failures from CI :Data Management/ILM+SLM Index and Snapshot lifecycle management labels Mar 18, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (:Core/Features/ILM+SLM)

@dakrone
Copy link
Member

dakrone commented Mar 18, 2020

@jkakavas this class has over 45 tests, I don't think we should mute the entire class because 3-4 tests fail, that can lead to a regression for the stable tests in this class. There is an issue already open #53488 to refactor it into multiple classes, but I think muting the entire class is a bad idea here.

@jkakavas
Copy link
Member Author

fair enough, I'll revert this.

@dakrone
Copy link
Member

dakrone commented Mar 18, 2020

Please do mute the tests that did fail, I agree wholeheartedly for doing that, I just don't want to skip all the passing tests in this class.

@jkakavas
Copy link
Member Author

I just want to note that there are many tests that fail in this suite, not just 2,3 : https://build-stats.elastic.co/app/kibana#/discover?_g=(refreshInterval:(pause:!t,value:0),time:(from:now-30d,mode:quick,to:now))&_a=(columns:!(test),index:e58bf320-7efd-11e8-bf69-63c8ef516157,interval:auto,query:(language:lucene,query:TimeSeriesLifecycleActionsIT),sort:!(time,desc)) and it's hard for me ( as in random ES person on test triage ) to know if the historical failures of the last month are fixed in the meantime, or just never muted etc.

We can take a failure at a time if you think this is better, I don't have a reason to push back, I just wanted to make sure the extent of the failures is known

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/ILM+SLM Index and Snapshot lifecycle management >test-failure Triaged test failures from CI
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants