CI: rolling upgrade CCRIT.testAutoFollowing fails

[alpar-t]

Example build failure

https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+6.x+release-tests/384/
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+intake/1390/

Reproduction line

does not reproduce locally

./gradlew :x-pack:qa:rolling-upgrade:with-system-key:v6.7.0#twoThirdsUpgradedTestRunner -Dtests.seed=45813FEF3A0677C -Dtests.class=org.elasticsearch.upgrades.CCRIT -Dtests.method="testAutoFollowing" -Dtests.security.manager=true -Dtests.locale=hu-HU -Dtests.timezone=Asia/Nicosia -Dcompiler.java=11 -Druntime.java=8

Example relevant log:

12:37:16 FAILURE 10.7s | CCRIT.testAutoFollowing <<< FAILURES!
12:37:16    > Throwable #1: java.lang.AssertionError: 
12:37:16    > Expected: <0>
12:37:16    >      but: was <-1>
12:37:16    > 	at __randomizedtesting.SeedInfo.seed([45813FEF3A0677C:ABB635FEF709CC51]:0)
12:37:16    > 	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
12:37:16    > 	at org.elasticsearch.upgrades.CCRIT.assertFollowerGlobalCheckpoint(CCRIT.java:275)
12:37:16    > 	at org.elasticsearch.upgrades.CCRIT.lambda$testAutoFollowing$9(CCRIT.java:159)
12:37:16    > 	at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:847)
12:37:16    > 	at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:821)
12:37:16    > 	at org.elasticsearch.upgrades.CCRIT.testAutoFollowing(CCRIT.java:156)
12:37:16    > 	at java.lang.Thread.run(Thread.java:748)
12:37:16    > 	Suppressed: java.lang.AssertionError: 
12:37:16    > Expected: <3>
12:37:16    >      but: was <2>
12:37:16    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
12:37:16    > 		at org.elasticsearch.upgrades.CCRIT.lambda$testAutoFollowing$9(CCRIT.java:158)
12:37:16    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:835)
12:37:16    > 		... 39 more
12:37:16    > 	Suppressed: java.lang.AssertionError: 

Frequency

14 / last 24 hours

cc @martijnvg maybe similar to #37231 ?

[martijnvg]

@atorok I will check. I'm surprised to see it fail so often now. If I can't figure it out then I will mute this test.

[martijnvg]

Both of these build didn't run with this commit that I pushed yesterday:
a3180fd

I like to see and wait if this test still fails after this commit.

[elasticmachine]

Pinging @elastic/es-distributed

[alpar-t]

@martijnvg the frequency includes PRs fro which we can't know if your change was incorporated or not.

[martijnvg]

This test hasn't failed since the above commit was pushed.