Begin replacing static index tests with full restart tests #24846
Conversation
These tests spin up two nodes of an older version of Elasticsearch, create some stuff, shut down the nodes, start the current version, and verify that the created stuff works. You can run `gradle qa:full-cluster-restart:check` to run these tests against the head of the previous branch of Elasticsearch (5.x for master, 5.4 for 5.x, etc) or you can run `gradle qa:full-cluster-restart:bwcTest` to run this test against all "index compatible" versions, one after the other. For master this is every released version in the 5.x.y version *and* the tip of the 5.x branch. I'd love to add more to these tests in the future but these currently just cover the functionality of the `create_bwc_index.py` script and start to cover the assertions in the `OldIndexBackwardsCompatibilityIT` test.
nik9000
added
review
>test
There was a problem hiding this comment.
Thanks for doing this! This is a good first step. One thing I realized is to make this reproducible (not just this test, but any that use multiple rest tests which have randomization) we need to make sure the random seed passed is consistent. We can't rely on the generated random seed inside randomized runner, since each runner would get it's own unique independent seed.
qa/full-cluster-restart/build.gradle
Outdated
|
|
||
| configure(extensions.findByName("${baseName}#upgradedClusterTestCluster")) { | ||
| dependsOn oldClusterTestRunner, | ||
| "${baseName}#oldClusterTestCluster#node0.stop", |
There was a problem hiding this comment.
You should be able to depend on oldClusterTest instead of oldClusterTestRunner and the cluster will be shut down.
| * after projects are evaluated. */ | ||
| gradle.projectsEvaluated { | ||
| // Disable cleaning the repository so we can test loading a snapshot | ||
| tasks.getByName("${baseName}#upgradedClusterTestCluster#prepareCluster.cleanShared").enabled = false |
There was a problem hiding this comment.
This is kind of sneaky...maybe have a flag on cluster configuration that means "don't create a shared clean task"? Something about "reuse"?
There was a problem hiding this comment.
I'll open a followup for this.
| doc.field("int", randomInt(100)); | ||
| doc.field("float", randomFloat()); | ||
| // be sure to create a "proper" boolean (True, False) for the first document so that automapping is correct | ||
| doc.field("bool", i > 0 && supportsLenientBooleans ? randomLenientBoolean() : randomBoolean()); |
There was a problem hiding this comment.
I don't think we need to test lenient booleans. Sure the old cluster supported them, but they supported proper booleans too. They are gone from master now, so let's just stick with keeping this simple, there is enough complicated stuff here.
There was a problem hiding this comment.
I think it is important that we make sure we can query indices with old style booleans in there. I know that it'll work because we've already built the index, but I think it is worth checking.
There was a problem hiding this comment.
I had mistakenly thought master did not have any lenient booleans support. This sounds fine.
Yes. |
* master: (38 commits) Fix Lucene version expectation Verify Lucene version constants Avoid double decrement on current query counter Remove the need for _UNRELEASED suffix in versions (elastic#24798) Adjust available and free bytes to be non-negative on huge FSes Begin replacing static index tests with full restart tests (elastic#24846) Fix plugin docs for using custom config dir Update context-suggest.asciidoc Move BWC version to 5.5 after backport Support Multiple Collapse Inner Hits Scripting: Rename CompiledType to FactoryType in ScriptContext (elastic#24897) Scripting: Make contexts available to ScriptEngine construction (elastic#24896) Mute index and relocate concurrently Build: Add back explicit exclusions and remove gradle exclusions (elastic#24879) Scripting: Move context definitions to instance type classes (elastic#24883) Build: Fix hadoop integ test error on windows (elastic#24885) Put mapping and index template requests do not need content type detection for 5.3.0+ (elastic#24835) Add the ability to store objects with a ScrollContext (elastic#24777) add docs example for Ingest scripts manipulating document metadata (elastic#24875) Fix error message if an incompatible node connects (elastic#24884) ...
This adds an option to `ClusterConfiguration` to preserve the `shared` directory when starting up a new cluster and switches the `qa:full-cluster-restart` tests to use it rather than disable the clean shared task. Relates to elastic#24846
This adds an option to `ClusterConfiguration` to preserve the `shared` directory when starting up a new cluster and switches the `qa:full-cluster-restart` tests to use it rather than disable the clean shared task. Relates to #24846
These tests spin up two nodes of an older version of Elasticsearch,
create some stuff, shut down the nodes, start the current version,
and verify that the created stuff works.
You can run
gradle qa:full-cluster-restart:checkto run thesetests against the head of the previous branch of Elasticsearch
(5.x for master, 5.4 for 5.x, etc) or you can run
gradle qa:full-cluster-restart:bwcTestto run this test againstall "index compatible" versions, one after the other. For master
this is every released version in the 5.x.y version and the tip
of the 5.x branch.
I'd love to add more to these tests in the future but these
currently just cover the functionality of the
create_bwc_index.pyscript and start to cover the assertions in the
OldIndexBackwardsCompatibilityITtest.