Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testclusters: implement starting, waiting for and stopping single cluster nodes #35599

Merged
merged 18 commits into from
Dec 4, 2018

Conversation

alpar-t
Copy link
Contributor

@alpar-t alpar-t commented Nov 15, 2018

The bulk of the request is new functionality in ElasticsearchNode.
Note that this offers 0 configuration, this ability will be added as we start to use testclsuters and find a need for things to be configurable to keep these to a minimum.
My thinking is that we can auto configure in some of the cases where we have configuration for cluster-formation ( e.x. detect that security is installed and do the right checks automatically).

The accompanying tests are still the only users of this functionality.

A quick map to help navigate the PR:

  • setting up a directory layout and writing config file writeConfigurationFile
    • we don't copy the entire distribution, only the config folder and create folders in a different location to store data, logs, etc.
  • We start the process with ProcessBuilder and redirect the streams to files start
    • There are cleanup hooks to make sure processes started by the build won't outlive it registerCleanupHooks
  • Stopping the cluster stops all processes with the same api in stop and reads the files into the stream on failure.
  • We wait for the cluster to be ready only when and each time we get a request for it's address. There's a generic mechanism to deal with waiting, but it's not configurable for now.

Tests are passing on Windows.

@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

@alpar-t
Copy link
Contributor Author

alpar-t commented Nov 16, 2018

Thanks for the review @original-brownbear ! I left some answers.

@original-brownbear
Copy link
Member

@atorok thanks! Looks good in general :), but see my comment about the thread leak.

@alpar-t
Copy link
Contributor Author

alpar-t commented Nov 19, 2018

I moved the cleanup of processes outside of ElasticsearchNode I think it's a better fit for the plugin, and this also takes care to make sure the thread is not leaked in the daemon.

@alpar-t
Copy link
Contributor Author

alpar-t commented Nov 21, 2018

@nik9000 can you take another look please.

Copy link
Member

@nik9000 nik9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy with this. I think it pushes the project forwards and i like it. I think there are a couple of unrelated changes bundled up in it. At least, I can't see how they are related. Could you explain them? Or maybe move them into their own PR so they get a nice commit message explaining them?

@alpar-t
Copy link
Contributor Author

alpar-t commented Nov 29, 2018

@nik9000 I should have offered more details on these changes . I now realize they seem out of place. They do need to be part of this PR.
In #34746 I moved buildSrc to target java 10 ( 11 doesn't work with gradle 4.10 ) but this is the first change that actually uses something newer than java 8. Turns out my previous change was incomplete as it changed buildSrc but not build-tools because the build plugin was setting it back to 8 so we need to be specific about it again after applying the plugin. I'll look to clean up the package naming change, that's unintentional.

@alpar-t
Copy link
Contributor Author

alpar-t commented Nov 29, 2018

@nik9000 we do also need to run the naming convention check with the compiler java as build-tools how has classes for newer java versions. We could do it locally for build tools only but it doesn't matter much since we'll be using compiler java for this check with testingConventions soon anyhow.

@alpar-t
Copy link
Contributor Author

alpar-t commented Nov 30, 2018

@elasticmachine run gradle tests 1

@alpar-t
Copy link
Contributor Author

alpar-t commented Nov 30, 2018

@elasticmachine run gradle build tests 1

@alpar-t
Copy link
Contributor Author

alpar-t commented Dec 3, 2018

@nik9000 I double checked to be sure. All comments on the PR should be addressed.

@alpar-t
Copy link
Contributor Author

alpar-t commented Dec 3, 2018

@elasticmachine run default distro tests

1 similar comment
@alpar-t
Copy link
Contributor Author

alpar-t commented Dec 3, 2018

@elasticmachine run default distro tests

@alpar-t alpar-t merged commit d036e0c into elastic:master Dec 4, 2018
@alpar-t alpar-t deleted the testclusters-start-node branch December 4, 2018 08:17
alpar-t added a commit that referenced this pull request Dec 4, 2018
@mark-vieira mark-vieira added the Team:Delivery Meta label for Delivery team label Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Delivery/Build Build or test infrastructure >non-issue Team:Delivery Meta label for Delivery team v6.6.0 v7.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants