Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Auto configure TLS for new nodes of new clusters #77231

Merged
merged 184 commits into from
Oct 14, 2021
Merged

Auto configure TLS for new nodes of new clusters #77231

merged 184 commits into from
Oct 14, 2021

Conversation

albertzaharovits
Copy link
Contributor

@albertzaharovits albertzaharovits commented Sep 3, 2021 •
edited
Loading

This commit introduces TLS auto-configuration for elasticsearch nodes, during
the first startup. A number of heuristics are performed in order to determine if
the node should get TLS auto-configuration which can also be explicitly
disallowed with the use of xpack.security.autoconfiguration.enabled setting.

This affects archive installations and docker. Packaged installations are
handled in #75144 and #75704 .

Co-Authored-By: Ioannis Kakavas ioannis@elastic.co

@albertzaharovits albertzaharovits added :Security/Security Security issues without another label v8.0.0 :Core/Infra/CLI CLI utilities, scripts, and infrastructure labels Sep 3, 2021
@albertzaharovits albertzaharovits self-assigned this Sep 3, 2021
jkakavas
jkakavas reviewed Sep 3, 2021
View reviewed changes
distribution/src/bin/elasticsearch-env Outdated Show resolved Hide resolved
x-pack/plugin/security/src/main/bin/elasticsearch-security-config Outdated Show resolved Hide resolved
...lugin/security/cli/src/main/java/org/elasticsearch/xpack/security/cli/ConfigInitialNode.java Outdated Show resolved Hide resolved
distribution/src/bin/elasticsearch Outdated Show resolved Hide resolved
...lugin/security/cli/src/main/java/org/elasticsearch/xpack/security/cli/ConfigInitialNode.java Outdated Show resolved Hide resolved
...lugin/security/cli/src/main/java/org/elasticsearch/xpack/security/cli/ConfigInitialNode.java Outdated Show resolved Hide resolved
...lugin/security/cli/src/main/java/org/elasticsearch/xpack/security/cli/ConfigInitialNode.java Outdated Show resolved Hide resolved
...lugin/security/cli/src/main/java/org/elasticsearch/xpack/security/cli/ConfigInitialNode.java Outdated Show resolved Hide resolved
...lugin/security/cli/src/main/java/org/elasticsearch/xpack/security/cli/ConfigInitialNode.java Show resolved Hide resolved
...lugin/security/cli/src/main/java/org/elasticsearch/xpack/security/cli/ConfigInitialNode.java Outdated Show resolved Hide resolved
@jkakavas jkakavas assigned jkakavas and unassigned albertzaharovits Sep 5, 2021
@jkakavas jkakavas mentioned this pull request Sep 5, 2021
Merged
@jkakavas
Copy link
Member

jkakavas commented Sep 6, 2021 •
edited
Loading

Tracked down CI failures to the fact that with given changes we always need the xpack plugin in our TestClusters( because we need to use ConfigInitialNode even when to just determine that we should not autoconfigure security). That means that we need to add this to INTEG_TEST distribution

I'll adjust the rest of the changes according to discussions above in my morning and reach out to es-delivery folks about ideas for the INTEG_TEST distribution. Then I'll work through the remaining packaging tests failures

@albertzaharovits
Copy link
Contributor Author

@mark-vieira @pugnascotia @jkakavas I've finally tracked it down!!!
In the test code we were leaking a file handler to the C:\tmp\elasticsearch\config directory when doing a directory listing, see f544536 . It appears that Files.list(path).findFirst().get() leaks the file handler to path; at best a case of trappy API.

I've stepped debugged the tests on an Azure Win Server 2016 instance (IntelliJ running locally), and using the Resource Monitor and Process Explorer tools.

I'll be polishing the PR, and ask for reviews tomorrow.

@pugnascotia
Copy link
Contributor

@albertzaharovits you're a hero! 🏆

@jkakavas jkakavas self-requested a review October 13, 2021 20:21
@albertzaharovits
Copy link
Contributor Author

@mark-vieira I have pushed 1f3b26d , which I think addresses your point from #77231 (comment) .

Please take another look.

@mark-vieira
Copy link
Contributor

@mark-vieira I have pushed 1f3b26d , which I think addresses your point from #77231 (comment) .

Changes look good. Thanks @albertzaharovits!

@albertzaharovits albertzaharovits merged commit b257da1 into elastic:master Oct 14, 2021
jkakavas added a commit to jkakavas/elasticsearch that referenced this pull request Oct 17, 2021
@jkakavas
Add xpack core and security to INTEG TEST (elastic#77632)
4355afe 
This change makes it so x-pack-core and x-pack-security are bundled
in the INTEG TEST distribution that we use for testClusters in our
tests. There are two reasons for this:

- In elastic#77231 where we
are looking into enabling and auto-configuring security by default
for all nodes, we need to call out to ConfigInitialNode to
determine whether we should do the auto-configuration or not.
- Since we are enabling security by default, we should be looking
into enabling security for all for our tests moving forward, or
at least make a conscious decision about which ones run without
security. This change is a step towards that direction.

# Conflicts:
#	distribution/archives/build.gradle
#	distribution/packages/build.gradle
@jmlrt jmlrt mentioned this pull request Nov 5, 2021
Closed
@jmlrt jmlrt mentioned this pull request Jan 4, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jkakavas jkakavas jkakavas approved these changes

@rjernst rjernst rjernst approved these changes

@williamrandolph williamrandolph Awaiting requested review from williamrandolph

@pugnascotia pugnascotia Awaiting requested review from pugnascotia

@mark-vieira mark-vieira Awaiting requested review from mark-vieira

Assignees

@jkakavas jkakavas

Labels
:Core/Infra/CLI CLI utilities, scripts, and infrastructure :Delivery/Packaging RPM and deb packaging, tar and zip archives, shell and batch scripts >feature :Security/Security Security issues without another label Team:Core/Infra Meta label for core/infra team Team:Delivery Meta label for Delivery team Team:Security Meta label for security team v8.0.0-beta1
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@albertzaharovits @jkakavas @elasticmachine @elasticsearchmachine @pugnascotia @mark-vieira @rjernst @jakelandis