Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Features: Allow the TCP Input to receive events over a TLS connection #7056

Merged
merged 1 commit into from
May 17, 2018

Conversation

ph
Copy link
Contributor

@ph ph commented May 9, 2018

We can now receive events on the TCP input with a TLS connection, the
input uses existing type to make sure we have the same naming convention
and code used by outputs that support TLS communication (Elasticsearch
and Logstash).

The configuration will look like this:

  host: "localhost:9000"
  ssl.enabled: true
  ssl.verification_mode: full # default
  ssl.supported_protocols: [TLSv1.1]
  ssl.cipher_suites: []
  ssl.certificate_authorities: ["/etc/cacert"]
  ssl.certificate: /etc/mycert.crt
  ssl.key: /etc/mycert.key
  ssl.client_authentication: required

One added configuration is client_authentication, this option is
only used in the context of server and define how we will force the
authentification, it support the three following options:

  • required: Assume that the client will provide a certificate and we
    will verify it. (default)
  • optional: If a certificate is given by the client.
  • none: We don't validate client certificate.

*Note: This commit contains a script to generate certs from a self
signed CA.

Fixes: #6873

Depends on #7054

@ph ph force-pushed the feature/add-tls-to-tcp branch from c0398c3 to fb7a60d Compare May 9, 2018 15:30
@ph ph added review Filebeat Filebeat labels May 9, 2018
@ph
Copy link
Contributor Author

ph commented May 9, 2018

Traceback (most recent call last):
  File "/go/src/github.com/elastic/beats/filebeat/tests/system/test_tcp_tls.py", line 264, in test_tcp_tls_with_a_plain_text_socket
    sock.send("Hello World: " + str(n) + "\n")
AssertionError: IOError not raised

This is a valid error, current guest is don't write enough stuff to the socket to trigger the error and the message are in the buffer.

CAs []string `config:"certificate_authorities"`
Certificate CertificateConfig `config:",inline"`
CurveTypes []tlsCurveType `config:"curve_types"`
ClientAuth tlsClientAuth `config:"client_authentification"` //`none`, `optional` or `required`
Copy link
Contributor

@kvch kvch May 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you rename it to client_authentication, so non-native English speakers like me don't need to check the dictionary? :D Also, according to English on StackExchange it's less idiomatic. :D https://english.stackexchange.com/questions/7844/is-authentification-a-real-word

@@ -260,7 +260,7 @@ def test_tcp_tls_with_a_plain_text_socket(self):
# The TLS handshake will close the connection, resulting in a broken pipe.
# no events should be written on disk.
with assert_raises(IOError):
for n in range(0, 100):
for n in range(0, 100000):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you increase this number? How much time does it add to test execution time?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, it's outdated. Nevermind.

Copy link
Contributor

@kvch kvch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a nice refactor. However, the name of the new option should be changed.

@ph
Copy link
Contributor Author

ph commented May 11, 2018 via email

@ph
Copy link
Contributor Author

ph commented May 11, 2018

@ph ph force-pushed the feature/add-tls-to-tcp branch from 0925e3d to ff2e7cc Compare May 11, 2018 15:29
@ph
Copy link
Contributor Author

ph commented May 11, 2018

I have added the in progress label the failures on windows are a bit concerning.

@ph ph added in progress Pull request is currently in progress. and removed review labels May 11, 2018
@ph ph force-pushed the feature/add-tls-to-tcp branch from c53731d to 8cf0c38 Compare May 17, 2018 13:38
@ph
Copy link
Contributor Author

ph commented May 17, 2018

@kvch Updated with latest change from the original TLS PR

@kvch
Copy link
Contributor

kvch commented May 17, 2018

Could you rebase this branch, so moving stuff to tlscommon disappear? Is it ready to be reviewed/merged?

@ph ph force-pushed the feature/add-tls-to-tcp branch from 8cf0c38 to 0996973 Compare May 17, 2018 18:25
@ph
Copy link
Contributor Author

ph commented May 17, 2018

@kvch I've rebased the PR and its ready to get merged. :)

@ph ph added review and removed in progress Pull request is currently in progress. labels May 17, 2018
We can now receive events on the TCP input with a TLS connection, the
input uses existing type to make sure we have the same naming convention
and code used by outputs that support TLS communication (Elasticsearch
and Logstash).

The configuration will look like this:

```
  host: "localhost:9000"
  ssl.enabled: true
  ssl.verification_mode: full # default
  ssl.supported_protocols: [TLSv1.1]
  ssl.cipher_suites: []
  ssl.certificate_authorities: ["/etc/cacert"]
  ssl.certificate: /etc/mycert.crt
  ssl.key: /etc/mycert.key
  ssl.client_authentication: required

```

One added configuration is `client_authentication`, this option is
only used in the context of server and define how we will force the
authentification, it support the three following options:

- `required`: Assume that the client will provide a certificate and we
will verify it. (default)
- `optional`: If a certificate is given by the client.
- `none`: We don't validate client certificate.

*Note: This commit contains a script to generate certs from a self
signed CA.

Fixes: elastic#6873
@ph ph force-pushed the feature/add-tls-to-tcp branch from 0996973 to d686b9b Compare May 17, 2018 18:35
@ph
Copy link
Contributor Author

ph commented May 17, 2018

jenkins test this please

@kvch kvch merged commit 85c0d05 into elastic:master May 17, 2018
stevea78 pushed a commit to stevea78/beats that referenced this pull request May 20, 2018
…elastic#7056)

We can now receive events on the TCP input with a TLS connection, the
input uses existing type to make sure we have the same naming convention
and code used by outputs that support TLS communication (Elasticsearch
and Logstash).

The configuration will look like this:

```
  host: "localhost:9000"
  ssl.enabled: true
  ssl.verification_mode: full # default
  ssl.supported_protocols: [TLSv1.1]
  ssl.cipher_suites: []
  ssl.certificate_authorities: ["/etc/cacert"]
  ssl.certificate: /etc/mycert.crt
  ssl.key: /etc/mycert.key
  ssl.client_authentication: required

```

One added configuration is `client_authentication`, this option is
only used in the context of server and define how we will force the
authentification, it support the three following options:

- `required`: Assume that the client will provide a certificate and we
will verify it. (default)
- `optional`: If a certificate is given by the client.
- `none`: We don't validate client certificate.

*Note: This commit contains a script to generate certs from a self
signed CA.

Fixes: elastic#6873
stevea78 pushed a commit to stevea78/beats that referenced this pull request May 20, 2018
…elastic#7056)

We can now receive events on the TCP input with a TLS connection, the
input uses existing type to make sure we have the same naming convention
and code used by outputs that support TLS communication (Elasticsearch
and Logstash).

The configuration will look like this:

```
  host: "localhost:9000"
  ssl.enabled: true
  ssl.verification_mode: full # default
  ssl.supported_protocols: [TLSv1.1]
  ssl.cipher_suites: []
  ssl.certificate_authorities: ["/etc/cacert"]
  ssl.certificate: /etc/mycert.crt
  ssl.key: /etc/mycert.key
  ssl.client_authentication: required

```

One added configuration is `client_authentication`, this option is
only used in the context of server and define how we will force the
authentification, it support the three following options:

- `required`: Assume that the client will provide a certificate and we
will verify it. (default)
- `optional`: If a certificate is given by the client.
- `none`: We don't validate client certificate.

*Note: This commit contains a script to generate certs from a self
signed CA.

Fixes: elastic#6873
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants