Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for ~UDP~ Datagram unix socket input #18632

Closed
phlax opened this issue May 19, 2020 · 6 comments · Fixed by #22699
Closed

Add support for ~UDP~ Datagram unix socket input #18632

phlax opened this issue May 19, 2020 · 6 comments · Fixed by #22699
Labels
enhancement Team:Services (Deprecated) Label for the former Integrations-Services team

Comments

@phlax
Copy link

phlax commented May 19, 2020

Describe the enhancement:

There is support for unix socket inputs in filebeat from version 7.8

This is done using protocol.unix as described here

This creates a TCP socket - but eg rsyslog assumes UDP

for ref, the ticket where support was added #10934

Describe a specific use case for the enhancement or feature:

use with any syslog engine that outputs to a UDP socket

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 19, 2020
@andresrc andresrc added enhancement Team:Services (Deprecated) Label for the former Integrations-Services team labels May 22, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 22, 2020
@andresrc
Copy link
Contributor

We currently support syslog via UDP. Can you be more specific regarding what's missing?

Thanks

@phlax
Copy link
Author

phlax commented May 25, 2020

i followed the above instructions, and then set rsyslog - the default on debian - to output to the input socket.

That didnt work, as in i received no logs.

I then tested by creating a socket connection to the filebeat socket input myself.

First with tcp . This worked - echoing some "logs" into the socket resulted in indexed logs in kibana.

I then tried again with udp - which silently (iirc) failed - as in no log output.

This may work with other implementations of syslog but with the default on debian it seems to fail

@phlax
Copy link
Author

phlax commented May 25, 2020

To be absolutely clear this is regarding unix socket input - no doubt network udp works

@dcode
Copy link
Contributor

dcode commented Jun 2, 2020

I think the specific request is support for UNIX datagram socket versus the newly added support for UNIX stream socket.

@phlax
Copy link
Author

phlax commented Jun 2, 2020

mostly yes - but the protocol isnt really unix and this also makes it confusing

@phlax phlax changed the title Add support for UDP unix socket input Add support for ~UDP~ Datagram unix socket input Jun 11, 2020
kvch added a commit that referenced this issue Nov 30, 2020
## What does this PR do?

This PR adds support for reading from UNIX datagram sockets both from the `unix` input and the `syslog` input. A new option is added to select the type of the socket named `socket_type`. Available options are: `stream` and `datagram`.

## Why is it important?

A few applications which send logs over Unix sockets, use datagrams not streams. From now on, Filebeat can accept input from these applications as well.

Closes #18632
kvch added a commit to kvch/beats that referenced this issue Nov 30, 2020
## What does this PR do?

This PR adds support for reading from UNIX datagram sockets both from the `unix` input and the `syslog` input. A new option is added to select the type of the socket named `socket_type`. Available options are: `stream` and `datagram`.

## Why is it important?

A few applications which send logs over Unix sockets, use datagrams not streams. From now on, Filebeat can accept input from these applications as well.

Closes elastic#18632

(cherry picked from commit 3a1d1ae)
kvch added a commit that referenced this issue Nov 30, 2020
## What does this PR do?

This PR adds support for reading from UNIX datagram sockets both from the `unix` input and the `syslog` input. A new option is added to select the type of the socket named `socket_type`. Available options are: `stream` and `datagram`.

## Why is it important?

A few applications which send logs over Unix sockets, use datagrams not streams. From now on, Filebeat can accept input from these applications as well.

Closes #18632

(cherry picked from commit 3a1d1ae)
@zube zube bot removed the [zube]: Done label Mar 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Team:Services (Deprecated) Label for the former Integrations-Services team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants