Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Filebeat] Module for Ubiquiti Firewall Logs #8781

Closed
andrewkroh opened this issue Oct 29, 2018 · 2 comments
Closed

[Filebeat] Module for Ubiquiti Firewall Logs #8781

andrewkroh opened this issue Oct 29, 2018 · 2 comments
Assignees
@jamesspi
Labels
enhancement Filebeat Filebeat module

Comments

@andrewkroh
Copy link
Member

andrewkroh commented Oct 29, 2018 •
edited
Loading

As a user I want to be able to ingest firewall logs from Ubiquiti network gear. Ubiquiti firewall logs are essentially Linux iptables log message with a prefix that designates the source interface. In my experience the primary means of getting these logs is via syslog. Here are some samples (without the syslog header).

  • [wan-local-default-D]IN=eth0 OUT= MAC=05:10:d6:f0:81:b4:f8:e4:00:9a:f9:00:08:00 SRC=23.102.178.95 DST=192.168.1.4 LEN=1357 TOS=0x00 PREC=0x00 TTL=116 ID=18905 DF PROTO=TCP SPT=443 DPT=33785 WINDOW=514 RES=0x00 ACK PSH URGP=0
  • [wan-lan-3-A]IN=eth0 OUT=eth1 MAC=05:10:d6:f0:81:b4:f8:e4:fb:00:f9:00:08:00 SRC=1.2.3.4 DST=10.0.0.5 LEN=60 TOS=0x00 PREC=0x00 TTL=55 ID=3097 DF PROTO=TCP SPT=54488 DPT=22 WINDOW=14600 RES=0x00 SYN URGP=0
@elasticmachine
Copy link
Collaborator

Pinging @elastic/secops

@jamesspi
Copy link
Contributor

@andrewkroh - plan on having a PR for this early Jan. Sorry for the delay!

@adriansr adriansr mentioned this issue Jan 25, 2019
Merged
adriansr added a commit to adriansr/beats that referenced this issue Jan 28, 2019
@adriansr @jamesspi
Iptables module for Filebeat
938b032 
This adds a new x-pack module to parse iptables logs from a file.
The module understands additional fields added by Ubiquiti firewalls.

Example iptables event:
```
"iptables": {
      "ether_type": 2048,
      "input_device": "eth0",
      "output_device": "wan1",
      "tcp": {
        "reserved_bits": 0,
        "flags": "ACK",
        "window": 128
      },
      "precedence_bits": 0,
      "length": 123,
      "tos": 0,
      "output_device": "lo",
      "id": 12345,
      "ttl": 255
    },
```

For Ubiquiti logs, it parses the Ubiquiti tag before iptables logs.

Format: RuleSet-RuleNumber-Action

It's a common pattern to format the RuleSet as "InputDevice-OutputDevice".
Action can be A for allow and D for Deny. RuleNumber can be a number
or "default".

For [source-dest-42-D]:
```
        "iptables.ubiquiti.input_zone": "source",
        "iptables.ubiquiti.output_zone": "dest",
        "iptables.ubiquiti.rule_number": "42",
        "iptables.ubiquiti.rule_set": "source-dest",
        "event.outcome": "deny",
```
The following ECS fields are populated:

```
        "destination.ip": "10.4.0.5",
        "destination.mac": "90:10:20:76:8d:20",
        "destination.port": 443,
        "ecs.version": "1.0.0-beta2",
        "event.dataset": "iptables.log",
        "event.module": "iptables",
        "event.outcome": "deny", (only when Ubiquiti extension found)
        "fileset.name": "log",
        "network.transport": "tcp",
        "network.type": "ipv4",
        "source.geo.city_name": "Bellaterra",
        "source.geo.continent_name": "Europe",
        "source.geo.country_iso_code": "ES",
        "source.geo.location.lat": 41.5026,
        "source.geo.location.lon": 2.0875,
        "source.geo.region_iso_code": "ES-B",
        "source.geo.region_name": "Barcelona",
        "source.ip": "158.109.0.1",
        "source.mac": "90:10:65:29:b6:2a",
        "source.port": 38842
```

Closes elastic#8781

Co-Authored-by: James Spiteri <james.spiteri@elastic.co>
adriansr added a commit that referenced this issue Jan 29, 2019
@adriansr @jamesspi
Iptables module for Filebeat (#10176)
d7c14df 
This adds a new x-pack module to parse iptables logs from a file.
The module understands additional fields added by Ubiquiti firewalls.

Example iptables event:
```
"iptables": {
      "ether_type": 2048,
      "input_device": "eth0",
      "output_device": "wan1",
      "tcp": {
        "reserved_bits": 0,
        "flags": "ACK",
        "window": 128
      },
      "precedence_bits": 0,
      "length": 123,
      "tos": 0,
      "output_device": "lo",
      "id": 12345,
      "ttl": 255
    },
```

For Ubiquiti logs, it parses the Ubiquiti tag before iptables logs.

Format: RuleSet-RuleNumber-Action

It's a common pattern to format the RuleSet as "InputDevice-OutputDevice".
Action can be A for allow and D for Deny. RuleNumber can be a number
or "default".

For [source-dest-42-D]:
```
        "iptables.ubiquiti.input_zone": "source",
        "iptables.ubiquiti.output_zone": "dest",
        "iptables.ubiquiti.rule_number": "42",
        "iptables.ubiquiti.rule_set": "source-dest",
        "event.outcome": "deny",
```
The following ECS fields are populated:

```
        "destination.ip": "10.4.0.5",
        "destination.mac": "90:10:20:76:8d:20",
        "destination.port": 443,
        "ecs.version": "1.0.0-beta2",
        "event.dataset": "iptables.log",
        "event.module": "iptables",
        "event.outcome": "deny", (only when Ubiquiti extension found)
        "fileset.name": "log",
        "network.transport": "tcp",
        "network.type": "ipv4",
        "source.geo.city_name": "Bellaterra",
        "source.geo.continent_name": "Europe",
        "source.geo.country_iso_code": "ES",
        "source.geo.location.lat": 41.5026,
        "source.geo.location.lon": 2.0875,
        "source.geo.region_iso_code": "ES-B",
        "source.geo.region_name": "Barcelona",
        "source.ip": "158.109.0.1",
        "source.mac": "90:10:65:29:b6:2a",
        "source.port": 38842
```

Closes #8781

Co-Authored-by: James Spiteri <james.spiteri@elastic.co>
adriansr added a commit to adriansr/beats that referenced this issue Jan 31, 2019
@adriansr
Iptables module for Filebeat (elastic#10176)
fe1b368 
This adds a new x-pack module to parse iptables logs from a file.
The module understands additional fields added by Ubiquiti firewalls.

Example iptables event:
```
"iptables": {
      "ether_type": 2048,
      "input_device": "eth0",
      "output_device": "wan1",
      "tcp": {
        "reserved_bits": 0,
        "flags": "ACK",
        "window": 128
      },
      "precedence_bits": 0,
      "length": 123,
      "tos": 0,
      "output_device": "lo",
      "id": 12345,
      "ttl": 255
    },
```

For Ubiquiti logs, it parses the Ubiquiti tag before iptables logs.

Format: RuleSet-RuleNumber-Action

It's a common pattern to format the RuleSet as "InputDevice-OutputDevice".
Action can be A for allow and D for Deny. RuleNumber can be a number
or "default".

For [source-dest-42-D]:
```
        "iptables.ubiquiti.input_zone": "source",
        "iptables.ubiquiti.output_zone": "dest",
        "iptables.ubiquiti.rule_number": "42",
        "iptables.ubiquiti.rule_set": "source-dest",
        "event.outcome": "deny",
```
The following ECS fields are populated:

```
        "destination.ip": "10.4.0.5",
        "destination.mac": "90:10:20:76:8d:20",
        "destination.port": 443,
        "ecs.version": "1.0.0-beta2",
        "event.dataset": "iptables.log",
        "event.module": "iptables",
        "event.outcome": "deny", (only when Ubiquiti extension found)
        "fileset.name": "log",
        "network.transport": "tcp",
        "network.type": "ipv4",
        "source.geo.city_name": "Bellaterra",
        "source.geo.continent_name": "Europe",
        "source.geo.country_iso_code": "ES",
        "source.geo.location.lat": 41.5026,
        "source.geo.location.lon": 2.0875,
        "source.geo.region_iso_code": "ES-B",
        "source.geo.region_name": "Barcelona",
        "source.ip": "158.109.0.1",
        "source.mac": "90:10:65:29:b6:2a",
        "source.port": 38842
```

Closes elastic#8781

Co-Authored-by: James Spiteri <james.spiteri@elastic.co>
(cherry picked from commit d7c14df)
adriansr added a commit that referenced this issue Feb 4, 2019
@adriansr
Cherry-pick of #10176 to 6.x: [Filebeat] Iptables / ubiquiti module (#…
190d2a1 
…10462)

This adds a new x-pack module to parse iptables logs from a file.
The module understands additional fields added by Ubiquiti firewalls.

Example iptables event:
```
"iptables": {
      "ether_type": 2048,
      "input_device": "eth0",
      "output_device": "wan1",
      "tcp": {
        "reserved_bits": 0,
        "flags": "ACK",
        "window": 128
      },
      "precedence_bits": 0,
      "length": 123,
      "tos": 0,
      "output_device": "lo",
      "id": 12345,
      "ttl": 255
    },
```

For Ubiquiti logs, it parses the Ubiquiti tag before iptables logs.

Format: RuleSet-RuleNumber-Action

It's a common pattern to format the RuleSet as "InputDevice-OutputDevice".
Action can be A for allow and D for Deny. RuleNumber can be a number
or "default".

For [source-dest-42-D]:
```
        "iptables.ubiquiti.input_zone": "source",
        "iptables.ubiquiti.output_zone": "dest",
        "iptables.ubiquiti.rule_number": "42",
        "iptables.ubiquiti.rule_set": "source-dest",
        "event.outcome": "deny",
```
The following ECS fields are populated:

```
        "destination.ip": "10.4.0.5",
        "destination.mac": "90:10:20:76:8d:20",
        "destination.port": 443,
        "ecs.version": "1.0.0-beta2",
        "event.dataset": "iptables.log",
        "event.module": "iptables",
        "event.outcome": "deny", (only when Ubiquiti extension found)
        "fileset.name": "log",
        "network.transport": "tcp",
        "network.type": "ipv4",
        "source.geo.city_name": "Bellaterra",
        "source.geo.continent_name": "Europe",
        "source.geo.country_iso_code": "ES",
        "source.geo.location.lat": 41.5026,
        "source.geo.location.lon": 2.0875,
        "source.geo.region_iso_code": "ES-B",
        "source.geo.region_name": "Barcelona",
        "source.ip": "158.109.0.1",
        "source.mac": "90:10:65:29:b6:2a",
        "source.port": 38842
```

Closes #8781

Co-Authored-by: James Spiteri <james.spiteri@elastic.co>
(cherry picked from commit d7c14df)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jamesspi jamesspi

Labels
enhancement Filebeat Filebeat module
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@andrewkroh @elasticmachine @jamesspi