Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry-pick #17738 to 7.x: [Filebeat] Improve ECS categorization field mappings for zeek module #18076

Merged
merged 2 commits into from
May 4, 2020

Conversation

leehinman
Copy link
Contributor

Cherry-pick of PR #17738 to 7.x branch. Original message:

What does this PR do?

Updates the filesets of zeek module to ECS 1.5

Why is it important?

ECS categorization fields improve experience in SIEM module and across inputs from different sources.

Checklist

  • My code follows the style guidelines of this project
    - [ ] I have commented my code, particularly in hard-to-understand areas
    - [ ] I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

TESTING_FILEBEAT_MODULES=zeek mage -v pythonIntegTest

Related issues

Closes #16029

Changes to each Fileset

  • capture_loss
    • convert pipeline to yaml
    • event.kind
    • event.type
  • connection
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • related.ip
  • dce_rpc
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.action
    • related.ip
    • source.geo
    • destination.geo
  • dhcp
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • related.ip
  • dnp3
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.action
    • related.ip
    • source.geo
    • destination.geo
    • source.as
    • destiantion.as
  • dns
    • event.kind
    • event.category
    • event.type
    • event.outcome
    • dns.question.top_level_domain
    • related.ip
  • dpd
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • source.geo
    • destiantion.geo
    • source.as
    • destiantion.as
    • related.ip
  • files
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • file.mime_type
    • file.name
    • file.hash.md5
    • file.hash.sha1
    • file.hash.sha256
    • client.ip
    • server.ip
    • related.ip
    • related.hash
  • ftp
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.action
    • user.name
    • file.mime_type
    • file.size
    • related.ip
    • related.user
    • source.geo
    • destination.geo
  • http
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.outcome
    • event.action
    • url.port type to number
    • http.request.method lowercase
    • related.ip
    • related.users
  • intel
    • event.kind
    • event.type
    • related.ip
  • irc
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.action
    • user.name
    • file.name
    • file.size
    • file.mime_type
    • related.ip
    • related.user
    • source.geo
    • destination.geo
  • kerberos
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.outcome
    • event.action
    • user.name
    • user.domain
    • source.geo
    • destination.geo
    • related.ip
    • related.user
  • modbus
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.outcome
    • event.action
    • related.ip
    • source.geo
    • destination.geo
  • mysql
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.outcome
    • source.geo
    • destiantion.geo
    • related.ip
  • notice
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • related.ip
    • file.size
    • file.mime_type
    • rule.description
    • rule.name
  • ntlm
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.outcome
    • user.name
    • user.domain
    • source.geo
    • destination.geo
    • related.ip
    • related.user
  • ocsp
    • convert pipeline to yaml
    • event.kind
    • related.hash
  • pe
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
  • radius
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.outcome
    • user.name
    • source.geo
    • destination.geo
    • related.ip
    • related.user
  • rdp
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • related.ip
    • source.geo
    • destination.geo
  • rfb
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • related.ip
    • source.geo
    • destination.geo
  • sip
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.outcome
    • event.action
    • related.ip
    • source.geo
    • destination.geo
    • url.full
  • smb_cmd
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.outcome
    • event.action
    • related.ip
    • related.user
    • source.geo
    • destination.geo
    • user.name
  • smb_files
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.action
    • related.ip
    • related.user
    • source.geo
    • destination.geo
    • user.name
    • file.accessed
    • file.ctime
    • file.created
    • file.mtime
    • file.path
    • file.name
    • file.size
  • smb_mapping
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • related.ip
    • source.geo
    • destination.geo
  • smtp
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • related.ip
    • source.geo
    • destination.geo
  • snmp
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • related.ip
    • source.geo
    • destination.geo
  • socks
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.outcome
    • related.ip
    • source.geo
    • destination.geo
    • user.name
    • related.user
  • ssh
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • event.outcome
    • related.ip
    • source.geo
    • destination.geo
  • ssl
    • event.kind
    • event.category
    • event.type
    • related.ip
  • stats
    • convert pipeline to yaml
    • event.kind
  • syslog
    • convert pipeline to yaml
    • event.kind
    • related.ip
    • source.geo
    • destination.geo
    • log.syslog.facility.name
    • log.syslog.severity.name
  • traceroute
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • related.ip
    • source.geo
    • destination.geo
  • tunnel
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • related.ip
    • source.geo
    • destination.geo
  • weird
    • convert pipeline to yaml
    • event.kind
    • event.category
    • event.type
    • related.ip
    • source.geo
    • destination.geo
    • rule.name
  • x509
    • event.kind
    • event.type

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Apr 29, 2020
…lastic#17738)

* Improve ECS categorization field mappings for zeek module

- capture_loss
  + convert pipeline to yaml
  + event.kind
  + event.type
- connection
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + related.ip
- dce_rpc
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.action
  + related.ip
  + source.geo
  + destination.geo
- dhcp
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + related.ip
- dnp3
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.action
  + related.ip
  + source.geo
  + destination.geo
  + source.as
  + destiantion.as
- dns
  + event.kind
  + event.category
  + event.type
  + event.outcome
  + dns.question.top_level_domain
  + related.ip
- dpd
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + source.geo
  + destiantion.geo
  + source.as
  + destiantion.as
  + related.ip
- files
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + file.mime_type
  + file.name
  + file.hash.md5
  + file.hash.sha1
  + file.hash.sha256
  + client.ip
  + server.ip
  + related.ip
  + related.hash
- ftp
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.action
  + user.name
  + file.mime_type
  + file.size
  + related.ip
  + related.user
  + source.geo
  + destination.geo
- http
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.outcome
  + event.action
  + url.port type to number
  + http.request.method lowercase
  + related.ip
  + related.users
- intel
  + event.kind
  + event.type
  + related.ip
- irc
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.action
  + user.name
  + file.name
  + file.size
  + file.mime_type
  + related.ip
  + related.user
  + source.geo
  + destination.geo
- kerberos
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.outcome
  + event.action
  + user.name
  + user.domain
  + source.geo
  + destination.geo
  + related.ip
  + related.user
- modbus
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.outcome
  + event.action
  + related.ip
  + source.geo
  + destination.geo
- mysql
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.outcome
  + source.geo
  + destiantion.geo
  + related.ip
- notice
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + related.ip
  + file.size
  + file.mime_type
  + rule.description
  + rule.name
- ntlm
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.outcome
  + user.name
  + user.domain
  + source.geo
  + destination.geo
  + related.ip
  + related.user
- ocsp
  + convert pipeline to yaml
  + event.kind
  + related.hash
- pe
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
- radius
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.outcome
  + user.name
  + source.geo
  + destination.geo
  + related.ip
  + related.user
- rdp
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + related.ip
  + source.geo
  + destination.geo
- rfb
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + related.ip
  + source.geo
  + destination.geo
- sip
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.outcome
  + event.action
  + related.ip
  + source.geo
  + destination.geo
  + url.full
- smb_cmd
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.outcome
  + event.action
  + related.ip
  + related.user
  + source.geo
  + destination.geo
  + user.name
- smb_files
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.action
  + related.ip
  + related.user
  + source.geo
  + destination.geo
  + user.name
  + file.accessed
  + file.ctime
  + file.created
  + file.mtime
  + file.path
  + file.name
  + file.size
- smb_mapping
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + related.ip
  + source.geo
  + destination.geo
- smtp
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + related.ip
  + source.geo
  + destination.geo
- snmp
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + related.ip
  + source.geo
  + destination.geo
- socks
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.outcome
  + related.ip
  + source.geo
  + destination.geo
  + user.name
  + related.user
- ssh
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + event.outcome
  + related.ip
  + source.geo
  + destination.geo
- ssl
  + event.kind
  + event.category
  + event.type
  + related.ip
- stats
  + convert pipeline to yaml
  + event.kind
- syslog
  + convert pipeline to yaml
  + event.kind
  + related.ip
  + source.geo
  + destination.geo
  + log.syslog.facility.name
  + log.syslog.severity.name
- traceroute
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + related.ip
  + source.geo
  + destination.geo
- tunnel
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + related.ip
  + source.geo
  + destination.geo
- weird
  + convert pipeline to yaml
  + event.kind
  + event.category
  + event.type
  + related.ip
  + source.geo
  + destination.geo
  + rule.name
- x509
  + event.kind
  + event.type

Closes elastic#16029

(cherry picked from commit b552dd8)
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview stats

Expand to view the summary

Build stats

Test stats 🧪

Test Results
Failed 0
Passed 1175
Skipped 124
Total 1299

@andresrc andresrc removed the needs_team Indicates that the issue/PR needs a Team:* label label May 2, 2020
@leehinman leehinman merged commit 878affb into elastic:7.x May 4, 2020
@leehinman leehinman deleted the backport_17738_7.x branch May 4, 2020 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants