Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Filebeat][Cisco Module] Adding various smaller hotfixes related to github issues #20565

Merged
merged 6 commits into from
Aug 25, 2020

Conversation

P1llus
Copy link
Member

@P1llus P1llus commented Aug 11, 2020

What does this PR do?

This PR is a try to resolve some commonly reported issues and requests, together with a few small hotfixes, new testlog cases to catch these issues and generated new golden files for existing logs.

Overview of the changes:

  • Copies observer related information into the different observer fields but leaves the original fields for now to not break anything.
  • Fixes conditions around source+destination nat mapping, since a port can be nat'ed without a new IP for example.
  • Apply fixes for various github issues, see related issues below.
  • Apply support for new message_ids related to various github issues, see related issues below.
  • Move certain dissect to grok to support various message formats.

Why is it important?

Resolves variously reported issues about the ingest pipelines and adds support for more message fields.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

@P1llus P1llus added enhancement in progress Pull request is currently in progress. Filebeat Filebeat Team:SIEM labels Aug 11, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Aug 11, 2020
@P1llus P1llus marked this pull request as draft August 11, 2020 21:22
@elasticmachine
Copy link
Collaborator

elasticmachine commented Aug 11, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #20565 updated]

  • Start Time: 2020-08-20T05:16:46.465+0000

  • Duration: 51 min 15 sec

Test stats 🧪

Test Results
Failed 0
Passed 5558
Skipped 822
Total 6380

@P1llus
Copy link
Member Author

P1llus commented Aug 11, 2020

Still needs a few small fixes and tests before totally ready for a review, but any comments appreciated.

@P1llus P1llus marked this pull request as ready for review August 12, 2020 02:40
@P1llus P1llus added review and removed in progress Pull request is currently in progress. labels Aug 12, 2020
@P1llus
Copy link
Member Author

P1llus commented Aug 12, 2020

PR is now ready for Review.

A few pointers that might come up:

  • The reason I added some new Teardown messages outside the existing grok is because there is even more possible combinations for it, and in the end the grok would be unreadable.
  • Plenty of new fields that can be mapped to fields.yml, though currently holding that off due to the fields.yml size limit issue.
  • Event.outcome is currently staying and will be resolved in a follow-up PR.

@P1llus P1llus requested a review from andrewstucki August 12, 2020 08:48
@P1llus P1llus requested a review from adriansr August 23, 2020 09:14
Copy link
Contributor

@adriansr adriansr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@marc-gr marc-gr merged commit 3f025e1 into elastic:master Aug 25, 2020
marc-gr pushed a commit to marc-gr/beats that referenced this pull request Aug 25, 2020
…ithub issues (elastic#20565)

* applying fixes to existing message_ids, adding support for new message_ids, fixing nat mapping and a few more

* adding the last missing fields

* updating changelog

* mage fmt update

* Updating test data to be a bit more realistic instead of just localhost

(cherry picked from commit 3f025e1)
@marc-gr marc-gr added the v7.9.1 label Aug 25, 2020
@P1llus P1llus deleted the filebeat_cisco_new_message_patterns branch August 25, 2020 09:08
marc-gr pushed a commit to marc-gr/beats that referenced this pull request Aug 27, 2020
…ithub issues (elastic#20565)

* applying fixes to existing message_ids, adding support for new message_ids, fixing nat mapping and a few more

* adding the last missing fields

* updating changelog

* mage fmt update

* Updating test data to be a bit more realistic instead of just localhost

(cherry picked from commit 3f025e1)
andrewkroh pushed a commit that referenced this pull request Aug 27, 2020
…ithub issues (#20565) (#20770)

* applying fixes to existing message_ids, adding support for new message_ids, fixing nat mapping and a few more

* adding the last missing fields

* updating changelog

* mage fmt update

* Updating test data to be a bit more realistic instead of just localhost

(cherry picked from commit 3f025e1)

Co-authored-by: Marius Iversen <pillus@chasenet.org>
marc-gr added a commit that referenced this pull request Aug 31, 2020
…ithub issues (#20565) (#20767)

* applying fixes to existing message_ids, adding support for new message_ids, fixing nat mapping and a few more

* adding the last missing fields

* updating changelog

* mage fmt update

* Updating test data to be a bit more realistic instead of just localhost

(cherry picked from commit 3f025e1)

Co-authored-by: Marius Iversen <pillus@chasenet.org>
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020
…ithub issues (elastic#20565)

* applying fixes to existing message_ids, adding support for new message_ids, fixing nat mapping and a few more

* adding the last missing fields

* updating changelog

* mage fmt update

* Updating test data to be a bit more realistic instead of just localhost
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
…ithub issues (elastic#20565) (elastic#20770)

* applying fixes to existing message_ids, adding support for new message_ids, fixing nat mapping and a few more

* adding the last missing fields

* updating changelog

* mage fmt update

* Updating test data to be a bit more realistic instead of just localhost

(cherry picked from commit 8739dae)

Co-authored-by: Marius Iversen <pillus@chasenet.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment