Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object.getClass()" because "receiver" is null #24167

Merged
merged 2 commits into from
Feb 22, 2021
Merged

[Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object.getClass()" because "receiver" is null #24167

merged 2 commits into from
Feb 22, 2021

Conversation

adriansr
Copy link
Contributor

@adriansr adriansr commented Feb 22, 2021 •
edited
Loading

What does this PR do?

Fixes Filebeat's aws/vpcflow ingest pipeline to avoid painless exceptions when aws.vpcflow fields are missing in the ingested document. This happens for empty lines in text logs, or any lines that don't conform to the expected format.

Why is it important?

The fileset was generating documents with the following error.message:

Cannot invoke "Object.getClass()" because "receiver" is null

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@adriansr
Fix errors from aws/vpcflow ingest pipeline
b3ff82a 
The pipeline failed with an obscure error.message:
`Cannot invoke "Object.getClass()" because "receiver" is null`
when the ingested message didn't contain `aws.vpcflow.*` fiels.

Filebeat generates documents that lack those fields when parsing
lines from a .log file (not .json) that doesn't conform to the expected
formats. This happened for empty lines in particular.
@adriansr
changelog
bc00e65
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Feb 22, 2021
@adriansr adriansr added bug needs_backport PR is waiting to be backported to other branches. review labels Feb 22, 2021
@adriansr adriansr changed the title Fix aws vpcflow pipeline errors [Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object.getClass()" because "receiver" is null Feb 22, 2021
kaiyan-sheng
kaiyan-sheng approved these changes Feb 22, 2021
View reviewed changes
Copy link
Contributor

@kaiyan-sheng kaiyan-sheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #24167 opened

  • Start Time: 2021-02-22T20:23:26.561+0000

  • Duration: 50 min 15 sec

  • Commit: bc00e65

Test stats 🧪

Test Results
Failed 0
Passed 6841
Skipped 1153
Total 7994

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 6841
Skipped 1153
Total 7994

@adriansr adriansr merged commit f4b7a25 into elastic:master Feb 22, 2021
adriansr added a commit to adriansr/beats that referenced this pull request Feb 22, 2021
@adriansr
[Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object…
0e16c53 
….getClass()" because "receiver" is null (elastic#24167)

The pipeline failed with an obscure error.message:
`Cannot invoke "Object.getClass()" because "receiver" is null`
when the ingested message didn't contain `aws.vpcflow.*` fiels.

Filebeat generates documents that lack those fields when parsing
lines from a .log file (not .json) that doesn't conform to the expected
formats. This happened for empty lines in particular.

(cherry picked from commit f4b7a25)
@adriansr adriansr mentioned this pull request Feb 22, 2021
Merged
4 tasks
@adriansr adriansr added v7.13.0 and removed needs_backport PR is waiting to be backported to other branches. labels Feb 22, 2021
adriansr added a commit to adriansr/beats that referenced this pull request Feb 22, 2021
@adriansr
[Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object…
3706d9e 
….getClass()" because "receiver" is null (elastic#24167)

The pipeline failed with an obscure error.message:
`Cannot invoke "Object.getClass()" because "receiver" is null`
when the ingested message didn't contain `aws.vpcflow.*` fiels.

Filebeat generates documents that lack those fields when parsing
lines from a .log file (not .json) that doesn't conform to the expected
formats. This happened for empty lines in particular.

(cherry picked from commit f4b7a25)
@adriansr adriansr mentioned this pull request Feb 22, 2021
Merged
4 tasks
@adriansr adriansr mentioned this pull request Feb 22, 2021
Merged
4 tasks
adriansr added a commit to adriansr/beats that referenced this pull request Feb 22, 2021
@adriansr
[Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object…
4dab85c 
….getClass()" because "receiver" is null (elastic#24167)

The pipeline failed with an obscure error.message:
`Cannot invoke "Object.getClass()" because "receiver" is null`
when the ingested message didn't contain `aws.vpcflow.*` fiels.

Filebeat generates documents that lack those fields when parsing
lines from a .log file (not .json) that doesn't conform to the expected
formats. This happened for empty lines in particular.

(cherry picked from commit f4b7a25)
adriansr added a commit that referenced this pull request Feb 23, 2021
@adriansr
[Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object…
f45f32e 
….getClass()" because "receiver" is null (#24167) (#24170)

The pipeline failed with an obscure error.message:
`Cannot invoke "Object.getClass()" because "receiver" is null`
when the ingested message didn't contain `aws.vpcflow.*` fiels.

Filebeat generates documents that lack those fields when parsing
lines from a .log file (not .json) that doesn't conform to the expected
formats. This happened for empty lines in particular.

(cherry picked from commit f4b7a25)
adriansr added a commit that referenced this pull request Feb 23, 2021
@adriansr
[Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object…
5e1031d 
….getClass()" because "receiver" is null (#24167) (#24169)

The pipeline failed with an obscure error.message:
`Cannot invoke "Object.getClass()" because "receiver" is null`
when the ingested message didn't contain `aws.vpcflow.*` fiels.

Filebeat generates documents that lack those fields when parsing
lines from a .log file (not .json) that doesn't conform to the expected
formats. This happened for empty lines in particular.

(cherry picked from commit f4b7a25)
adriansr added a commit that referenced this pull request Feb 23, 2021
@adriansr
[Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object…
7305b35 
….getClass()" because "receiver" is null (#24167) (#24168)

The pipeline failed with an obscure error.message:
`Cannot invoke "Object.getClass()" because "receiver" is null`
when the ingested message didn't contain `aws.vpcflow.*` fiels.

Filebeat generates documents that lack those fields when parsing
lines from a .log file (not .json) that doesn't conform to the expected
formats. This happened for empty lines in particular.

(cherry picked from commit f4b7a25)
v1v added a commit to v1v/beats that referenced this pull request Feb 23, 2021
@v1v
Merge remote-tracking branch 'upstream/master' into feature/retry-win…
3ea53ee 
…dows-7

* upstream/master:
  Remove OSS reference for kibana and elasticsearch (elastic#24164)
  Skip flaky TestActions on MacOSx (elastic#23966)
  [Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object.getClass()" because "receiver" is null (elastic#24167)
  [Elastic Agent] Fix docker entrypoint for elastic-agent. (elastic#24155)
  [PACKAGING] Push docker images with the architecture in the version (elastic#24121)
  [Agent] Add agent standalone manifests for system module & Pod's log collection (elastic#23938)
  indicator type url is in upper case (elastic#24152)
  [Filebeat] Document netflow internal_networks and set default (elastic#24110)
  [Filebeat] Adding fixes to the TI module (elastic#24133)
  [Enhancement] Add RotateOnStartup feature flag for file output (elastic#19347)
  [Ingest Manager] Fix: Successfully installed and enrolled agent running standalone (elastic#24128)
  Set Elastic licence type for APM server Beats update job (elastic#24122)
  Add logrotation section on Running Filebeat on k8s (elastic#24120)
  [CI] Run if manual UI (elastic#24116)
  [CI] enable x-pack/heartbeat in the CI (elastic#23873)
  chore: comment out the E2E (elastic#24109)
  chore: add-backport-next (elastic#24098)
  Adjust the position of the architecture name in Dockerlogbeat tarball (elastic#24095)
  Update dependencies for M1 support in System (elastic#24019)
v1v added a commit to v1v/beats that referenced this pull request Feb 23, 2021
@v1v
Merge remote-tracking branch 'upstream/master' into feature/packaging…
2e57ac2 
…-arm

* upstream/master: (24 commits)
  Add example input autodsicover config (elastic#24157)
  Empty configuration options generate `<no value>` string for azure-eventhub input (elastic#24156)
  Remove OSS reference for kibana and elasticsearch (elastic#24164)
  Skip flaky TestActions on MacOSx (elastic#23966)
  [Filebeat][AWS] Fix vpcflow pipeline exception: Cannot invoke "Object.getClass()" because "receiver" is null (elastic#24167)
  [Elastic Agent] Fix docker entrypoint for elastic-agent. (elastic#24155)
  [PACKAGING] Push docker images with the architecture in the version (elastic#24121)
  [Agent] Add agent standalone manifests for system module & Pod's log collection (elastic#23938)
  indicator type url is in upper case (elastic#24152)
  [Filebeat] Document netflow internal_networks and set default (elastic#24110)
  [Filebeat] Adding fixes to the TI module (elastic#24133)
  [Enhancement] Add RotateOnStartup feature flag for file output (elastic#19347)
  [Ingest Manager] Fix: Successfully installed and enrolled agent running standalone (elastic#24128)
  Set Elastic licence type for APM server Beats update job (elastic#24122)
  Add logrotation section on Running Filebeat on k8s (elastic#24120)
  [CI] Run if manual UI (elastic#24116)
  [CI] enable x-pack/heartbeat in the CI (elastic#23873)
  chore: comment out the E2E (elastic#24109)
  chore: add-backport-next (elastic#24098)
  Adjust the position of the architecture name in Dockerlogbeat tarball (elastic#24095)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kaiyan-sheng kaiyan-sheng kaiyan-sheng approved these changes

Assignees
No one assigned
Labels
bug review v7.11.2 v7.12.0 v7.13.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@adriansr @elasticmachine @kaiyan-sheng