Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry-pick #19552 to 6.8: Add max bytes in line limit #20087

Merged
merged 1 commit into from
Jul 23, 2020

Conversation

kvch
Copy link
Contributor

@kvch kvch commented Jul 21, 2020

Cherry-pick of PR #19552 to 6.8 branch. Original message:

What does this PR do?

Handles OOM on long lines gracefully, writes the error into log.

Why is it important?

Handle bad data gracefully without OOM.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have added tests that prove my fix is effective or that my feature works

Author's Checklist

  • I just recently started looking at the beats code so might not have enough knowledge of any possible side-effects of this change. Please review with that in mind.

How to test this PR locally

Refer to #19500 for test case.
The filebeat doesn't get killed due to OOM with this change and the error is logged (see the screenshot attached).

Related issues

Screenshots

Screen Shot 2020-07-01 at 9 16 06 AM

Logs

2020-07-01T13:15:17.630Z	INFO	[crawler]	beater/crawler.go:108	Loading and starting Inputs completed. Enabled inputs: 1
2020-07-01T13:15:17.633Z	INFO	log/harvester.go:297	Harvester started for file: /test/readme.log
2020-07-01T13:15:17.635Z	ERROR	log/harvester.go:330	Read line error: exceeded max bytes in line limit; File: /test/readme.log

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 21, 2020
@kvch kvch force-pushed the backport_19552_6.8 branch from 2dfdfe9 to 96df3d4 Compare July 21, 2020 10:34
@kvch kvch added [zube]: In Review Team:Services (Deprecated) Label for the former Integrations-Services team and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jul 21, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

@urso
Copy link

urso commented Jul 21, 2020

Please check the lint output. Looks like Filebeat does not compile with the backport.

@kvch kvch force-pushed the backport_19552_6.8 branch from 96df3d4 to 25b3ebf Compare July 22, 2020 15:58
@kvch
Copy link
Contributor Author

kvch commented Jul 23, 2020

Failing tests are unrelated.

@kvch kvch added the v6.8.12 label Jul 23, 2020
@kvch kvch merged commit f431ae6 into elastic:6.8 Jul 23, 2020
@zube zube bot removed the [zube]: Done label Oct 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport review Team:Services (Deprecated) Label for the former Integrations-Services team v6.8.12
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants