Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

default bulk sizes #628

Merged
merged 4 commits into from
Jan 7, 2016
Merged

default bulk sizes #628

merged 4 commits into from
Jan 7, 2016

Commits on Jan 7, 2016

  1. Update default spooler size and logstash max bulk size 

    - set default spooler size in filebeat to 2048, in order to increase number of
  lines to be combined into one bulk request

- set default bulk max size to 2048, to guarantee not to split bigger bulk
  requests into too many small requests.

Tested different spooler and bulk sizes using NASA-HTTP logs:

spooler          bulk  | cpu(%)   mem-res(MB)  throughput(1k lines/s)
---------------------------------------------------------------------
 1024            200   |  26        20              2.1
 1024            1024  |  55        20             15.7
 2048            2048  |  66        25             18.2
 3072            3072  |  66        28             18.5
 4096            4096  |  70        33             18.7
    urso committed Jan 7, 2016
    Configuration menu
    Copy the full SHA
    43914f7 View commit details
    Browse the repository at this point in the history

  2. Update changelog 

    - add changelog entries about spool_size and bulk_max_size defaults
    urso committed Jan 7, 2016
    Configuration menu
    Copy the full SHA
    7ef39f6 View commit details
    Browse the repository at this point in the history

  3. Update spool_size default in docs and sample configs

    urso committed Jan 7, 2016
    Configuration menu
    Copy the full SHA
    97142b4 View commit details
    Browse the repository at this point in the history

  4. Fix config test

    urso committed Jan 7, 2016
    Configuration menu
    Copy the full SHA
    cb6eea1 View commit details
    Browse the repository at this point in the history