Enhance the Beats documentation with more guidance information about configuration

[dedemorton]

I'm creating this meta issue to track topics we need to add the the Beats reference guides to provide more guidance about configuring and using each Beat.

Why? Right now, the doc for each Beat has a long reference topic that describes all the configuration options for the Beat. The topics are overwhelming to read unless you know what you're looking for. The reference information is useful and necessary, but it isn't a good place to explain groups of related configuration options or provide conceptual information + extended examples. My proposal is to build up (over time) a set of topics that describe the most common configuration changes that users need to make.

Here are the topics listed in no particular order. Please add to the list. This list is not limited to config-related topics. Let's try to capture whatever we think is missing.

Filebeat

• Performance Tuning - discuss configuration settings that improve performance, such as increasing throughput, improving memory usage, etc
• Load Balancing - separate topic or part of performance tuning? See related discussion here: https://discuss.elastic.co/t/logstash-output-load-balancing-and-workers-settings/48176/3 and Document load-balancing modes in filebeat #1852 and https://discuss.elastic.co/t/filebeat-load-balancing-failing-because-of-ssl-certificate/52500/4
• Handling Multiline Messages - provide additional examples for dealing with common log formats that have multiline messages - we need something similar to the topic that Logstash provides here: https://www.elastic.co/guide/en/logstash/2.3/multiline.html#_java_stack_traces
• Filtering Exported Lines - maybe not the best title - discuss how to use include_lines and exclude_lines plus provide some real-world examples
• Specifying Multiple Prospectors - Move details of specifying second prosector to separate topic in Filebeat Reference #685 - take the info that is currently in the getting started topic and move it to a separate topic - also explain why you would want to specify more than one prospector
• Discuss when Filebeat can lose log lines. For example when using the tail_files option, or when old files are deleted before Filebeat had the chance to read them.
• Character encoding of source files
• Explain how backpressure from Logstash affects Filebeat (see https://discuss.elastic.co/t/duplicate-documents-using-filebeat/53905). Might be a good topic for the FAQ.

Packetbeat

• Maintaining the Real-Time State of the Network Topology - move this section out of the reference topic and more completely describes the options for setting this up - the reference info will still contain information about the config options.

All Beats

• Debugging - discuss how to set logging options and capture debug info
• Handling Failures - discuss what happens when there's a failure - how to configure retry behavior - (Add a separate topic to describe how events are handled when max_retries is exceeded #652) - explain max_retries and how it relates to other configuration options (such as backoff, max_backoff, and backoff_factor)
• Working with Firewalls - discuss opening ports - not sure where this belongs or if it's a common enough stumbling block that it's worth documenting outside of the discussion forum
• Regular Expression Support - this could probably live in libbeat since it's pure reference and relates to all the beats
• FAQ - Create an FAQ for resolving common issues #699 - this will be a set of questions and answers. Each Beat should have its own FAQ.
• Configuring logging output

[tsg]

I like the idea, it would be like our own "Definitive Guide" :-). The performance tuning topic in Filebeat might be too much of a moving target at the moment. +1 for all the other topics.

The filtering one will be more interesting after this is done.

[tsg]

Added a point about safety semantics in Filebeat, follow up from a slack conversation.

[dedemorton]

Closing and moving unresolved issues to #2482