[Metricbeat] Add reporting interface with error

[ruflin]

In Metricbeat modules we often see the pattern that before fetching the actual data, some processing is done and checks for errors happen. If an error happened, we have 3 lines of code:

• creating the error with wrap
• reporting the error
• logging the error

By introducing the reporter interface with support for return an error I would like to eliminate the overhead and allow to directly return and error which is then reported and also logged.

So far we logged the error on the Error level. I'm now wondering if we should log these errors actually on the Info level as it's normally not a misbehaving of the Beat but the service does not respond as expected. So for the operator of the Beat normally no actions are needed.

As an example metricset I took php_fpm.process.

[cachedout]

I like this a lot! I was just wondering about this last night so I am very happy to see it.

[ycombinator]

So far we logged the error on the Error level. I'm now wondering if we should log these errors actually on the Info level as it's normally not a misbehaving of the Beat but the service does not respond as expected. So for the operator of the Beat normally no actions are needed.

I'm still in favor of logging errors at the ERROR level. From a debugging perspective, when looking at the Metricbeat logs, it makes it really easy to spot errors. Then, from the error, we can figure out if it's a service error or a beat error. My concern with dropping service errors down to INFO level is that they'll get lost in the log noise pretty easily, making debugging harder.

[kaiyan-sheng]

Maybe adding a log level as input parameter so we can set to ERROR or INFO depends on the each specific case?

[sayden]

Nice idea!

[ruflin]

For the logging level, have a look at my longer comment here: #10727 (comment) I think error logging should be focused on the production and not the setup / testing use case.

[jsoriano]

I'd try to avoid adding a new fetcher interface, but as we can still reuse ReporterV2 and it is so easy to implement from the ReportingMetricSetV2 one I am not going to complain 🙂