logparser support mixed parser decoding

[ghost]

We are trying to solve a very specific problem, but this may be generally useful.

logparser should support decoding JSON from the body of a log message.

There are two ways this can occur, one is that the JSON is in a syslog style "field", for example:
<38>2013-07-22T13:45:54 localhost prg00000[1234]: {"MESSAGE": "foo", "PROGRAM": "bar"}

The other is that the entire log is just one JSON item per line. In both cases, what we would want here is that selected JSON fields [including possibly dicts in dicts and so on] are mapped to tags or metrics in telegraf (if they exist in that row).

The only sane way that I can think of to handle #1 is a regular expression to tell the plugin what to attempt to JSON decode. #2 is obviously trivial to deal with, although less common (it happens to be what we are dealing with though).

If we submitted a patch to achieve this, would that work? Is there another approach that i'm missing?

[danielnelson]

For the line separated JSON, you might be able to use the tail plugin which uses the normal parsers. For the first one, I think an option to strip a prefix could be useful, whatever remains would be passed to the parsers.

Long term, tail and logparser should probably be merged and grok be made into one of the parsers. There are some unfortunate ways the config is processed that make this difficult but perhaps we can solve them with the work to move to a plugin based configuration.

[ghost]

We ended up writing a custom plugin to solve our problem before it gets to a log file, which is probably the right way to handle this - but long term I can see a lot of uses for more generic log parsing. Your suggestion makes total sense to me.

I'm not sure if there is already an issue to do what you propose -feel free to close this if its not helpful or rename it!

[danielnelson]

I think what we need here is basically grok to process the line, and then hand the output over to one of the parsers. Perhaps grok could specify a parser as a conversion? I think the syslog case could be pretty common and we likely need the timestamp out of it.

[danielnelson]

The solution we are planning for this issue is to create a parser processor (#4428). The initial parse would use tail+grok or the syslog input, placing the JSON data into a field. Then the parser processor could use the json parser to pull out the individual fields.

[danielnelson]

In 1.8 this can be accomplished using syslog -> parser processor(json) or from a logfile using tail(grok) -> parser processor(json).