Forward stderr from executed scripts

[hahnjo]

Proposal:

The exec plugin should forward any output of stderr, especially in the case of errors.

Current behavior:

[[inputs.exec]]
  commands = [ "./test.sh" ]

#!/bin/sh

echo "Error message" >&2
exit 1

$ ./telegraf -config test.conf
[...]
2018-03-02T15:57:20Z E! Error in plugin [inputs.exec]: exec: exit status 1 for command './test.sh'
2018-03-02T15:57:30Z E! Error in plugin [inputs.exec]: exec: exit status 1 for command './test.sh'

Desired behavior:

$ ./telegraf -config test.conf
[...]
2018-03-02T15:57:20Z E! Error in plugin [inputs.exec]: exec: exit status 1 for command './test.sh'
2018-03-02T15:57:20Z E! './test.sh': Error message
2018-03-02T15:57:30Z E! Error in plugin [inputs.exec]: exec: exit status 1 for command './test.sh'
2018-03-02T15:57:30Z E! './test.sh': Error message

Use case: [Why is this important (helps with prioritizing requests)]

Probably a lot. In my case, I have a small C executable that does some non-trivial initialization work that can fail for quite some different reasons. Getting error messages on the log would speed up finding the problem and even allow pathological analysis (why didn't we get data at that time?).

[hahnjo]

I might be able to contribute this feature but I first wanted to discuss if it's worth it. I'm not sure either if it's just as simple as adding the output in the exec plugin...

[danielnelson]

I think this is a good idea, but we need to make sure that we gracefully handle the case where there is lots of output to stderr. Probably just need a limit of something like 512 chars and make sure the output is utf-8 encoded. I would also add it to the exiting error to it's one line:

E! Error in plugin [inputs.exec]: exec: exit status 1 for command './test.sh': error message

[hahnjo]

Okay, so only in the case of a non-zero return value and not for warnings when the command did gather some data (but possibly not all)?

Your suggestion also implies that we should only display one line so I think this limit makes more sense. What do you think?

[danielnelson]

Yeah, I think we should only report on error exit status since I'm sure many are currently using stderr for debugging messages.