Agents re-configure applications to replace plain-text formatters with ECS formatters

[felixbarny]

Users can set a configuration option to make APM Agents automatically re-configure logging for them. After opting into it, agents instrument the construction of log appenders (such as file or stdout appenders) and replace the default plain-text formatter (such as PatternLayout) with the equivalent from the ECS logger that formats the logs to ECS-compliant JSON.

This is an interesting option when manually changing the logging setup of an application is not a trivial task such as when there are long dev release cycles, onboarding logs for legacy applications, or standard off-the-shelf applications.
Users still have to manually configure Fleet/Filebeat to ship the logs.

This is only feasible for agents that support some kind of instrumentation. This is the result of an early feasibility check per agent:

Agent		Comment
.NET core	❓	1-liner code change is needed. Potentially doable with no code changes via runtime IL manipulation with profiler
.NET Framework	❌	1-liner code change needed, Possible for biggest frameworks Serilog and NLog
Go	❌	3-liner code change needed
Java	✅	POC exists
Node.js	✅	seems feasible
PHP	✅	seems feasible
Python	✅	seems feasible
Ruby	✅	seems feasible

Spec issue

• [META 373] SPEC for supporting replacement of plain-text formatters with ECS formatters #403

Agent issues

Extend ECS logging documentation

• [META 373] Extend ECS logging documentation  apm-agent-java#1617
• [META 373] Extend ECS logging documentation  apm-agent-dotnet#1130
• [META 373] Extend ECS logging documentation  apm-agent-go#869
• [META 373] Extend ECS logging documentation  apm-agent-php#321
• [META 373] Extend ECS logging documentation  apm-agent-ruby#931
• [META 373] Extend ECS logging documentation  apm-agent-nodejs#1943
• [META 373] Extend ECS logging documentation  apm-agent-python#1007

Support replacement of plain-text formatters with ECS formatters

• [META 373] Support replacement of plain-text formatters with ECS formatters apm-agent-java#1616
• [META 373] Support replacement of plain-text formatters with ECS formatters apm-agent-dotnet#1129
• [META 373] Support replacement of plain-text formatters with ECS formatters apm-agent-go#868
• [META 373] Support replacement of plain-text formatters with ECS formatters apm-agent-php#320
• [META 373] Support replacement of plain-text formatters with ECS formatters apm-agent-ruby#930
• [META 373] Support replacement of plain-text formatters with ECS formatters apm-agent-nodejs#1942
• [META 373] Support replacement of plain-text formatters with ECS formatters apm-agent-python#1006

[trentm]

Answering tentatively for Node.js, it seems feasible that we should be able to do this. Each of the logging frameworks that we might support (winston, pino, morgan, bunyan) have their own quirks that might make complexity an issue.

[eyalkoren]

See related spec for more details.