Split logback logs into multiple subloggers | Implement a LogsServer

[ivangsa]

Hi

we are creating a lot of karate for API E2E testing, and we would like to stop maintaining Postman collections and leverage the same karate tests for both automation and manual debugging/developing, and debug http request/responses is hard just by reading karate logs, we usually end up scrolling up and down, copy/pasting payloads into editors, format json...

so we have developed a quick poor-man-solution extending @kirksl vscode extension and sending karate logs using this logback JSON+SocketAppender https://github.com/logstash/logstash-logback-encoder

https://github.com/ivangsa/karate-runner/tree/features/KarateNetworkLogsProvider
https://github.com/ivangsa/karate-runner/blob/features/KarateNetworkLogsProvider/resources/KarateNetworkLogs-Screenshoot.png

it's simple and it works nice

We would like to further explore this approach, it would be great if logs could be split into multiple logback sub-loggers and levels so filtering could be fine tuned

For instance we would like to receive logs when scenarios start (with its __args) and scenarios exit with context and status (ok/ko) so we can build and execution tree along http logs...

We thought about a TraceServerProtocol like https://github.com/theia-ide/trace-server-protocol may be overkilling, and this socket appender seems to work

We would like to explore with you @ptrthomas and @kirksl this approach..

What do you think about splitting logs into multiple sub-loggers?

Also, in vscode at this moment debug variables are showing as text (instead of object) very hard to debug..
kirksl/karate-runner#68

[ptrthomas]

@ivangsa ok thanks for this. I'd really like to build as much of these generic needs into karate. just that next 1-2 months I'm busy wrapping up the 1.0 - but I'll do my best to roll some of these in

can you take a look at this "hook", it emits logs in a special format that IntelliJ understands: https://github.com/intuit/karate/blob/develop/karate-core/src/main/java/com/intuit/karate/cli/IntellijHook.java

even better I think all the things you need may be available in the ScenarioResult object, so try it once

point 2: I'm actually seriously considering extending the HTML report so that you can export a cURL command at the very least - or even submit the request via the browser swagger style, I'll be taking a shot at this over the year-end so do wait for that and then I'll request feedback

[ptrthomas]

closing for now, but the discussion can continue. my opinion is that if the HTML report gives you

a) an easy way to cut and paste requests and responses
b) as a bonus - an easy way to make the request via cURL or a similar "swagger style" mechanism

there is no need for anything else. teams can of course implement hooks for any specific custom needs

[ivangsa]

Ok, using ExcecutionHook I think we can send all information we need to the IDE at runtime

I see that we are thinking of two different use cases:

One is analysing results, once the tests are up and running on the pipeline, and something goes wrong then a good report with all the information it's necesary to track down what was wrong (the environment, the apis, the test itself...)

A different use case is the local environment/IDE when starting coding and it still doesn't work or it seems to work but it doesn't... then copy-pasting pretty printed strings is suboptimal, we need something more productive, closer to chrome devtools or postman..

In any case, I think separating logs into different subloggers (com.intuit.karate.network, com.intuit.karate.print, com.intuit.karate.core,...) would be usefull in general..

[ptrthomas]

I think separating logs into different subloggers (com.intuit.karate.network, com.intuit.karate.print, com.intuit.karate.core,...) would be usefull in general..

agree. I don't see this as urgent though. I hope we can leave LogsServer out of this discussion.

for the "something more productive" part, I'd like some clear examples. and yes I see that as the responsibility of the IDE plugin, not the core framework. of course if there is something that the core framework should do to enable anything the IDE + developer experience needs, we can discuss that