new function that generates junit xml for checks #32
Conversation
There was a problem hiding this comment.
Thanks for this! Reviewing changes in the current jslib format is a bit unwieldy, so we're probably going to make a separate repo for the summary library, similar to how we have one for the httpx one (https://github.com/k6io/k6-jslib-httpx) soon and we'd just use this repo for the distributions.
| } | ||
| return ( | ||
| `<?xml version="1.0" encoding="UTF-8" ?>\n<testsuites tests="${cases.length}" ` + | ||
| `failures="${failures}\" time="${time}">\n` + |
There was a problem hiding this comment.
Hmm this seems wrong. According to https://llg.cubic.org/docs/junit/, this should be
Time taken (in seconds) to execute the tests in the suite
We don't have that data readily available in handleSummary() yet, though we could easily add it (e.g. in grafana/k6#1975).
There was a problem hiding this comment.
i did not find anything closer than iteration_duration in the summary.json to find the time taken to execute the tests
There was a problem hiding this comment.
Yeah... 😞 We can correct that in k6 v0.33.0, since I imagine it will be useful for other things as well. I added a note about it in grafana/k6#1975 (comment)
the old logic did not work on batch requests with checks on all the responses.
|
Sorry for leaving you hanging for a while here... 😅 Here are some updates! Yesterday we merged this PR in k6: grafana/k6#1975. Among a lot of other changes, it adds I already mentioned that in my previous comment, but we'll probably also going to make a separate GitHub repo for the JS summary functions, similar to how we have one for the httpx one (https://github.com/k6io/k6-jslib-httpx) before we release k6 v0.33.0. Then we'd ask you to make the PR there and we'd just use this repo for the versioned distribution. That way PRs will be much easier to review. |
|
|
Background:
k6-to-junitwas used to generate junit XML files for the checks with names matching the regex. Refer comment. k6 now has the support of generating junit XML report from version v0.30.0.Issue:
For functional automated tests using k6, it would be very helpful to report each check in the test script, as a test case in a junit format enabling reporting in the CI build pipelines. And existing jUnit does not solve the purpose of having a check as a test case.
Solution:
added a new version of
k6-summaryexporting a new function:jUnitForCheckswhich returns a string in the format of junit xml string where each passed check is considered as a passed test case and failed check as a failed test case.Note: this is considered only when using for automated functional regression test where the k6 test script is run with 1 user and 1 iteration.
Note: the name of the new function can be discussed 😄 and i have not added tests 😉 since there was no existing tests for jUnit.
Think this will be helpful for many and moving one step towards k6's goal of using it across different types of tests. 😎