Failsafe TestHTMLReporter parameter.toString

[seregamorph]

TestNG Version

7.6.2-SNAPSHOT

Expected behavior

Failsafe TestHTMLReporter when parameter.toString() of test execution throws RuntimeException.

Actual behavior

TestHTMLReporter does not fail if test parameter toString method throws Exception. Sequentially it leads to maven build failure with non-clear diagnostics (missing full stacktrace, only message) like

[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
...
build	10-Nov-2022 05:44:17	[INFO] ------------------------------------------------------------------------
build	10-Nov-2022 05:44:17	[INFO] BUILD FAILURE
build	10-Nov-2022 05:44:17	[INFO] ------------------------------------------------------------------------
build	10-Nov-2022 05:44:17	[INFO] Total time:  10:19 min (Wall Clock)
build	10-Nov-2022 05:44:17	[INFO] Finished at: 2022-11-10T05:44:17Z
build	10-Nov-2022 05:44:17	[INFO] ------------------------------------------------------------------------
build	10-Nov-2022 05:44:17	[ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.22.2:verify (default-cli) on project users-integration-test: There are test failures.
build	10-Nov-2022 05:44:17	[ERROR] 
build	10-Nov-2022 05:44:17	[ERROR] Please refer to /home/bamboo/bamboo-agent-home/xml-data/build-dir/RTB-NRB22926-JOB10/server/general/users-integration-test/target/failsafe-reports for the individual test results.
build	10-Nov-2022 05:44:17	[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
build	10-Nov-2022 05:44:17	[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process
build	10-Nov-2022 05:44:17	[ERROR] game_type_id=10000006, model=Model [class=projects.pt.server.users.models.userid.UserIdModel, name=UserId, key=-4268742994815057915:UserId, id=145, serverOnly=true], field=PersistentFieldPersistentModelField[UserIdEntity UserIdModel.userIdEntity]
build	10-Nov-2022 05:44:17	[ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.fork(ForkStarter.java:656)
build	10-Nov-2022 05:44:17	[ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:282)
build	10-Nov-2022 05:44:17	[ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:245)
build	10-Nov-2022 05:44:17	[ERROR] 	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1183)
build	10-Nov-2022 05:44:17	[ERROR] 	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1011)
build	10-Nov-2022 05:44:17	[ERROR] 	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:857)
build	10-Nov-2022 05:44:17	[ERROR] 	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
build	10-Nov-2022 05:44:17	[ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
build	10-Nov-2022 05:44:17	[ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
build	10-Nov-2022 05:44:17	[ERROR] 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
build	10-Nov-2022 05:44:17	[ERROR] 	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
build	10-Nov-2022 05:44:17	[ERROR] 	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:190)
build	10-Nov-2022 05:44:17	[ERROR] 	at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:186)
build	10-Nov-2022 05:44:17	[ERROR] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
build	10-Nov-2022 05:44:17	[ERROR] 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
build	10-Nov-2022 05:44:17	[ERROR] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
build	10-Nov-2022 05:44:17	[ERROR] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
build	10-Nov-2022 05:44:17	[ERROR] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
build	10-Nov-2022 05:44:17	[ERROR] 	at java.base/java.lang.Thread.run(Thread.java:833)
build	10-Nov-2022 05:44:17	[ERROR] -> [Help 1]
build	10-Nov-2022 05:44:17	org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.22.2:verify (default-cli) on project users-integration-test: There are test failures.

The good thing is that failure can be found in target/failsafe-reports/*-jvmRun*.dump and there is stacktrace. But if test suite is executed on the CI/CD it can be problem to obtain it.
How it can happen: objects that are parameters of tests may have stateful toString() method (actually antipattern, but failsafe still makes sense to handle such failures).

Is the issue reproducible on runner?

• Shell
• Maven (with HTML reporting)
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

Test case sample

The PR with the proposed fix #2831

Contribution guidelines

Incase you plan to raise a pull request to fix this issue, please make sure you refer our Contributing section for detailed set of steps.