Got SecurityException when running JUnit 5 tests

[jbelford]

I've had this issue for a while. Essentially, if I open up a project folder directly I can run JUnit5 tests.

If I am in a multi-root workspace it will skip JUnit5 tests when I try to run them. JUnit4 tests seem to still work fine though.

Here's the log

.
[ERROR] @@<TestRunner-{"name":"error", "attributes":{"message":"Failed to run Junit tests", "details":
"java.lang.SecurityException: class \"org.junit.platform.engine.discovery.ModuleSelector\"'s signer information does not match signer information of other classes in the same package
  at java.lang.ClassLoader.checkCerts(ClassLoader.java:898)
  at java.lang.ClassLoader.preDefineClass(ClassLoader.java:668)
  at java.lang.ClassLoader.defineClass(ClassLoader.java:761)
  at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
  at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
  at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
  at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
  at java.security.AccessController.doPrivileged(Native Method)
  at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  at com.microsoft.java.test.runner.junit5.DiscoveryRequestCreator.createExplicitDiscoverySelectors(DiscoveryRequestCreator.java:83)
  at com.microsoft.java.test.runner.junit5.DiscoveryRequestCreator.createDiscoverySelectors(DiscoveryRequestCreator.java:61)
  at com.microsoft.java.test.runner.junit5.DiscoveryRequestCreator.toDiscoveryRequest(DiscoveryRequestCreator.java:44)
  at com.microsoft.java.test.runner.junit5.CustomizedConsoleTestExecutor.lambda$executeTests$0(CustomizedConsoleTestExecutor.java:41)
  at com.microsoft.java.test.runner.junit5.CustomContextClassLoaderExecutor.invoke(CustomContextClassLoaderExecutor.java:29)
  at com.microsoft.java.test.runner.junit5.CustomizedConsoleTestExecutor.executeTests(CustomizedConsoleTestExecutor.java:38)
  at com.microsoft.java.test.runner.junit5.CustomizedConsoleLauncher.execute(CustomizedConsoleLauncher.java:29)
  at com.microsoft.java.test.runner.Launcher.main(Launcher.java:53)\n"}}-TestRunner>.
[ERROR] 
.

Using extension version 0.11.1 and vscode 1.29.1

[jbelford]

I did a quick look into the code and the problem seemed to be from trying to add a selector based on the Module in DiscoveryRequestCreator.java.

options.getSelectedModules().stream().map(DiscoverySelectors::selectModule).forEach(selectors::add);

I removed this selector and tested it out and everything seems to work fine now.

[jdneo]

@jbelford Thank you for your contribution.

I tried to use a multi-root workspace which has JUnit 5 projects but cannot repro this issue. Could you please describe more about your workspaces? Or it will be great if you are willing to share a sample to repro.

[jbelford]

Interesting. I think it has something to do with large multi-root workspaces then. I've got a 40+ multi-root workspace composed of various maven projects. It might be a bug with vscode handling that many roots in a workspace.

I also double checked again and I still see this error happening prior to the change but not after.

OS: macOS High Sierra v10.13.3

[jdneo]

I see, worth investigating more on this. The original code is copied from here

Do you have any idea why removing this line will make it work?

[jbelford]

I will need to look into this more. I do not know why it doesn't work for large workspaces - so far only that selecting modules causes issue and doesn't seem to be needed.

[jdneo]

@jbelford Much appreciate for your effort! 😄

[jbelford]

Small update to this problem - it turns out that multi-root workspaces are likely not the problem here.

The tests I am seeing getting skipped and run into errors are when a maven project is part of a multi-maven project.

[jdneo]

@jbelford Does this problem still occur in the newest Test Runner extension?

[jbelford]

@jdneo Unfortunately this is still present 😕

[jdneo]

@keigezellig Did you meet the same problem as @jbelford's?

[keigezellig]

@jdneo Well, not entirely, it is the same error message, but in my case it occurs like i mentioned in #208 when i try to run any JUnit 5 test with the test runner.

[jdneo]

@jbelford I'm suspecting that the problem is caused because the jars are signed before publishing. See: https://stackoverflow.com/questions/2877262/java-securityexception-signer-information-does-not-match

Let me take a look.

[jdneo]

@keigezellig @jbelford Would be very appreciated if you can provide any sample project that can help us to repro this issue.

[jdneo]

So after some investigation, I think I might find out the root cause of this issue:

Now we are using the assembly plugin to generate a fat jar and sign the fat jar, this will break the sign information of the dependencies.

The recommended way is to use jar-plugin to package the executable jar and sign it. Meanwhile, use dependency-plugin to put the dependencies into the lib folder

[jbelford]

Thanks for taking a look :D I'll try changing the plugins and testing on my local to see if the issue is present. Nvm looks like you have a branch. Let me know if you want me to test.

[jdneo]

Thank you for your help @jbelford.

I'll check if it is possible to provide a private signed-build to you 😃

[jdneo]

Hi @jbelford, could you please find me in the gitter channel, I'll give you a private build there. Thank you.