Test discovery failing with NullPointerException when trying to find methods of a test class

[pshevche]

Context

We observe test discovery failing with a NullPointerException when trying to find test class methods (stacktrace is attached below). This seems to be a regression in 1.11.x version of junit-platform-commons. Discovery succeeds when using 1.10.x versions. Apparently, the .getPackage() in this method can yield null:

junit5/junit-platform-commons/src/main/java/org/junit/platform/commons/util/ReflectionUtils.java

Lines 1905 to 1907 in 0de9d10

	private static boolean declaredInSamePackage(Method m1, Method m2) {
	return m1.getDeclaringClass().getPackage().getName().equals(m2.getDeclaringClass().getPackage().getName());
	}

According to the docs this can happen if the package object is not created by the classloader. I have not been able to create a standalone reproducer yet, but I'll keep trying :) So far, we've seen this happen only when running Spock 1.3 tests located in the root package.

Stacktrace

Caused by: java.lang.NullPointerException
	at org.junit.platform.commons.util.ReflectionUtils.declaredInSamePackage(ReflectionUtils.java:1902)
	at org.junit.platform.commons.util.ReflectionUtils.isMethodOverriddenBy(ReflectionUtils.java:1888)
	at org.junit.platform.commons.util.ReflectionUtils.lambda$isMethodOverriddenByLocalMethods$35(ReflectionUtils.java:1870)
	at java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
	at java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1361)
	at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
	at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
	at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:516)
	at org.junit.platform.commons.util.ReflectionUtils.isMethodOverriddenByLocalMethods(ReflectionUtils.java:1870)
	at org.junit.platform.commons.util.ReflectionUtils.lambda$findAllMethodsInHierarchy$29(ReflectionUtils.java:1661)
	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
	at org.junit.platform.commons.util.ReflectionUtils.findAllMethodsInHierarchy(ReflectionUtils.java:1662)
	at org.junit.platform.commons.util.ReflectionUtils.streamMethods(ReflectionUtils.java:1642)
	at org.junit.platform.commons.util.ReflectionUtils.findMethods(ReflectionUtils.java:1627)
	at org.junit.platform.commons.util.ReflectionUtils.findMethods(ReflectionUtils.java:1618)
	at org.junit.vintage.engine.descriptor.TestSourceProvider.lambda$findMethod$1(TestSourceProvider.java:75)
	at java.util.HashMap.computeIfAbsent(HashMap.java:1128)
	at java.util.Collections$SynchronizedMap.computeIfAbsent(Collections.java:2674)
	at org.junit.vintage.engine.descriptor.TestSourceProvider.findMethod(TestSourceProvider.java:75)
	at org.junit.vintage.engine.descriptor.TestSourceProvider.computeTestSource(TestSourceProvider.java:56)
	at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
	at org.junit.vintage.engine.descriptor.TestSourceProvider.findTestSource(TestSourceProvider.java:47)
	at org.junit.vintage.engine.discovery.RunnerTestDescriptorPostProcessor.addChildrenRecursively(RunnerTestDescriptorPostProcessor.java:62)
	at org.junit.vintage.engine.discovery.RunnerTestDescriptorPostProcessor.applyFiltersAndCreateDescendants(RunnerTestDescriptorPostProcessor.java:41)
	at org.junit.vintage.engine.discovery.VintageDiscoverer.discover(VintageDiscoverer.java:46)
	at org.junit.vintage.engine.VintageTestEngine.discover(VintageTestEngine.java:64)
	at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:152)
	... 13 more

[marcphilipp]

@pshevche Thanks for reporting!

I have a reproducer on Java 8 (probably related to JDK-8193889) when a class in the default package extends a class from a non-default package and both declare a package private method with the same signature.

[pshevche]

@marcphilipp , aha, that's the bit I was missing: the parent class needs to be in a non-default package. Thank you for taking care of it so rapidly!