`zipfs` library classes missing from runtime classpath when testing #370

ryan-williams · 2018-06-10T22:20:34Z

(mill 0.2.3)

Project Structure

build.sc:

import mill._, mill.scalalib._
object foo extends ScalaModule {
  def scalaVersion = "2.12.4"
  object test extends Tests {
    def ivyDeps = Agg(ivy"com.lihaoyi::utest:0.6.0")
    def testFrameworks = Seq("utest.runner.Framework")
  }
}

foo/test/src/Main.scala:

import com.sun.nio.zipfs._
object Main {
  def main(args: Array[String]): Unit = {
    println(classOf[ZipFileSystemProvider])
  }
}

foo/test/src/Test.scala:

import com.sun.nio.zipfs._
import utest._
object Test extends TestSuite {
  println(classOf[ZipFileSystemProvider])
  val tests = Tests {}
}

`mill foo.test.run` works

Prints:

class com.sun.nio.zipfs.ZipFileSystemProvider

`mill foo.test` throws `NoClassDefFoundError`

-------------------------------- Running Tests --------------------------------
X Test 0ms
  java.lang.NoClassDefFoundError: com/sun/nio/zipfs/ZipFileSystemProvider
    Test$.<init>(Test.scala:4)
    Test$.<clinit>(Test.scala:-1)
    sun.misc.Unsafe.ensureClassInitialized(Unsafe.java:-2)
    sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
    sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:142)
    java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088)
    java.lang.reflect.Field.getFieldAccessor(Field.java:1069)
    java.lang.reflect.Field.get(Field.java:393)
    utest.runner.BaseRunner.$anonfun$runSuite$1(BaseRunner.scala:97)
  java.lang.ClassNotFoundException: com.sun.nio.zipfs.ZipFileSystemProvider
    java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    mill.util.ClassLoader$$anon$2.super$findClass(ClassLoader.scala:33)
    mill.util.ClassLoader$$anon$2.$anonfun$findClass$1(ClassLoader.scala:33)
    scala.Option.getOrElse(Option.scala:121)
    mill.util.ClassLoader$$anon$2.findClass(ClassLoader.scala:33)
    java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    Test$.<init>(Test.scala:4)
    Test$.<clinit>(Test.scala:-1)
    sun.misc.Unsafe.ensureClassInitialized(Unsafe.java:-2)
    sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
    sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:142)
    java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088)
    java.lang.reflect.Field.getFieldAccessor(Field.java:1069)
    java.lang.reflect.Field.get(Field.java:393)
    utest.runner.BaseRunner.$anonfun$runSuite$1(BaseRunner.scala:97)
1 targets failed
foo.test.test Test Test.

Notes

I've tried to ask mill for relevant runClasspaths and compileClasspaths, but haven't seen leads for debugging this.
It happens for all classes under com.sun.nio.zipfs._, which come from the zipfs.jar JDK module.
The same thing happens if I use e.g. scalatest runner instead of utest.

The text was updated successfully, but these errors were encountered:

ryan-williams · 2018-06-11T01:19:07Z

I've found that these classes exist in the ClassLoader.getSystemClassLoader, but not in Thread.currentThread().getContextClassLoader or getClass.getClassLoader, which are the same instance of type mill.util.ClassLoader$$anon$.

I haven't figured out where in Mill the problem originates; Jvm.inprocess seems to null out the "parent" ClassLoader in a few places, but afaict the test task that I'm running goes via Jvm.subprocess.

lefou · 2019-04-30T07:35:58Z

Is this still relevant?

hmf · 2019-10-11T16:17:38Z

Can we open this one? I have a test using the GraalVM here that seems to be related.

To test this clone the repo and run maven's test to generate the jars. Then correct the paths here and here to point to you local directory.

If we execute mill -i graaljs.run all is well.
If we execute mill -i graaljs.testwe get:

Test com.mycompany.app.AppTest.testGraalPolyglotSpeed failed: java.lang.IllegalStateException: No language and polyglot implementation was found on the classpath. Make sure the truffle-api.jar is on the classpath., took 0.01 sec

In the test function that breaks I print out the classpath. For the runcase I get the expected:

/home/hmf/IdeaProjects/graal-js-jdk11-maven-demo/graaljs/resources
/home/hmf/IdeaProjects/graal-js-jdk11-maven-demo/out/graaljs/compile/dest/classes
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/regex/regex/19.2.0.1/regex-19.2.0.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/js/js/19.2.0.1/js-19.2.0.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm-commons/6.2.1/asm-commons-6.2.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-api/19.2.0.1/truffle-api-19.2.0.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm-analysis/6.2.1/asm-analysis-6.2.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm-tree/6.2.1/asm-tree-6.2.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/js/js-scriptengine/19.2.0.1/js-scriptengine-19.2.0.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/tools/chromeinspector/19.2.0.1/chromeinspector-19.2.0.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm/6.2.1/asm-6.2.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/tools/profiler/19.2.0.1/profiler-19.2.0.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/ibm/icu/icu4j/62.1/icu4j-62.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/graal-sdk/19.2.0.1/graal-sdk-19.2.0.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/ow2/asm/asm-util/6.2.1/asm-util-6.2.1.jar

but for the test I get something else (see below - run ivyDeps are not included). I am expecting the same classpath plus the JUnit library. Interestingly enough mill -i show graaljs.test.runClasspath shows the correct path (the elements above plus the required testing classes and library).

Note that I have done as few changes as possible to the original code in order to guarantee that I don't mess up somewhere else.

If this is not related to this issue I can open up another ticket. If you need any additional test/diagnostics please tell me.

BTW, any workarounds appreciated.

TIA

/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/javassist/javassist/3.21.0-GA/javassist-3.21.0-GA.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-terminal/3.6.2/jline-terminal-3.6.2.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/mill-main_2.12/0.5.1/mill-main_2.12-0.5.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upack_2.12/0.7.5/upack_2.12-0.7.5.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/scalaparse_2.12/2.1.3/scalaparse_2.12-2.1.3.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-xml_2.12/1.2.0/scala-xml_2.12-1.2.0.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-interpApi_2.12.8/1.6.9/ammonite-interpApi_2.12.8-1.6.9.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/sourcecode_2.12/0.1.7/sourcecode_2.12-0.1.7.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/chuusai/shapeless_2.12/2.3.3/shapeless_2.12-2.3.3.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/os-lib_2.12/0.3.0/os-lib_2.12-0.3.0.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scalameta/scalafmt-interfaces/2.0.0-RC6/scalafmt-interfaces-2.0.0-RC6.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/macro-compat_2.12/1.1.1/macro-compat_2.12-1.1.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite_2.12.8/1.6.9/ammonite_2.12.8-1.6.9.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/io/argonaut/argonaut_2.12/6.2.3/argonaut_2.12-6.2.3.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-reader/3.6.2/jline-reader-3.6.2.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna-platform/4.5.0/jna-platform-4.5.0.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/mill-main-core_2.12/0.5.1/mill-main-core_2.12-0.5.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-collection-compat_2.12/2.0.0/scala-collection-compat_2.12-2.0.0.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle_2.12/0.7.5/upickle_2.12-0.7.5.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/geirsson/coursier-small_2.12/1.3.1/coursier-small_2.12-1.3.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/test-interface/1.0/test-interface-1.0.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/requests_2.12/0.2.0/requests_2.12-0.2.0.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.12.8/scala-reflect-2.12.8.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.12.8/scala-compiler-2.12.8.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier_2.12/2.0.0-RC2/coursier_2.12-2.0.0-RC2.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/pprint_2.12/0.5.5/pprint_2.12-0.5.5.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/mill-main-api_2.12/0.5.1/mill-main-api_2.12-0.5.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-repl_2.12.8/1.6.9/ammonite-repl_2.12.8-1.6.9.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/ipcsocket/ipcsocket/1.0.0/ipcsocket-1.0.0.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/github/scopt/scopt_2.12/3.7.1/scopt_2.12-3.7.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/geny_2.12/0.1.8/geny_2.12-0.1.8.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle-implicits_2.12/0.7.5/upickle-implicits_2.12-0.7.5.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.12.8/scala-library-2.12.8.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-terminal_2.12/1.6.9/ammonite-terminal_2.12-1.6.9.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/mill-scalalib-api_2.12/0.5.1/mill-scalalib-api_2.12-0.5.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-runtime_2.12/1.6.9/ammonite-runtime_2.12-1.6.9.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/fansi_2.12/0.2.7/fansi_2.12-0.2.7.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/mill-main-client/0.5.1/mill-main-client-0.5.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-core_2.12/2.0.0-RC2/coursier-core_2.12-2.0.0-RC2.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-interp_2.12.8/1.6.9/ammonite-interp_2.12.8-1.6.9.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/typesafe/config/1.3.3/config-1.3.3.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna/4.5.0/jna-4.5.0.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/mill-scalalib_2.12/0.5.1/mill-scalalib_2.12-0.5.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scalameta/scalafmt-dynamic_2.12/2.0.0-RC6/scalafmt-dynamic_2.12-2.0.0-RC6.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-util_2.12/1.6.9/ammonite-util_2.12-1.6.9.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/io/get-coursier/coursier-cache_2.12/2.0.0-RC2/coursier-cache_2.12-2.0.0-RC2.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ujson_2.12/0.7.5/ujson_2.12-0.7.5.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/upickle-core_2.12/0.7.5/upickle-core_2.12-0.7.5.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-ops_2.12/1.6.9/ammonite-ops_2.12-1.6.9.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/ammonite-replApi_2.12.8/1.6.9/ammonite-replApi_2.12.8-1.6.9.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/mill-main-moduledefs_2.12/0.5.1/mill-main-moduledefs_2.12-0.5.1.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/lihaoyi/fastparse_2.12/2.1.3/fastparse_2.12-2.1.3.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/github/alexarchambault/argonaut-shapeless_6.2_2.12/1.2.0-M11/argonaut-shapeless_6.2_2.12-1.2.0-M11.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/com/github/javaparser/javaparser-core/3.2.5/javaparser-core-3.2.5.jar
/home/hmf/.cache/coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-terminal-jna/3.6.2/jline-terminal-jna-3.6.2.jar

hmf · 2019-10-12T09:10:13Z

Added a new clean version for testing - mill_utest branch. Just clone and execute the Mill commands.

Same as reported above but now I use uTest to perform the testing (just in case the issue was with JUnit):

mill -i graaljs.run: runs three instances of the JS engine. The Nashorn version should execute nearly twice as slow.

mill -i graaljs.testshould do the same but checks that Nashorn is always slower. However the test fails because the class cannot be loaded.

Note that the original Maven script with JUnit had no problems. Because uTest fails in the same way, maybe the problem is with the class path.

This fixes issues with loading zipfs library classes which reside in `com.sun.*` package. See #370 Pull request: #1848

lefou · 2022-04-26T18:42:36Z

Fixed by Fix #370 Use ExtClassLoader instead of null as parent #1848

ryan-williams mentioned this issue Jun 11, 2018

fall-back to system classloader for com.sun.nio.zipfs._ #372

Closed

This was referenced Oct 15, 2019

Possible incorrect path to tests #715

Closed

Tests fail due to class loading issue - GraalVM cannot be used. #716

Closed

lolgab added a commit to lolgab/mill that referenced this issue Apr 23, 2022

Fix com-lihaoyi#370 Use ExtClassLoader instead of null as parent

dbe464e

lefou pushed a commit that referenced this issue Apr 26, 2022

Use ExtClassLoader instead of null as parent (#1848)

59cb2fd

This fixes issues with loading zipfs library classes which reside in `com.sun.*` package. See #370 Pull request: #1848

lefou closed this as completed Apr 26, 2022

lefou added this to the after 0.10.3 milestone Apr 26, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`zipfs` library classes missing from runtime classpath when testing #370

`zipfs` library classes missing from runtime classpath when testing #370

ryan-williams commented Jun 10, 2018

ryan-williams commented Jun 11, 2018

lefou commented Apr 30, 2019

hmf commented Oct 11, 2019 •

edited

Loading

hmf commented Oct 12, 2019

lefou commented Apr 26, 2022 •

edited

Loading

zipfs library classes missing from runtime classpath when testing #370

zipfs library classes missing from runtime classpath when testing #370

Comments

ryan-williams commented Jun 10, 2018

Project Structure

mill foo.test.run works

mill foo.test throws NoClassDefFoundError

Notes

ryan-williams commented Jun 11, 2018

lefou commented Apr 30, 2019

hmf commented Oct 11, 2019 • edited Loading

hmf commented Oct 12, 2019

lefou commented Apr 26, 2022 • edited Loading

`zipfs` library classes missing from runtime classpath when testing #370

`zipfs` library classes missing from runtime classpath when testing #370

`mill foo.test.run` works

`mill foo.test` throws `NoClassDefFoundError`

hmf commented Oct 11, 2019 •

edited

Loading

lefou commented Apr 26, 2022 •

edited

Loading