Java 9 support #3138

smarter · 2017-09-17T20:48:50Z

No description provided.

smarter · 2017-09-17T23:33:04Z

sbt 0.13 cannot be used with Java 9 because it tries to compile the project build with Scala 2.10 which really does not like Java 9, so we'll probably have to wait until we switch to sbt 1 to test this properly. But it seems to work:

$ JAVA_HOME=~/opt/openjdk-9-181 ./bin/dotr
scala> java.util.List.of(1,2) 
val res0: java.util.List[Int] = [1, 2]

retronym · 2017-09-18T07:33:19Z

The lastest sbt launcher script extracts rt.jar when starting on java 9 which lets scalac 2.10 work

smarter · 2017-09-18T22:54:10Z

@retronym I gave it a try but got an assert in scala.reflect: https://gist.github.com/smarter/67ee91680bc3cc346fb7500ee023112f am I doing something wrong?

smarter · 2017-09-18T23:17:54Z

The ExposedValues mentioned in the stacktrace is this thing stolen from Scala.js to avoid having to put everything into build.sbt

retronym · 2017-09-18T23:33:15Z

Yep, that's a bug in reflection.

Workaround: move your .autoplugin, and maybe your whole ./project/*.scala out of the empty package: scala/scala-dev#304

We did this in retronym/scala@96e8e97

smarter · 2017-09-18T23:46:07Z

Nice! Thanks :). I now have a bootstrapped dotty on Java 9. My sbt ExtractDependencies phase warned me I wasn't handling PlainNioFile which is also something that is missing from zinc as far as I can tell: https://github.com/sbt/zinc/blob/1.x/internal/compiler-bridge/src/main/scala/xsbt/Dependency.scala#L113-L123

retronym · 2017-09-18T23:56:33Z

Good catch, I've added a todo item to scala/scala-dev#139

smarter · 2017-09-19T11:59:34Z

I now have a bootstrapped dotty on Java 9.

I spoke too soon. the EA 181 build from http://jdk.java.net/9/ works, but a nightly from https://builds.shipilev.net/openjdk-jdk9-dev-release/ doesn't. It crashes with:

Exception in thread "main" java.lang.BootstrapMethodError: call site initialization exception
        at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:385)
        at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:250)
        at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:240)
        at dotty.runtime.LazyVals$.<init>(LazyVals.scala:87)
        at dotty.runtime.LazyVals$.<clinit>(LazyVals.scala)
        at dotty.tools.dotc.core.Contexts$ContextBase.<clinit>(Contexts.scala:542)
        at dotty.tools.dotc.Driver.initCtx(Driver.scala:36)
        at dotty.tools.dotc.Driver.process(Driver.scala:88)
        at dotty.tools.dotc.Driver.process(Driver.scala:105)
        at dotty.tools.dotc.Driver.main(Driver.scala:132)
        at dotty.tools.dotc.Main.main(Main.scala)
Caused by: java.lang.invoke.LambdaConversionException: Type mismatch for instantiated parameter 0: int is not a subtype of class java.lang.Object
        at java.base/java.lang.invoke.AbstractValidatingLambdaMetafactory.checkDescriptor(AbstractValidatingLambdaMetafactory.java:308)
        at java.base/java.lang.invoke.AbstractValidatingLambdaMetafactory.validateMetafactoryArgs(AbstractValidatingLambdaMetafactory.java:294)
        at java.base/java.lang.invoke.LambdaMetafactory.metafactory(LambdaMetafactory.java:311)
        at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:332)
        ... 10 more

The corresponding LMF bootstrap method is:

  3: #68 invokestatic java/lang/invoke/LambdaMetafactory.metafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/Met
    Method arguments:
      #70 (Ljava/lang/Object;)Ljava/lang/Object;
      #153 invokespecial dotty/runtime/LazyVals$.$init$$$anonfun$4:(I)Ljava/lang/Object;
      #154 (I)Ljava/lang/Object;

which indeed looks fishy. The check was added in http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/4be3ef759ead but the referenced bug report appears to be private.

smarter · 2017-09-19T12:26:35Z

I spoke too soon. the EA 181 build from http://jdk.java.net/9/ works, but a nightly from https://builds.shipilev.net/openjdk-jdk9-dev-release/ doesn't.

My bad, EA 181 doesn't work either, I messed up and was accidentally running an old jdk9 build.

See scala#3138.

Blaisorblade · 2018-05-08T00:28:00Z

FWIW we’re now on sbt 1 which might help. Also FWIW now java 9 is EOL: http://www.oracle.com/technetwork/java/javase/downloads/jdk9-downloads-3848520.html

Glavo · 2018-05-23T17:15:42Z

Hmmm, what is the status of this PR? Is it frozen? Now Dotty is my only obstruction to using Java 10 by default.

smarter · 2018-05-23T17:24:18Z

Work in progress, will come back to it at some point.

allanrenucci · 2018-08-24T10:41:14Z

project/Build.scala

+          "-Ddotty.tests.classes.scalaAsm=" + findLib(attList, "scala-asm"),
+          "-Ddotty.tests.classes.scalaXml=" + findLib(attList, "scala-xml"),
+          "-Ddotty.tests.classes.jlineTerminal=" + findLib(attList, "jline-terminal"),
+          "-Ddotty.tests.classes.jlineReader=" + findLib(attList, "jline-reader")


We also use BuildInfoPlugin to achieve the same thing. Should we pick one over the other?

No preference, but let's not change this now.

smarter · 2018-08-24T13:54:42Z

This is finally ready for review ;)

allanrenucci

LGTM! 🎉

How do we test this? Should we use Java 9 on the CI instead of Java 8?

allanrenucci · 2018-08-24T14:36:37Z

compiler/test/dotty/tools/vulpix/TestConfiguration.scala

      file.getAbsolutePath
-    } mkString(":")
-  }
+    }).mkString(":")


We should use File.pathSeparator

allanrenucci · 2018-08-24T14:37:29Z

compiler/test/dotty/tools/dotc/CompilationTests.scala

-      Jars.dottyInterfaces + ":" + Jars.jlineTerminal + ":" + Jars.jlineReader,
+      // and the other compiler dependenies:
+      Properties.compilerInterface + ":" + Properties.scalaLibrary + ":" + Properties.scalaAsm + ":" +
+      Properties.dottyInterfaces + ":" + Properties.jlineTerminal + ":" + Properties.jlineReader,


Let's use TestConfiguration.mkClasspath?

mkClasspath checks that each directory exists so won't work on libGroup and dotty1Group here. I agree this code would be cleaned up more but let's keep that for another pr.

Ok. Then we should use File.pathSeparator here as well

There's a lot of place where we use ":" right now which need to be fixed so I've opened an issue for it: #5000

smarter · 2018-08-24T16:05:56Z

How do we test this? Should we use Java 9 on the CI instead of Java 8?

No because we still want to support Java 8, instead ideally we should run the tests both with Java 8 and with Java 10 (and replace 10 by 11 when it's released).

allanrenucci · 2018-08-24T16:12:55Z

Ok. I guess I'll set this up as a nightly job then. We don't want to double CI time.

The dotty implementation of Properties.isJavaAtLeast is broken for Java 9 (Fixed in Scalac at scala/scala#5276, but maybe we just shouldn't have a copy of Properties.scala in Dotty). The test is useless now anyway since we only compile with 2.12 which requires Java 8, so this code is never going to run on an older version of the JVM.

The standard Java library as well as any other module used will not be available in the regular Java classpath but only in the jrt:/ filesystem.

It's no longer introspectable under Java 9. The scalac 2.12 shell scripts work around this by passing the bootclasspath jars as an additional system property, but we can avoid this complication by just using -classpath instead. This may have a slight negative impact on startup performance since the JVM bytecode verifier is not run for classes from the bootclasspath, but we can worry about that later.

- scala-library and scala-xml used to be on the bootclasspath and need to be on the regular classpath now. - Jars.findJarFromRuntime relied on ClassLoader.getSystemClassLoader being an URLClassLoader, this isn't true anymore in Java 9. This is replaced by passing properties from the sbt build instead, like we were already doing for the dotty jars. While I was at it I completely killed dotty.Jars and cleaned things a bit.

This broke tests/pos/t2484.scala with Java 9 and doesn't seem to actually be useful for anything.

Blaisorblade · 2018-08-25T13:05:39Z

Bootstrap started failing since the merge with FileAlreadyExistsException http://dotty-ci.epfl.ch/lampepfl/dotty/7060/5 @smarter @allanrenucci

The classpath is intended for type checking, not for compiler run-time. Thus, they should be passed as compiler options, not JVM options. It works before because the compiler makes bootstrap classpath available for type checking. However, we can no longer use bootstrap classpath after Java 9 support (scala#3138).

smarter changed the title ~~[WIP] Java9 support~~ [WIP] Java 9 support Sep 17, 2017

smarter force-pushed the java9 branch 2 times, most recently from fb369f4 to 3ba9411 Compare September 17, 2017 20:51

smarter mentioned this pull request Sep 17, 2017

Fix IncompatibleClassChangeError under Java 9 lampepfl/scala#24

Merged

smarter changed the title ~~[WIP] Java 9 support~~ Java 9 support Sep 17, 2017

smarter force-pushed the java9 branch from 026bd85 to c5c6937 Compare September 17, 2017 23:03

smarter changed the title ~~Java 9 support~~ [WIP] Java 9 support Sep 17, 2017

smarter force-pushed the java9 branch 2 times, most recently from ecfd639 to 98812e0 Compare September 17, 2017 23:31

smarter mentioned this pull request Oct 10, 2017

Dotty 0.4.0-RC1 shell scripts should detect Java 9 and stop with an error #3304

Closed

smarter force-pushed the java9 branch 3 times, most recently from ce74021 to 778dc73 Compare October 12, 2017 15:59

smarter added this to the 0.5 Tech Preview milestone Oct 18, 2017

smarter removed this from the 0.5 Tech Preview milestone Nov 24, 2017

OlivierBlanvillain assigned smarter Jan 18, 2018

Blaisorblade added a commit to dotty-staging/dotty that referenced this pull request Feb 14, 2018

Clarify we do NOT yet support Java 9.

f9c58f5

See scala#3138.

Blaisorblade mentioned this pull request Feb 14, 2018

Clarify we do NOT yet support Java 9. #3998

Merged

smarter force-pushed the java9 branch from 778dc73 to 1dee611 Compare August 23, 2018 17:05

allanrenucci reviewed Aug 24, 2018

View reviewed changes

smarter force-pushed the java9 branch 4 times, most recently from 2ddd739 to e28af77 Compare August 24, 2018 13:27

smarter changed the title ~~[WIP] Java 9 support~~ Java 9 support Aug 24, 2018

smarter requested a review from allanrenucci August 24, 2018 13:54

smarter assigned allanrenucci and unassigned smarter Aug 24, 2018

allanrenucci approved these changes Aug 24, 2018

View reviewed changes

smarter force-pushed the java9 branch from e28af77 to 9e6e4ce Compare August 24, 2018 16:50

smarter added 8 commits August 25, 2018 14:52

Upgrade backend for Java 9 support

c58495d

Stop using depcreated Class#newInstance

1cd8cf3

Resolve classes found in the jrt:/ filesystem

9ff6f09

The standard Java library as well as any other module used will not be available in the regular Java classpath but only in the jrt:/ filesystem.

Fix crash when recording dependency information

e9634de

Remove broken special case when typing annotations

5061c29

This broke tests/pos/t2484.scala with Java 9 and doesn't seem to actually be useful for anything.

smarter force-pushed the java9 branch from 9e6e4ce to 5061c29 Compare August 25, 2018 05:52

smarter merged commit 422342d into scala:master Aug 25, 2018

Blaisorblade deleted the java9 branch August 25, 2018 12:43

liufengyun mentioned this pull request Aug 26, 2018

Fail jhm on error and fix benchmarks #5020

Merged

Blaisorblade mentioned this pull request Aug 26, 2018

Bootstrapped builds fail sporadically since Java 9 merge #5021

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Java 9 support #3138

Java 9 support #3138

smarter commented Sep 17, 2017

smarter commented Sep 17, 2017

retronym commented Sep 18, 2017 via email

smarter commented Sep 18, 2017

smarter commented Sep 18, 2017

retronym commented Sep 18, 2017 •

edited

Loading

smarter commented Sep 18, 2017

retronym commented Sep 18, 2017

smarter commented Sep 19, 2017

smarter commented Sep 19, 2017

Blaisorblade commented May 8, 2018 •

edited

Loading

Glavo commented May 23, 2018

smarter commented May 23, 2018

allanrenucci Aug 24, 2018

smarter Aug 24, 2018

smarter commented Aug 24, 2018

allanrenucci left a comment

allanrenucci Aug 24, 2018

smarter Aug 24, 2018

allanrenucci Aug 24, 2018

smarter Aug 24, 2018

allanrenucci Aug 24, 2018

smarter Aug 24, 2018 •

edited

Loading

smarter commented Aug 24, 2018

allanrenucci commented Aug 24, 2018

Blaisorblade commented Aug 25, 2018

Java 9 support #3138

Java 9 support #3138

Conversation

smarter commented Sep 17, 2017

smarter commented Sep 17, 2017

retronym commented Sep 18, 2017 via email

smarter commented Sep 18, 2017

smarter commented Sep 18, 2017

retronym commented Sep 18, 2017 • edited Loading

smarter commented Sep 18, 2017

retronym commented Sep 18, 2017

smarter commented Sep 19, 2017

smarter commented Sep 19, 2017

Blaisorblade commented May 8, 2018 • edited Loading

Glavo commented May 23, 2018

smarter commented May 23, 2018

allanrenucci Aug 24, 2018

Choose a reason for hiding this comment

smarter Aug 24, 2018

Choose a reason for hiding this comment

smarter commented Aug 24, 2018

allanrenucci left a comment

Choose a reason for hiding this comment

allanrenucci Aug 24, 2018

Choose a reason for hiding this comment

smarter Aug 24, 2018

Choose a reason for hiding this comment

allanrenucci Aug 24, 2018

Choose a reason for hiding this comment

smarter Aug 24, 2018

Choose a reason for hiding this comment

allanrenucci Aug 24, 2018

Choose a reason for hiding this comment

smarter Aug 24, 2018 • edited Loading

Choose a reason for hiding this comment

smarter commented Aug 24, 2018

allanrenucci commented Aug 24, 2018

Blaisorblade commented Aug 25, 2018

retronym commented Sep 18, 2017 •

edited

Loading

Blaisorblade commented May 8, 2018 •

edited

Loading

smarter Aug 24, 2018 •

edited

Loading