Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SemanticDB data generation for Java modules #2227

Merged
merged 14 commits into from
Jan 5, 2023

Conversation

lefou
Copy link
Member

@lefou lefou commented Jan 2, 2023

This adds support for SemanticDB data generation for Java modules and Mixed Java-/Scala-Modules.

Requires the following features:

Old PR comment: ...

For the sake for experimenting, I added a test project under scratch-hello-mixed.

± mill --version
Mill Build Tool version 0.10.10
Java version: 11.0.15, vendor: Eclipse Adoptium, runtime: /opt/openjdk-bin-11.0.15_p10
Default locale: de_DE, platform encoding: UTF-8
OS name: "Linux", version: 5.15.69-gentoo-x86_64, arch: amd64

± mill -i dev.run scratch-hello-mixed -i showNamed java.semanticDbData
[69/763] de.tobiasroeser.mill.vcs.version.VcsVersion.vcsState 
[763/763] dev.run 
[1/1] showNamed 
[1/1] showNamed > [23/23] java.semanticDbData 
[info] compiling 1 Java source to /home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes ...
[error] /home/lefou/work/opensource/mill/scratch-hello-mixed/java/src/example/HelloJava.java:1:1: 
[error] ## Exception when compiling 1 sources to /home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes
[error] java.lang.RuntimeException: java.lang.RuntimeException: boom
[error] jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:163)
[error] jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100)
[error] jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94)
[error] sbt.internal.inc.javac.LocalJavaCompiler.run(LocalJava.scala:345)
[error] sbt.internal.inc.javac.AnalyzingJavaCompiler.$anonfun$compile$12(AnalyzingJavaCompiler.scala:172)
[error] scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
[error] sbt.internal.inc.javac.AnalyzingJavaCompiler.timed(AnalyzingJavaCompiler.scala:262)
[error] sbt.internal.inc.javac.AnalyzingJavaCompiler.compile(AnalyzingJavaCompiler.scala:161)
[error] sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compileJava$2(MixedAnalyzingCompiler.scala:103)
[error] sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compileJava$2$adapted(MixedAnalyzingCompiler.scala:91)
[error] sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:239)
[error] sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compileJava$1(MixedAnalyzingCompiler.scala:91)
[error] scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
[error] sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:248)
[error] sbt.internal.inc.MixedAnalyzingCompiler.compileJava(MixedAnalyzingCompiler.scala:61)
[error] sbt.internal.inc.MixedAnalyzingCompiler.compileJava0$1(MixedAnalyzingCompiler.scala:198)
[error] sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:211)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:534)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:534)
[error] sbt.internal.inc.Incremental$.$anonfun$apply$5(Incremental.scala:179)
[error] sbt.internal.inc.Incremental$.$anonfun$apply$5$adapted(Incremental.scala:177)
[error] sbt.internal.inc.Incremental$$anon$2.run(Incremental.scala:463)
[error] sbt.internal.inc.IncrementalCommon$CycleState.next(IncrementalCommon.scala:116)
[error] sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:56)
[error] sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:52)
[error] sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:263)
[error] sbt.internal.inc.Incremental$.$anonfun$incrementalCompile$8(Incremental.scala:418)
[error] sbt.internal.inc.Incremental$.withClassfileManager(Incremental.scala:506)
[error] sbt.internal.inc.Incremental$.incrementalCompile(Incremental.scala:405)
[error] sbt.internal.inc.Incremental$.apply(Incremental.scala:171)
[error] sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:534)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:488)
[error] sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:332)
[error] sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:425)
[error] sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
[error] mill.scalalib.worker.ZincWorkerImpl.compileInternal(ZincWorkerImpl.scala:561)
[error] mill.scalalib.worker.ZincWorkerImpl.compileJava0(ZincWorkerImpl.scala:285)
[error] mill.scalalib.worker.ZincWorkerImpl.compileJava(ZincWorkerImpl.scala:269)
[error] mill.scalalib.SemanticDbJavaModule.$anonfun$semanticDbData$12(SemanticDbJavaModule.scala:157)
[error] mill.define.Task$TraverseCtx.evaluate(Task.scala:380)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$13(Evaluator.scala:631)
[error] scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[error] scala.Console$.withErr(Console.scala:193)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$12(Evaluator.scala:631)
[error] scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[error] scala.Console$.withOut(Console.scala:164)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$11(Evaluator.scala:630)
[error] scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[error] scala.Console$.withIn(Console.scala:227)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$8(Evaluator.scala:629)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$8$adapted(Evaluator.scala:590)
[error] scala.collection.immutable.Vector.foreach(Vector.scala:1895)
[error] mill.eval.Evaluator.evaluateGroup(Evaluator.scala:590)
[error] mill.eval.Evaluator.$anonfun$evaluateGroupCached$21(Evaluator.scala:482)
[error] scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[error] mill.eval.Evaluator.evaluateGroupCached(Evaluator.scala:473)
[error] mill.eval.Evaluator.$anonfun$sequentialEvaluate$2(Evaluator.scala:202)
[error] scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
[error] scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
[error] scala.collection.AbstractIterator.foreach(Iterator.scala:1300)
[error] mill.eval.Evaluator.sequentialEvaluate(Evaluator.scala:177)
[error] mill.eval.Evaluator.evaluate(Evaluator.scala:162)
[error] mill.main.RunScript$.evaluateNamed(RunScript.scala:337)
[error] mill.main.RunScript$.$anonfun$evaluateTasksNamed$1(RunScript.scala:307)
[error] scala.util.Either.map(Either.scala:382)
[error] mill.main.RunScript$.evaluateTasksNamed(RunScript.scala:305)
[error] mill.main.MainModule$.evaluateTasksNamed(MainModule.scala:50)
[error] mill.main.MainModule.$anonfun$showNamed$1(MainModule.scala:247)
[error] mill.define.Task$TraverseCtx.evaluate(Task.scala:380)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$13(Evaluator.scala:631)
[error] scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[error] scala.Console$.withErr(Console.scala:193)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$12(Evaluator.scala:631)
[error] scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[error] scala.Console$.withOut(Console.scala:164)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$11(Evaluator.scala:630)
[error] scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[error] scala.Console$.withIn(Console.scala:227)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$8(Evaluator.scala:629)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$8$adapted(Evaluator.scala:590)
[error] scala.collection.immutable.Vector.foreach(Vector.scala:1895)
[error] mill.eval.Evaluator.evaluateGroup(Evaluator.scala:590)
[error] mill.eval.Evaluator.$anonfun$evaluateGroupCached$21(Evaluator.scala:482)
[error] scala.util.DynamicVariable.withValue(DynamicVariable.scala:59)
[error] mill.eval.Evaluator.evaluateGroupCached(Evaluator.scala:473)
[error] mill.eval.Evaluator.$anonfun$sequentialEvaluate$2(Evaluator.scala:202)
[error] scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
[error] scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
[error] scala.collection.AbstractIterator.foreach(Iterator.scala:1300)
[error] mill.eval.Evaluator.sequentialEvaluate(Evaluator.scala:177)
[error] mill.eval.Evaluator.evaluate(Evaluator.scala:162)
[error] mill.main.RunScript$.evaluateNamed(RunScript.scala:337)
[error] mill.main.RunScript$.evaluate(RunScript.scala:323)
[error] mill.main.RunScript$.$anonfun$evaluateTasks$1(RunScript.scala:288)
[error] scala.util.Either.map(Either.scala:382)
[error] mill.main.RunScript$.evaluateTasks(RunScript.scala:286)
[error] mill.main.RunScript$.$anonfun$runScript$8(RunScript.scala:104)
[error] ammonite.util.Res$Success.flatMap(Res.scala:62)
[error] mill.main.RunScript$.runScript(RunScript.scala:103)
[error] mill.main.MainRunner.$anonfun$runScript$1(MainRunner.scala:119)
[error] mill.main.MainRunner.watchLoop2(MainRunner.scala:67)
[error] mill.main.MainRunner.runScript(MainRunner.scala:92)
[error] mill.MillMain$.main0(MillMain.scala:320)
[error] mill.MillMain$.main(MillMain.scala:52)
[error] mill.MillMain.main(MillMain.scala)
[error] java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.base/java.lang.reflect.Method.invoke(Method.java:566)
[error] mill.main.client.IsolatedMillMainLoader.runMain(IsolatedMillMainLoader.java:58)
[error] mill.main.client.MillClientMain.main(MillClientMain.java:64)
[error]            
1 targets failed
showNamed 1 targets failed
java.semanticDbData java.lang.RuntimeException: java.lang.RuntimeException: boom
    jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:163)
    jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100)
    jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94)
    sbt.internal.inc.javac.LocalJavaCompiler.run(LocalJava.scala:345)
    sbt.internal.inc.javac.AnalyzingJavaCompiler.$anonfun$compile$12(AnalyzingJavaCompiler.scala:172)
    scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
    sbt.internal.inc.javac.AnalyzingJavaCompiler.timed(AnalyzingJavaCompiler.scala:262)
    sbt.internal.inc.javac.AnalyzingJavaCompiler.compile(AnalyzingJavaCompiler.scala:161)
    sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compileJava$2(MixedAnalyzingCompiler.scala:103)
    sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compileJava$2$adapted(MixedAnalyzingCompiler.scala:91)
    sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:239)
    sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compileJava$1(MixedAnalyzingCompiler.scala:91)
    scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
    sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:248)
    sbt.internal.inc.MixedAnalyzingCompiler.compileJava(MixedAnalyzingCompiler.scala:61)
    sbt.internal.inc.MixedAnalyzingCompiler.compileJava0$1(MixedAnalyzingCompiler.scala:198)
    sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:211)
    sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:534)
    sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:534)
    sbt.internal.inc.Incremental$.$anonfun$apply$5(Incremental.scala:179)
    sbt.internal.inc.Incremental$.$anonfun$apply$5$adapted(Incremental.scala:177)
    sbt.internal.inc.Incremental$$anon$2.run(Incremental.scala:463)
    sbt.internal.inc.IncrementalCommon$CycleState.next(IncrementalCommon.scala:116)
    sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:56)
    sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:52)
    sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:263)
    sbt.internal.inc.Incremental$.$anonfun$incrementalCompile$8(Incremental.scala:418)
    sbt.internal.inc.Incremental$.withClassfileManager(Incremental.scala:506)
    sbt.internal.inc.Incremental$.incrementalCompile(Incremental.scala:405)
    sbt.internal.inc.Incremental$.apply(Incremental.scala:171)
    sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:534)
    sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:488)
    sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:332)
    sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:425)
    sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
    mill.scalalib.worker.ZincWorkerImpl.compileInternal(ZincWorkerImpl.scala:561)
    mill.scalalib.worker.ZincWorkerImpl.compileJava0(ZincWorkerImpl.scala:285)
    mill.scalalib.worker.ZincWorkerImpl.compileJava(ZincWorkerImpl.scala:269)
    mill.scalalib.SemanticDbJavaModule.$anonfun$semanticDbData$12(SemanticDbJavaModule.scala:157)
    mill.define.Task$TraverseCtx.evaluate(Task.scala:380)
java.lang.RuntimeException: boom
    com.sourcegraph.semanticdb_javac.SemanticdbTaskListener.finished(SemanticdbTaskListener.java:76)
    jdk.compiler/com.sun.tools.javac.api.ClientCodeWrapper$WrappedTaskListener.finished(ClientCodeWrapper.java:828)
    jdk.compiler/com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:132)
    jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1418)
    jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1375)
    jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:973)
    jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0(JavacTaskImpl.java:104)
    jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:147)
    jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100)
    jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94)
    sbt.internal.inc.javac.LocalJavaCompiler.run(LocalJava.scala:345)
    sbt.internal.inc.javac.AnalyzingJavaCompiler.$anonfun$compile$12(AnalyzingJavaCompiler.scala:172)
    scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
    sbt.internal.inc.javac.AnalyzingJavaCompiler.timed(AnalyzingJavaCompiler.scala:262)
    sbt.internal.inc.javac.AnalyzingJavaCompiler.compile(AnalyzingJavaCompiler.scala:161)
    sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compileJava$2(MixedAnalyzingCompiler.scala:103)
    sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compileJava$2$adapted(MixedAnalyzingCompiler.scala:91)
    sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:239)
    sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compileJava$1(MixedAnalyzingCompiler.scala:91)
    scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
    sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:248)
    sbt.internal.inc.MixedAnalyzingCompiler.compileJava(MixedAnalyzingCompiler.scala:61)
    sbt.internal.inc.MixedAnalyzingCompiler.compileJava0$1(MixedAnalyzingCompiler.scala:198)
    sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:211)
    sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:534)
    sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:534)
    sbt.internal.inc.Incremental$.$anonfun$apply$5(Incremental.scala:179)
    sbt.internal.inc.Incremental$.$anonfun$apply$5$adapted(Incremental.scala:177)
    sbt.internal.inc.Incremental$$anon$2.run(Incremental.scala:463)
    sbt.internal.inc.IncrementalCommon$CycleState.next(IncrementalCommon.scala:116)
    sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:56)
    sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:52)
    sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:263)
    sbt.internal.inc.Incremental$.$anonfun$incrementalCompile$8(Incremental.scala:418)
    sbt.internal.inc.Incremental$.withClassfileManager(Incremental.scala:506)
    sbt.internal.inc.Incremental$.incrementalCompile(Incremental.scala:405)
    sbt.internal.inc.Incremental$.apply(Incremental.scala:171)
    sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:534)
    sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:488)
    sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:332)
    sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:425)
    sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
    mill.scalalib.worker.ZincWorkerImpl.compileInternal(ZincWorkerImpl.scala:561)
    mill.scalalib.worker.ZincWorkerImpl.compileJava0(ZincWorkerImpl.scala:285)
    mill.scalalib.worker.ZincWorkerImpl.compileJava(ZincWorkerImpl.scala:269)
    mill.scalalib.SemanticDbJavaModule.$anonfun$semanticDbData$12(SemanticDbJavaModule.scala:157)
    mill.define.Task$TraverseCtx.evaluate(Task.scala:380)
com.sourcegraph.semanticdb_javac.CompilationUnitException: vf://tmp//home/lefou/work/opensource/mill/scratch-hello-mixed/java/src/example/HelloJava.java
java.nio.file.FileSystemNotFoundException: Provider "vf" not installed
    java.base/java.nio.file.Path.of(Path.java:212)
    java.base/java.nio.file.Paths.get(Paths.java:97)
    com.sourcegraph.semanticdb_javac.SemanticdbTaskListener.absolutePathFromUri(SemanticdbTaskListener.java:127)
    com.sourcegraph.semanticdb_javac.SemanticdbTaskListener.semanticdbOutputPath(SemanticdbTaskListener.java:170)
    com.sourcegraph.semanticdb_javac.SemanticdbTaskListener.onFinishedAnalyze(SemanticdbTaskListener.java:81)
    com.sourcegraph.semanticdb_javac.SemanticdbTaskListener.finished(SemanticdbTaskListener.java:64)
    jdk.compiler/com.sun.tools.javac.api.ClientCodeWrapper$WrappedTaskListener.finished(ClientCodeWrapper.java:828)
    jdk.compiler/com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:132)
    jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1418)
    jdk.compiler/com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1375)
    jdk.compiler/com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:973)
    jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0(JavacTaskImpl.java:104)
    jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:147)
    jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:100)
    jdk.compiler/com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:94)
    sbt.internal.inc.javac.LocalJavaCompiler.run(LocalJava.scala:345)
    sbt.internal.inc.javac.AnalyzingJavaCompiler.$anonfun$compile$12(AnalyzingJavaCompiler.scala:172)
    scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
    sbt.internal.inc.javac.AnalyzingJavaCompiler.timed(AnalyzingJavaCompiler.scala:262)
    sbt.internal.inc.javac.AnalyzingJavaCompiler.compile(AnalyzingJavaCompiler.scala:161)
    sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compileJava$2(MixedAnalyzingCompiler.scala:103)
    sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compileJava$2$adapted(MixedAnalyzingCompiler.scala:91)
    sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:239)
    sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compileJava$1(MixedAnalyzingCompiler.scala:91)
    scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
    sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:248)
    sbt.internal.inc.MixedAnalyzingCompiler.compileJava(MixedAnalyzingCompiler.scala:61)
    sbt.internal.inc.MixedAnalyzingCompiler.compileJava0$1(MixedAnalyzingCompiler.scala:198)
    sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:211)
    sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:534)
    sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:534)
    sbt.internal.inc.Incremental$.$anonfun$apply$5(Incremental.scala:179)
    sbt.internal.inc.Incremental$.$anonfun$apply$5$adapted(Incremental.scala:177)
    sbt.internal.inc.Incremental$$anon$2.run(Incremental.scala:463)
    sbt.internal.inc.IncrementalCommon$CycleState.next(IncrementalCommon.scala:116)
    sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:56)
    sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:52)
    sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:263)
    sbt.internal.inc.Incremental$.$anonfun$incrementalCompile$8(Incremental.scala:418)
    sbt.internal.inc.Incremental$.withClassfileManager(Incremental.scala:506)
    sbt.internal.inc.Incremental$.incrementalCompile(Incremental.scala:405)
    sbt.internal.inc.Incremental$.apply(Incremental.scala:171)
    sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:534)
    sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:488)
    sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:332)
    sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:425)
    sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
    mill.scalalib.worker.ZincWorkerImpl.compileInternal(ZincWorkerImpl.scala:561)
    mill.scalalib.worker.ZincWorkerImpl.compileJava0(ZincWorkerImpl.scala:285)
    mill.scalalib.worker.ZincWorkerImpl.compileJava(ZincWorkerImpl.scala:269)
    mill.scalalib.SemanticDbJavaModule.$anonfun$semanticDbData$12(SemanticDbJavaModule.scala:157)
    mill.define.Task$TraverseCtx.evaluate(Task.scala:380)
1 targets failed
dev.run java.lang.Exception: Interactive Subprocess Failed (exit code 1)
    mill.modules.Jvm$.runSubprocess(Jvm.scala:186)
    ammonite.$file.build$dev$.$anonfun$run$1(build.sc:1261)
    mill.define.Task$TraverseCtx.evaluate(Task.scala:380)

@com-lihaoyi com-lihaoyi deleted a comment from ckipp01 Jan 2, 2023
@lefou
Copy link
Member Author

lefou commented Jan 2, 2023

I now have some working implementation, but it has still issues.

The output path of the semanticDB files contains unexpected segments resembling the full root path to the source files.

± mill -i dev.run scratch-hello-mixed -i -d showNamed java.semanticDbData
[69/763] de.tobiasroeser.mill.vcs.version.VcsVersion.vcsState 
[143/763] main.core.compile 
[info] compiling 1 Scala source to /home/lefou/work/opensource/mill/out/main/core/compile.dest/classes ...
[info] done compiling
[179/763] main.client.compile 
[info] compiling 1 Java source to /home/lefou/work/opensource/mill/out/main/client/compile.dest/classes ...
[info] done compiling
[763/763] dev.run 
Using explicit system properties: Map()
Compiling /home/lefou/work/opensource/mill/scratch-hello-mixed/build.sc
[1/1] showNamed 
[1/1] showNamed > [23/23] java.semanticDbData 
effective javac options: List(-Xplugin:semanticdb -sourceroot:/home/lefou/work/opensource/mill/scratch-hello-mixed -targetroot:/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes -build-tool:sbt -verbose)
[debug] [zinc] IncrementalCompile -----------
[debug] IncrementalCompile.incrementalCompile
[debug] previous = Stamps for: 1 products, 1 sources, 0 libraries
[debug] current source = Set(/home/lefou/work/opensource/mill/scratch-hello-mixed/java/src/example/HelloJava.java)
[debug] > initialChanges = InitialChanges(Changes(added = Set(), removed = Set(), changed = Set(), unmodified = ...),Set(/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/example/HelloJava.class),Set(),API Changes: Set())
[debug] 
[debug] Initial source changes:
[debug]         removed: Set()
[debug]         added: Set()
[debug]         modified: Set()
[debug] Invalidated products: Set(/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/example/HelloJava.class)
[debug] External API changes: API Changes: Set()
[debug] Modified binary dependencies: Set()
[debug] Initial directly invalidated classes: Set()
[debug] Sources indirectly invalidated by:
[debug]         product: Set(/home/lefou/work/opensource/mill/scratch-hello-mixed/java/src/example/HelloJava.java)
[debug]         binary dep: Set()
[debug]         external source: Set()
[debug] all 1 sources are invalidated
[debug] Initial set of included nodes: 
[debug] Recompiling all sources: number of invalidated sources > 50.0 percent of all sources
[debug] compilation cycle 1
[info] compiling 1 Java source to /home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes ...
[debug] Attempting to call com.sun.tools.javac.api.JavacTool@115b0bf5 directly...
[debug] Java compilation took 0.622091886 s
[debug] [zinc] classfile.Parser parsing example.HelloJava
[debug] Java analysis took 0.095764563 s
[debug] Java compilation + analysis took 0.732980346 s
[info] done compiling
{
  "java.semanticDbData": "ref:9021afe1:/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes"
}

± find /home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes
/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes
/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/META-INF
/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/META-INF/semanticdb
/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/META-INF/semanticdb/home
/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/META-INF/semanticdb/home/lefou
/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/META-INF/semanticdb/home/lefou/work
/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/META-INF/semanticdb/home/lefou/work/opensource
/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/META-INF/semanticdb/home/lefou/work/opensource/mill
/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/META-INF/semanticdb/home/lefou/work/opensource/mill/scratch-hello-mixed
/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/META-INF/semanticdb/home/lefou/work/opensource/mill/scratch-hello-mixed/java
/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/META-INF/semanticdb/home/lefou/work/opensource/mill/scratch-hello-mixed/java/src
/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/META-INF/semanticdb/home/lefou/work/opensource/mill/scratch-hello-mixed/java/src/example
/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/META-INF/semanticdb/home/lefou/work/opensource/mill/scratch-hello-mixed/java/src/example/HelloJava.java.semanticdb
/home/lefou/work/opensource/mill/scratch-hello-mixed/out/java/semanticDbData.dest/classes/example

@ckipp01
Copy link
Collaborator

ckipp01 commented Jan 2, 2023

The output path of the semanticDB files contains unexpected segments resembling the full root path to the source files.

🤔 honestly I'm not really sure what's happening. I just tested with mill-scip to see if it also did that:

~/Documents/scala-workspace/coursier/out/io/kipp/mill/scip/Scip/generate.dest/META-INF/semanticdb/modules/directories/src/main/java/dev/dirs main
❯ l
.rw-r--r--  18k ckipp  2 Jan 19:01  BaseDirectories.java.semanticdb
.rw-r--r--  916 ckipp  2 Jan 19:01  GetWinDirs.java.semanticdb
.rw-r--r--  25k ckipp  2 Jan 19:01  ProjectDirectories.java.semanticdb
.rw-r--r-- 1.3k ckipp  2 Jan 19:01  UnsupportedOperatingSystemException.java.semanticdb
.rw-r--r--  20k ckipp  2 Jan 19:01  UserDirectories.java.semanticdb
.rw-r--r--  36k ckipp  2 Jan 19:01  Util.java.semanticdb

And stuff seems to be where I'd expect it without the weird segments.

@lefou
Copy link
Member Author

lefou commented Jan 2, 2023

The output path of the semanticDB files contains unexpected segments resembling the full root path to the source files.

thinking honestly I'm not really sure what's happening. I just tested with mill-scip to see if it also did that:

~/Documents/scala-workspace/coursier/out/io/kipp/mill/scip/Scip/generate.dest/META-INF/semanticdb/modules/directories/src/main/java/dev/dirs main
❯ l
.rw-r--r--  18k ckipp  2 Jan 19:01  BaseDirectories.java.semanticdb
.rw-r--r--  916 ckipp  2 Jan 19:01  GetWinDirs.java.semanticdb
.rw-r--r--  25k ckipp  2 Jan 19:01  ProjectDirectories.java.semanticdb
.rw-r--r-- 1.3k ckipp  2 Jan 19:01  UnsupportedOperatingSystemException.java.semanticdb
.rw-r--r--  20k ckipp  2 Jan 19:01  UserDirectories.java.semanticdb
.rw-r--r--  36k ckipp  2 Jan 19:01  Util.java.semanticdb

And stuff seems to be where I'd expect it without the weird segments.

Ah, but for you it seem to also contain more that just the packages, probably everything below sourceroot. My suspicion is the logic where sourceroot is matched against the full path to make it relative.

@lefou lefou changed the title semantic db task java 2 WIP: SemanticDB data generation for Java modules Jan 3, 2023
@lefou lefou changed the title WIP: SemanticDB data generation for Java modules SemanticDB data generation for Java modules Jan 3, 2023
@lefou lefou requested a review from ckipp01 January 3, 2023 15:44
@lefou lefou marked this pull request as ready for review January 3, 2023 15:44
Copy link
Collaborator

@ckipp01 ckipp01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very cool @lefou. Testing it out locally with Metals even though I just reviewed it I was still trying with java 17 and wondering why it wasn't working lol.

The biggest usecase for me would be mixed-java-scala targets.

scalalib/src/mill/scalalib/SemanticDbJavaModule.scala Outdated Show resolved Hide resolved
scalalib/src/mill/scalalib/SemanticDbJavaModule.scala Outdated Show resolved Hide resolved
@lefou
Copy link
Member Author

lefou commented Jan 3, 2023

Very cool @lefou. Testing it out locally with Metals even though I just reviewed it I was still trying with java 17 and wondering why it wasn't working lol.

The biggest usecase for me would be mixed-java-scala targets.

I think I need to steal your workaround from mill-scip to get Java 17 to work. Does it work with Java 11 for you?

@ckipp01
Copy link
Collaborator

ckipp01 commented Jan 3, 2023

I think I need to steal your workaround from mill-scip to get Java 17 to work. Does it work with Java 11 for you?

For pure Java modules yes!

@lefou
Copy link
Member Author

lefou commented Jan 3, 2023

I think I need to steal your workaround from mill-scip to get Java 17 to work. Does it work with Java 11 for you?

For pure Java modules yes!

For me, it already works for mixed project. I'm still writing the tests, which needs some extra care because of the invalid sourceroot handling, which I haven't figured out.

@lefou
Copy link
Member Author

lefou commented Jan 3, 2023

For mixed compilation, we still don't support forked Java compiler, which is an issue on Java 17 and above.

@lefou
Copy link
Member Author

lefou commented Jan 3, 2023

The biggest usecase for me would be mixed-java-scala targets.

Out of curiosity, are mixed-java-scala projects with Java 17 and SemanticDB data working with Bloop?

@lefou
Copy link
Member Author

lefou commented Jan 3, 2023

I think I need to steal your workaround from mill-scip to get Java 17 to work. Does it work with Java 11 for you?

For pure Java modules yes!

For me, it already works for mixed project. I'm still writing the tests, which needs some extra care because of the invalid sourceroot handling, which I haven't figured out.

Interesting, the sourceroot handling is fine with Java 17, but not on Java 11 or 8.

@ckipp01
Copy link
Collaborator

ckipp01 commented Jan 4, 2023

The biggest usecase for me would be mixed-java-scala targets.

Out of curiosity, are mixed-java-scala projects with Java 17 and SemanticDB data working with Bloop?

Yes it is.

2023-01-04 07 11 34

@lefou
Copy link
Member Author

lefou commented Jan 4, 2023

I opened a new PR

After its review and merge, I'll rebase this PR on top of it

lefou added a commit that referenced this pull request Jan 4, 2023
…2231)

We need this feature to support SemanticDB generation 
(#2227) via a Java
compiler plugin in mixed Scala/Java projects.

We already introduced non-local Java compilers in 
#1988. This change now
also allows the use of a non-local Java compiler in mixed Java/Scala
modules, if any `-J` option is given.

As compilers are cached, I added the `hashCode` of the relevant runtime
`javacOptions` to the cache key.

I took the opportunity to remove non-relevant options from the existing
Java compiler cache, to reduce the amount of cached compiler instances.

Pull request: #2231
@lefou
Copy link
Member Author

lefou commented Jan 4, 2023

This PR should now produce proper SemanticDB data with all supported Java runtimes and for all combinations of Java and Scala sources.

Copy link
Collaborator

@ckipp01 ckipp01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing, just tested it out and was happily able to be in a Java file and trigger a find references and have it work. Great work @lefou!

Copy link
Member Author

@lefou lefou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated comments

scalalib/src/mill/scalalib/SemanticDbJavaModule.scala Outdated Show resolved Hide resolved
scalalib/src/mill/scalalib/SemanticDbJavaModule.scala Outdated Show resolved Hide resolved
scalalib/src/mill/scalalib/SemanticDbJavaModule.scala Outdated Show resolved Hide resolved
@lefou lefou merged commit e8525bf into com-lihaoyi:main Jan 5, 2023
@lefou lefou deleted the semantic-db-task-java-2 branch January 5, 2023 09:50
@lefou lefou added this to the 0.11.0-M2 milestone Jan 5, 2023
ckipp01 added a commit to ckipp01/metals that referenced this pull request Jan 5, 2023
This is a follow-up to the work that I've done in
scalameta#4295. At that time I only
applied the changes to Scala Targets since Semanticdb was only produced
there when using Mill as your build server. Now that com-lihaoyi/mill#2227
is merged Java Targets and mixed targets will now also produce
semanticdb. We now add the extra check for Mill and its version when we
check for semanticdb java support. This will ensure that users won't get
the error in the Doctor about lacking support.
ckipp01 added a commit to ckipp01/metals that referenced this pull request Jan 5, 2023
This is a follow-up to the work that I've done in
scalameta#4295. At that time I only
applied the changes to Scala Targets since Semanticdb was only produced
there when using Mill as your build server. Now that com-lihaoyi/mill#2227
is merged Java Targets and mixed targets will now also produce
semanticdb. We now add the extra check for Mill and its version when we
check for semanticdb java support. This will ensure that users won't get
the error in the Doctor about lacking support.
ckipp01 added a commit to scalameta/metals that referenced this pull request Jan 5, 2023
…4816)

This is a follow-up to the work that I've done in
#4295. At that time I only
applied the changes to Scala Targets since Semanticdb was only produced
there when using Mill as your build server. Now that com-lihaoyi/mill#2227
is merged Java Targets and mixed targets will now also produce
semanticdb. We now add the extra check for Mill and its version when we
check for semanticdb java support. This will ensure that users won't get
the error in the Doctor about lacking support.
lolgab added a commit to lolgab/mill that referenced this pull request Mar 11, 2024
A copy mechanism was introduced in com-lihaoyi#2227
because of a bug in semanticdb-java
But we don't need to use it in ScalaModule as well
if the Scala semanticdb works correctly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Produce semanticDB for Java
2 participants