Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error in IntelliJ about Multiple 'scala3-library*.jar' files when using scala 3.x and scalajs #3344

Closed
malcolmredheron opened this issue Aug 7, 2024 · 16 comments · Fixed by #3645
Milestone

Comments

@malcolmredheron
Copy link

When I use GenIdea in a mill/scalajs project and then build in intellij, it works fine if build.sc uses scala 2.13.14 but for every version of scala 3 that I have tried IJ gives me errors of the form scala: Multiple 'scala3-library*.jar' files (scala3-library_3-3.3.1.jar, scala3-library_sjs1_3-3.3.1.jar) in Scala compiler classpath in Scala SDK scala-SDK-3.3.1

https://github.com/malcolmredheron/mill-scalajs-experiment contains code and exact steps that demonstrate the problem. This happens for me with mill 0.11.8, 0.11.10, 0.11.11 and 0.11.11-1-066ea7, which is the most recent commit that I could find a build for

@lefou
Copy link
Member

lefou commented Aug 7, 2024

Thank you for this report. We introduced the "scala-SDK" entry in Mill 0.11.8. It looks like some handling for ScalaJS is missing.

@lefou
Copy link
Member

lefou commented Aug 7, 2024

I just checked your example project.

> mill --version
Mill Build Tool version 0.11.11
Java version: 17.0.11, vendor: Eclipse Adoptium, runtime: /opt/openjdk-bin-17.0.11_p9
Default locale: de_DE, platform encoding: UTF-8
OS name: "Linux", version: 6.1.90-gentoo-x86_64, arch: amd64

> mill mill.idea.GenIdea/idea
[1/1] mill.idea.GenIdea.idea 
Analyzing modules ...
[build.sc] [1/1] mill.idea.GenIdea.idea > [7/9] sources.super.mill.scalalib.JavaModule.sources 
Writing 111 IDEA project files to /tmp/mill-scalajs-experiment/.idea ...

> mill show __.scalaCompilerClasspath
[1/1] show 
[
  "qref:v1:878e902e:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-compiler_3/3.3.1/scala3-compiler_3-3.3.1.jar",
  "qref:v1:7d4aa14c:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_sjs1_3/3.3.1/scala3-library_sjs1_3-3.3.1.jar",
  "qref:v1:14776eac:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-interfaces/3.3.1/scala3-interfaces-3.3.1.jar",
  "qref:v1:3a303a0e:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_3/3.3.1/scala3-library_3-3.3.1.jar",
  "qref:v1:4543c999:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/tasty-core_3/3.3.1/tasty-core_3-3.3.1.jar",
  "qref:v1:ab49893c:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-asm/9.5.0-scala-1/scala-asm-9.5.0-scala-1.jar",
  "qref:v1:9614657d:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/compiler-interface/1.3.5/compiler-interface-1.3.5.jar",
  "qref:v1:4ec30e0c:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-reader/3.19.0/jline-reader-3.19.0.jar",
  "qref:v1:83615674:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-terminal/3.19.0/jline-terminal-3.19.0.jar",
  "qref:v1:606a0c92:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-terminal-jna/3.19.0/jline-terminal-jna-3.19.0.jar",
  "qref:v1:e1bf2dc6:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-js/scalajs-library_2.13/1.12.0/scalajs-library_2.13-1.12.0.jar",
  "qref:v1:a94bfc28:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.10/scala-library-2.13.10.jar",
  "qref:v1:a26d3af1:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/com/google/protobuf/protobuf-java/3.7.0/protobuf-java-3.7.0.jar",
  "qref:v1:c491d6fc:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/util-interface/1.3.0/util-interface-1.3.0.jar",
  "qref:v1:b412aca9:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna/5.3.1/jna-5.3.1.jar",
  "qref:v1:283320ce:/home/lefou/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-js/scalajs-javalib/1.12.0/scalajs-javalib-1.12.0.jar"
]

> cat .idea/libraries/scala_SDK_3_3_1.xml 
<component name="libraryTable">
    <library name="scala-SDK-3.3.1" type="Scala">
        <properties>
            <language-level>Scala_3_3</language-level>
            <compiler-classpath>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/com/google/protobuf/protobuf-java/3.7.0/protobuf-java-3.7.0.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/net/java/dev/jna/jna/5.3.1/jna-5.3.1.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-reader/3.19.0/jline-reader-3.19.0.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-terminal-jna/3.19.0/jline-terminal-jna-3.19.0.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/org/jline/jline-terminal/3.19.0/jline-terminal-3.19.0.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-js/scalajs-javalib/1.12.0/scalajs-javalib-1.12.0.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-js/scalajs-library_2.13/1.12.0/scalajs-library_2.13-1.12.0.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/modules/scala-asm/9.5.0-scala-1/scala-asm-9.5.0-scala-1.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.10/scala-library-2.13.10.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-compiler_3/3.3.1/scala3-compiler_3-3.3.1.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-interfaces/3.3.1/scala3-interfaces-3.3.1.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_3/3.3.1/scala3-library_3-3.3.1.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala3-library_sjs1_3/3.3.1/scala3-library_sjs1_3-3.3.1.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/tasty-core_3/3.3.1/tasty-core_3-3.3.1.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/compiler-interface/1.3.5/compiler-interface-1.3.5.jar"/>
                <root url="file://$USER_HOME$/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-sbt/util-interface/1.3.0/util-interface-1.3.0.jar"/>
            </compiler-classpath>
        </properties>
    </library>
</component>

I think the scala-SDK entry should be a scala-js-SDK, to avoid collisions in projects that cross compile to JVM and JS.

But the compiler classpath that Mill uses looks at a first glance identical to the one in the .idea/libraries/scala_SDK_3_3_1.xml, except the ordering.

So the question is, who is wrong? Obviously Mill compiles successfully. So is IntelliJ reporting false here or should the classpath indeed contain less entries? @lolgab, WDYT, anything redundant or too much here?

@lefou
Copy link
Member

lefou commented Aug 7, 2024

From looking at the .idea/libraries/scala_SDK_3_3_1.xml file, it's most likely, that the <library> type should not be Scala but some ScalaJS specific type.

<component name="libraryTable">
    <library name="scala-SDK-3.3.1" type="Scala">
        <properties>
            <language-level>Scala_3_3</language-level>
            <compiler-classpath>
<!-- -->
            </compiler-classpath>
        </properties>
    </library>
</component>

@unkarjedy, do you have any insights?

@unkarjedy
Copy link
Contributor

It's most likely, that the type should not be Scala but some ScalaJS specific type.

There is no any ScalaJS specific library type.
Also Scala JS support is pretty limited:
https://youtrack.jetbrains.com/issues/SCL?q=%22scala%20js%22%20%23Unresolved%20


Scala Plugin has the best support for SBT projects.
If in doubts I would suggest to create a similar SBT project with similar structure and see how the project structure looks there. If something can't be done for SBT, most likely it can't be done for other build tools.

To fix this particular issue, just don't add Scala JS specific jars to the compiler classpath.
E.g. sbt show scalaInstance doesn't mix Scala JS libraries into the SDK

@lefou
Copy link
Member

lefou commented Aug 7, 2024

@unkarjedy Thank you! Does that also mean, IntelliJ isn't using any JS runtime to run tests?

@malcolmredheron
Copy link
Author

malcolmredheron commented Aug 7, 2024 via email

@lefou
Copy link
Member

lefou commented Aug 7, 2024

Would be nice if someone with sbt experience can prepare a sample project.

@malcolmredheron
Copy link
Author

I tried with bsp with these steps:

  • close the project in IJ
  • git clean
  • ./mill mill.bsp.BSP/install
  • Import project from existing sources. Then choose bsp

IJ compiles everything fine but I remembered why I don't like using bsp: it only shows compilation errors (or removes them when they are fixed) when I manually compile. Using GenIdea I get immediate feedback on errors. Unless there's a fix for this, I'd love to get GenIdea working

Also, my real project uses Mill (0.11.8, but I think that it works with 0.11.11 too), Scala 3.3.1 and ScalaJS 1.6.0, and even has a module that's shared between the JS and JVM builds. It works great (after a lot of trouble figuring out how to do the shared module). So GenIdea can work even with Scala 3.x. The problems started when I tried to upgrade to Scala 3.4. Pairing things back to find the source of the error led to this. I don't yet understand why my real project works with Scala 3.3.1 when this one does not

lefou added a commit that referenced this issue Aug 8, 2024
lefou added a commit that referenced this issue Aug 8, 2024
lefou added a commit that referenced this issue Aug 8, 2024
@unkarjedy
Copy link
Contributor

Does that also mean, IntelliJ isn't using any JS runtime to run tests?

Yes

@malcolmredheron
Copy link
Author

I've updated my demo repo to Mill 0.11.12 and get the same error as before. Is that expected? I can't tell whether #3345 was meant to fix all of this

@malcolmredheron
Copy link
Author

Strangely, my real project uses mill 0.11.8 and a mixture of ScalaJs and regular Scala, and builds fine, both in mill and in IntelliJ (using GenIdea to make the project). But upgrading to 0.11.12 gets me the same problems that my example project demonstrates

@hombre
Copy link

hombre commented Oct 1, 2024

With mill version 0.11.12 - 0.12.0-RC2 the example project [mill-scalajs-experiment] compiles in IntelliJ IDEA 2024.2.3 (Ultimate Edition) by removing the scala-library from scala_js_SDK.

mill -j 0 mill.idea.GenIdea/idea; sed -i '/scala3-library_3/d' .idea/libraries/scala_js_SDK_*.xml

@lefou
Copy link
Member

lefou commented Oct 1, 2024

With mill version 0.11.12 - 0.12.0-RC2 the example project [mill-scalajs-experiment] compiles in IntelliJ IDEA 2024.2.3 (Ultimate Edition) by removing the scala-library from scala_js_SDK.

mill -j 0 mill.idea.GenIdea/idea; sed -i '/scala3-library_3/d' .idea/libraries/scala_js_SDK_*.xml

So, you mean removing the scala3-library_3 or scala-library or both?

@lefou
Copy link
Member

lefou commented Oct 1, 2024

Although I think IDEA is misbehaving, if this solves the issue, I can implement this specific logic.

@hombre
Copy link

hombre commented Oct 1, 2024

Only scala3-library.

lefou added a commit to lefou/mill that referenced this issue Oct 1, 2024
IntelliJ IDEA doesn't have proper Scala.JS support but complains about multiple Scala library jars on the compiler classpath. Users reported that removing the scala3-library_3 fixes the issue. This PR applies this to `GenIdea`.

Hopefully fix com-lihaoyi#3344
@lefou lefou closed this as completed in 637732a Oct 2, 2024
@lefou lefou added this to the 0.12.0-RC3 milestone Oct 2, 2024
@malcolmredheron
Copy link
Author

Thank you! I can confirm that this fixed my real project, in addition to the sample that I linked to. (And sorry about the slow reply. I had to figure out some path issues caused by one of the other changes between 0.11.11 and 0.12-M3.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants