Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

abstract case class breaks Scala.js sourcemaps? #14240

Closed
armanbilge opened this issue Jan 10, 2022 · 8 comments
Closed

abstract case class breaks Scala.js sourcemaps? #14240

armanbilge opened this issue Jan 10, 2022 · 8 comments

Comments

@armanbilge
Copy link
Contributor

(originally reported by @mn98 as a possible Cats Effect bug)

Compiler version

3.1.1-RC2

Minimized code

// build.sbt
scalaVersion := "3.1.1-RC2"
scalaJSUseMainModuleInitializer := true
enablePlugins(ScalaJSPlugin, ScalaJSBundlerPlugin)

// project/plugins.sbt
addSbtPlugin("org.scala-js" % "sbt-scalajs" % "1.8.0")
addSbtPlugin("ch.epfl.scala" % "sbt-scalajs-bundler" % "0.20.0")

// src/main/scala/Main.scala
object Main {
  def main(args: Array[String]): Unit = ExitCode(0)
}
abstract case class ExitCode private (code: Int)
object ExitCode {
  def apply(i: Int): ExitCode = new ExitCode(i & 0xff) {}
  val Success: ExitCode = ExitCode(0)
}

Output

sbt> Compile / fastOptJS / webpack
[info] compiling 1 Scala source to /workspace/http4s4tw/target/scala-3.1.1-RC2/classes ...
[info] Updating NPM dependencies
[info] Fast optimizing /workspace/http4s4tw/target/scala-3.1.1-RC2/http4s4tw-fastopt
[error] npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
[error] npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
[error] npm WARN deprecated chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
[error] npm WARN deprecated chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
[error] npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
[error] npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
[info] added 603 packages, and audited 604 packages in 10s
[info] 20 packages are looking for funding
[info]   run `npm fund` for details
[info] 14 vulnerabilities (8 moderate, 6 high)
[info] To address issues that do not require attention, run:
[info]   npm audit fix
[info] To address all issues, run:
[info]   npm audit fix --force
[info] Run `npm audit` for details.
[info] Writing scalajs.webpack.config.js
[info] Bundling the application with its NPM dependencies
[error] /workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/source-map/lib/source-map-generator.js:298
[error]       throw new Error('Invalid mapping: ' + JSON.stringify({
[error]       ^
[error] Error: Invalid mapping: {"generated":{"line":969,"column":24},"source":"/workspace/http4s4tw/src/main/scala/Main.scala","original":{"line":4,"column":-1},"name":"toString"}
[error]     at SourceMapGenerator_validateMapping [as _validateMapping] (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/source-map/lib/source-map-generator.js:298:13)
[error]     at SourceMapGenerator_addMapping [as addMapping] (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/source-map/lib/source-map-generator.js:110:12)
[error]     at /workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/source-map/lib/source-node.js:351:13
[error]     at SourceNode_walk [as walk] (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/source-map/lib/source-node.js:230:9)
[error]     at SourceNode_walk [as walk] (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/source-map/lib/source-node.js:226:13)
[error]     at SourceNode_walk [as walk] (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/source-map/lib/source-node.js:226:13)
[error]     at SourceNode_toStringWithSourceMap [as toStringWithSourceMap] (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/source-map/lib/source-node.js:342:8)
[error]     at ConcatSource.proto.sourceAndMap (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/webpack-sources/lib/SourceAndMapMixin.js:29:32)
[error]     at CachedSource.sourceAndMap (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/webpack-sources/lib/CachedSource.js:58:31)
[error]     at getTaskForFile (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/webpack/lib/SourceMapDevToolPlugin.js:37:30)
[error]     at /workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/webpack/lib/SourceMapDevToolPlugin.js:136:20
[error]     at Array.forEach (<anonymous>)
[error]     at /workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/webpack/lib/SourceMapDevToolPlugin.js:130:12
[error]     at SyncHook.eval [as call] (eval at create (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/tapable/lib/HookCodeFactory.js:19:10), <anonymous>:7:1)
[error]     at SyncHook.lazyCompileHook (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/tapable/lib/Hook.js:154:20)
[error]     at /workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/webpack/lib/Compilation.js:1319:42
[error]     at AsyncSeriesHook.eval [as callAsync] (eval at create (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
[error]     at AsyncSeriesHook.lazyCompileHook (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/tapable/lib/Hook.js:154:20)
[error]     at /workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/webpack/lib/Compilation.js:1315:36
[error]     at AsyncSeriesHook.eval [as callAsync] (eval at create (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
[error]     at AsyncSeriesHook.lazyCompileHook (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/tapable/lib/Hook.js:154:20)
[error]     at /workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/webpack/lib/Compilation.js:1311:32
[error]     at AsyncSeriesHook.eval [as callAsync] (eval at create (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:6:1)
[error]     at AsyncSeriesHook.lazyCompileHook (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/tapable/lib/Hook.js:154:20)
[error]     at Compilation.seal (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/webpack/lib/Compilation.js:1248:27)
[error]     at /workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/webpack/lib/Compiler.js:625:18
[error]     at /workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/webpack/lib/Compilation.js:1171:4
[error]     at AsyncSeriesHook.eval [as callAsync] (eval at create (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
[error]     at AsyncSeriesHook.lazyCompileHook (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/tapable/lib/Hook.js:154:20)
[error]     at Compilation.finish (/workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/webpack/lib/Compilation.js:1163:28)
[error] Failure on parsing the output of webpack: No content to map due to end-of-input
[error]  at [Source: java.lang.ProcessImpl$ProcessPipeInputStream@3fa6bef7; line: 1, column: 0]
[error] You can try to manually execute the command
[error] node /workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/node_modules/webpack/bin/webpack --bail --profile --json --config /workspace/http4s4tw/target/scala-3.1.1-RC2/scalajs-bundler/main/scalajs.webpack.config.js
[error] 
[error] stack trace is suppressed; run last Compile / fastOptJS for the full output
[error] (Compile / fastOptJS / webpack) Non-zero exit code: 1
[error] Total time: 12 s, completed Jan 10, 2022, 4:01:39 PM

Expectation

Switching to Scala 2.13.7 there is no error.

@armanbilge
Copy link
Contributor Author

If it's helpful, here's how I was able to re-implement ExitCode to work around this issue.

@SethTisue
Copy link
Member

Is this truly minimized, is all of the code necessary? Or is it just any abstract case class?

@SethTisue
Copy link
Member

SethTisue commented Jan 12, 2022

Is there reason to believe the bug is in this repo and not in the scalajs-bundler repo?

@armanbilge
Copy link
Contributor Author

armanbilge commented Jan 12, 2022

Is this truly minimized, is all of the code necessary?

It can possibly be slightly more minimized. Like the Int mask thing, woops. But the val in the companion object was also necessary. Edit: oh, and the private constructor also seems entangled.

Is there reason to believe the bug is in this repo and not in the scalars-bundler repo?

Not directly. The same code does work with Scala 2.x, but I can't rule out the possibility that Scala 3.x is generating different but equally valid output that the bundling thing doesn't understand.

I don't understand the Scala.js pipeline well enough to know when/how sourcemaps get generated, but another experiment I did was using the Scala 2.13 artifacts for Cats Effect in a Scala 3.1 project, and I had no problems there. So whatever is different I assume is in the Scala 3 sjsir artifacts for Cats Effect.

@SethTisue
Copy link
Member

Okay. Let's wait for Scala.js expert to weigh in on where we should look next.

@sjrd
Copy link
Member

sjrd commented Jan 21, 2022

The column:-1 in "original":{"line":4,"column":-1} is suspicious. All line and column numbers should be >= 0.

Scala 2 has 1-based positions, but the IR has 0-based position, so we do {line,column} - 1 when translating the scalac Positions to the IR Positions. We do the same in Dotty, but could it be that Dotty has 0-based positions?


Edit: Oops, yes, looks like it does:
https://github.com/lampepfl/dotty/blob/c49a1cf106f2f4df06b8b06d94fcf76d687b1bdb/compiler/src/dotty/tools/dotc/util/SourceFile.scala#L174-L177
https://github.com/lampepfl/dotty/blob/c49a1cf106f2f4df06b8b06d94fcf76d687b1bdb/compiler/src/dotty/tools/dotc/util/SourceFile.scala#L197-L198

sjrd added a commit to dotty-staging/dotty that referenced this issue Jan 21, 2022
scalac positions are 1-based, while IR positions are 0-based. There
is therefore a `- 1` in the nsc plugin, that was ported over to
dotc without questioning. It turns out that dotc uses 0-based
positions as well (as documented in `SourceFile`), and so we should
not make any adaptation there.
@sjrd sjrd self-assigned this Jan 21, 2022
@armanbilge
Copy link
Contributor Author

armanbilge commented Jan 21, 2022

@sjrd Thanks for tracking this down! A question: assuming (IIUC) that this bug has affected all currently published Scala 3 SJSIR artifacts, it (mildly) breaks the promise undoes the advantage of backwards-compatibility since everyone would have to republish. Is there any way for the Scala.js linker to detect it's consuming IR generated by a pre-fix Scala 3 and apply an off-by-one correction? Thanks!

@sjrd
Copy link
Member

sjrd commented Jan 21, 2022

Is there any way for the Scala.js linker to detect it's consuming IR generated by a pre-fix Scala 3 and apply an off-by-one correction?

No. It can detect if it's consuming a previous version of the IR. But it cannot know whether that was emitted by Scala 2 or Scala 3, so it would still not know whether to apply the correction or not.

it (mildly) breaks the promise undoes the advantage of backwards-compatibility since everyone would have to republish.

Backwards compat is guaranteed when the old compiler did the right thing at the time it did it. In other words, we do not break correct IR. But if the compiler was buggy and emitted wrong code to begin with ... there is little we can do.

This is not specific to Scala.js. If the JVM backend emits the wrong bytecode for some piece of program, the JVM is not going to fix it for you later. You have to recompile and republish with a fixed version of the compiler. This has happened in the past: I remember fixing myself the codegen for intValue << longValue in 2.12.something.

smarter added a commit that referenced this issue Jan 26, 2022
Fix #14240: Fix off-by-1 when emitting Scala.js IR Positions.
Xavientois pushed a commit to Xavientois/dotty that referenced this issue Feb 2, 2022
scalac positions are 1-based, while IR positions are 0-based. There
is therefore a `- 1` in the nsc plugin, that was ported over to
dotc without questioning. It turns out that dotc uses 0-based
positions as well (as documented in `SourceFile`), and so we should
not make any adaptation there.
olsdavis pushed a commit to olsdavis/dotty that referenced this issue Apr 4, 2022
scalac positions are 1-based, while IR positions are 0-based. There
is therefore a `- 1` in the nsc plugin, that was ported over to
dotc without questioning. It turns out that dotc uses 0-based
positions as well (as documented in `SourceFile`), and so we should
not make any adaptation there.
raquo added a commit to raquo/scala-dom-types that referenced this issue Sep 27, 2022
raquo added a commit to raquo/scala-dom-types that referenced this issue Sep 27, 2022
raquo added a commit to raquo/Airstream that referenced this issue Sep 27, 2022
raquo added a commit to raquo/ew that referenced this issue Sep 27, 2022
raquo added a commit to raquo/scala-dom-testutils that referenced this issue Sep 27, 2022
raquo added a commit to raquo/scala-dom-testutils that referenced this issue Sep 27, 2022
raquo added a commit to raquo/url-dsl that referenced this issue Sep 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants