Update scala3-library, ... to 3.1.0

[scala-steward]

Updates

• org.scala-lang:scala3-library
• org.scala-lang:scala3-library_sjs1

from 3.0.2 to 3.1.0.
GitHub Release Notes - Version Diff - Version Diff

I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.

If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.

Configure Scala Steward for your repository with a .scala-steward.conf file.

Have a fantastic day writing Scala!

Ignore future updates

Add this to your .scala-steward.conf file to ignore future updates of this dependency:

updates.ignore = [ { groupId = "org.scala-lang" } ]

labels: library-update, semver-minor

[fthomas]

I guess we should defer this because of https://twitter.com/SethTisue/status/1450260785762947076:

PSA: Scala 3.0 cannot consume 3.1-compiled libraries. Scala 3 minor versions are backwards compatible only, not forwards. This is a big change from Scala 2.

[smarter]

Do what you think is best, but I would encourage you to upgrade to 3.1 sooner rather than later (unless downstream is known to be stuck with an older compiler for some reason), in the same way you already always track the latest Scala.js release. Especially at this stage, you're more likely to hit an unforeseen bug on an old compiler than a new one.

[djspiewak]

@smarter Scala.js has several orders of magnitude fewer users than JVM Scala does, and they tend to have fewer endemic issues that get them trapped on older versions (though it has happened once or twice, and it sucked). A huge swath of commercial Scala codebases are still on Scala 2.12. Obviously very few people are on Scala 3 at all right now, but once more people begin rolling out software on 3.x, there is exactly zero doubt in my mind that people will end up stuck on older versions due to complex and subtle issues they don't have time to track down or fix… but then the ecosystem forcibly leaves them behind because of the upgrade path.

To be clear, I'm not objecting to Scala eschewing forward-compatibility. What I'm more concerned about is the lack of a -target flag which would allow us to build for older versions, particularly since the bincompat suffix is _3. You're literally recreating the scenario which led to the JVM community en masse digging in its heels on Java 8, but without the easing function (the -target flag). I would also be okay with you saying "well, we see the value in -target, but we don't have the resources to pursue it right now, so someone please donate some time to build and maintain this". Like, commercial companies will ultimately be hit the hardest by this, so it's fair to say that they should put some money on the table to build the solution. But even that isn't being put forward as an alternative.

This problem is, overall, magnified in upstream projects like Cats and Cats Effect, since we cannot and will not ever fully know our entire downstream web, particularly since a lot of it is in commercial contexts outside of our visibility.

As things stand right now, I think we'll probably sit on this for the moment. At least to give time for the ecosystem to discuss and come to some sort of consensus as to what we should do.

[smarter]

"well, we see the value in -target, but we don't have the resources to pursue it right now, so someone please donate some time to build and maintain this"

That's my personal position (as in, I personally don't have the resources to investigate if this is even feasible), I can't speak for anyone else.

As things stand right now, I think we'll probably sit on this for the moment.

Fair. But you could still cross-compile in your CI between 3.0.2 and the latest release (or even better, the latest RC), and publish / skip := scalaVersion.value == latestVersion to only publish the 3.0 version. That way you'll immediately see if some bug you're hitting has been fixed in a more recent version, or if there's been a regression.

[SethTisue]

But you could still cross-compile in your CI between 3.0.2 and the latest release

An example PR that takes this approach: typelevel/scalacheck#847

[rossabaker]

CVEs are rare in this library, but common a hop or two downstream. This argument is mostly parametric on "cats".

Scenario A

1. We merge this now.
2. We unwittingly do something else that prevents a Scala 3.0 rollback.
3. We release as cats-2.6.2.
4. A CVE is discovered, originating in cats-2.6.1.

• We can't roll back to Scala 3.0 because >= 2.6.2 already depended on it.
• We've already burned our semver digit for this.
• I guess we'd have to make a cats-2.6.1.1?

Scala 3 is a lot harder to swallow if critical ecosystem patches start requiring non-trivial tooling upgrades.

Scenario B

1. We merge this now.
2. We unwittingly do something else that prevents a Scala 3.0 rollback.
3. We release as cats-2.7.0.
4. A CVE is discovered, originating in cats-2.6.1.
5. We branch off v2.6.1 and release v2.6.2.

• It only works downstream if authors respect minor-bump-for-minor-bump
• Downstream libraries in "early semver" have a lot more judgment calls to make.
• Insight from new compiler warnings might make a better product, but good use of CI can bring that same benefit.

This is tenable, but the more coupled upgrades get, the harder it is to execute them. I don't see any end user benefit unless the newer compiler generates demonstrably more efficient jars.

Scenario C

1. We build against 3.0.latest and 3.latest.latest, publishing only for the former but exploiting the wisdom of the latter.
2. We can't unwittingly do something that prevents a Scala 3.0 rollback.
3. We release as cats-2.6.2.
4. A CVE is discovered, originating in cats-2.6.1.
5. We release v2.6.3.

---

In all cases, applications should get on the latest compiler they can. And in most cases, it competes with a lot of other priorities. That extra cross-build is something we'd all love to leave behind, but empathy for users who can't spend all day on the version treadmill is necessary if we want a sustainable ecosystem with jobs and a pool of new contributors.

I'm strongly in favor of Scenario C, which is basically exactly what we do with Java versions.

[djspiewak]

@rossabaker I broadly agree with your analysis. Would you recommend the same for Scala.js versions?

[rossabaker]

Naively, yes, but I am weak in both Scala.js and JavaScript and listen to its experts. My concerns about leaving semver escape routes seem applicable, but I also perceive that users can upgrade Scala.js much more casually than the Java runtime or the Scala compiler.

[dwijnand]

What's the emoji for "I vote for Scenario B"? I like the minor-bump-for-minor-bump. I don't fully follow the bullet points below it, but I'm more interested and confused about this in Scenario A:

• We can't roll back to Scala 3.0 because >= 2.6.2 already depended on it.

So why can't you roll back?

[rossabaker]

We can't roll back to Scala 3.0 because >= 2.6.2 already depended on it.

So why can't you roll back?

It depends on what we did in this step:

We unwittingly do something else that prevents a Scala 3.0 rollback.

Did we start using some new encoding that can't be rolled back binary compatibly to Scala 3.0? It's a bit unlikely for as long as Scala 2 is supported, but more plausible when code gets shifted into scala-2 and scala-3 directories.

It's all so new, I don't have good examples of what a safe or unsafe change might be. Until we all get some experience with this, maintainers need to be careful not to ring bells that can't be unrung.

[johnynek]

just want to voice my +1 for @rossabaker 's plan C.

[armanbilge]

FWIW this is the approach that scala-js-dom takes (which holds a similar place to cats in the SJS dependency tree): it was fixed on SJS 1.0.x until Scala 3 support necessitated bumping to 1.5.x, where it will sit.

[bpholt]

I'm also in favor of plan C, at least for now, just to be conservative.

If that's the plan, it's probably worth trying to coordinate the high-profile Typelevel projects. For example, it looks like fs2 has already merged 3.1, although I don't think that's been released, so it could probably be reverted.

[mpilquist]

As of now, I am not convinced we should be reverting a 3.1.0 upgrade, for cats or fs2.

[rossabaker]

Let's game out early semver: FS2 follows Scenario B and releases 3.2.0 on Scala 3.1.0. http4s-0.23.6 is on fs2-3.1.5 and would have to pin to fs2-3.1.x to maintain Scala 3.0 support.

A critical bug is found in our gzip support. (I have a notification whose title suggests this may be real!) We find the bug is in fs2, and a fix is released as fs2-3.2.1.

To fix it, http4s-0.23.7:

• bumps to fs2-3.2.1. Scala 3.0 users must undertake a completely orthogonal compiler upgrade to fix their service.
• begs for a backport as fs2-3.1.6. For a CVE, sure. For a routine bugfix, that's exactly the burden Scala 3 saves us from.
• follows suit by opening an 0.24 series. Heck, nope.

Minor release activity in patch is normal in the 0.x phase, and I suppose we'd bump fs2 and break 3.0 on 0.23. But we've impaired users' orthogonality of upgrade for ... what?

Eventually, Scala 3.0 will exert real drag on a library and it should move forward, likely under Scenario B. Doing so eagerly is an own goal.

[mpilquist]

I’m suggesting fs2 3.1.6 on 3.1.0.

[rossabaker]

Well, that's Scenario A, and if you have a CVE, people stuck on Scala 3.0 are absolutely fucked. Please don't do that.

[armanbilge]

It's not just hypothetical, right now there is a fix in typelevel/fs2#2681 which has important implications for http4s.js and is blocking http4s-dom. (Not a CVE though :)

[mpilquist]

Why are they stuck on 3.0? This has not been an issue in practice with SJS. Why is it an issue for 3.1.0 but not 3.0.2? The difference is only a tiny set of changes in the stdlib.

[johnynek]

the difference is a new compiler, which is you are compiling a lot of code may expose issues in your code, or your code may expose bugs in the compiler, which in any case can't easily be upgraded. It may be that those things rarely happen, but I think the conservative stance is to not bet the farm on it, and instead just add to CI and see how it goes for a couple of years.

What is the cost? Are you concerned about one more axis in the CI time?

I would say, until we have dropped all scala 2, this is a very modest suggestion.

[rossabaker]

Well, there's not Metals support. That's fleeting, but it's real today.

We all have a lot more history on Scala 2. Most patch upgrades have been easy. But I've been delayed by lagging compiler plugins, by new warnings (could disable fatal in a pinch), and not just once by new compiler bugs. Extreme patch upgrades have taken me weeks, which is not acceptable with a CVE.

And you're doing this for what? Because it's there?

[mpilquist]

No, both 3.1.0 and 3.0.2 are new compilers when upgrading from 3.0.0 or 3.0.1. Literally the difference is that 3.1.0 has stdlib additions. That’s it.

[armanbilge]

No, both 3.1.0 and 3.0.2 are new compilers when upgrading from 3.0.0 or 3.0.1.

That's true, but isn't the difference that a project compiled with 3.0.0 can use a dependency compiled with 3.0.2, but cannot use a dependency compiled with 3.1.0?

[mpilquist]

Yes, and in the presumably rare case where such an upgrade introduces a regression in the compiler preventing an application from upgrading, we’d have a problem. But we don’t seem to have this issue with SJS and I trust EPFL and Virtus to improve this situation for Scala 3 - perhaps by changing overall strategy with how these stdlib changes are handled. I’m objecting to any additional complexity on library authors, which are already stretched entirely too thin supporting JVM and JS, not counting Native. Proactively adding 3.x compatibility concerns is too much for me.

[rossabaker]

If it's fs2-3.2.0, there's still zero speculative complexity, but still an escape route if a critical issue does emerge. It's just the routine minor bump from a minor dependency bump.

[bplommer]

Yes, and in the presumably rare case where such an upgrade introduces a regression in the compiler preventing an application from upgrading, we’d have a problem.

It seems like these cases won't be particularly rare for applications that make use of Scala 3-specific features. In the issue on this in Shapeless 3 @joroKr21 gave scala/scala3#13406 as an example of a (still unfixed) regression in 3.0.2 that could leave someone stuck on 3.0.1 - that's ok when your dependencies are on 3.0.2, but breaks you when they move to 3.1.0. I've been flip-flopping a lot on this most of the benefits and risks of updating aren't really visible until you start using Scala 3 features in earnest, so it's worth keeping an eye on how the few libraries that do so, such as shapeless 3, approach this.

Did we start using some new encoding that can't be rolled back binary compatibly to Scala 3.0? It's a bit unlikely for as long as Scala 2 is supported, but more plausible when code gets shifted into scala-2 and scala-3 directories.

A related question that I haven't seen explicitly addressed - if a library rolls back from Scala 3.1 to Scala 3.0, does that change preserve binary compatibility? i.e. will an application compiled against the Scala 3.1 version link against the 3.0 version? Hopefully @smarter can address this?

[smarter]

A related question that I haven't seen explicitly addressed - if a library rolls back from Scala 3.1 to Scala 3.0, does that change preserve binary compatibility?

The only scenario I can imagine where this wouldn't be the case involves result type inference picking something different in each compiler version, but that can also happen in patch releases of scala 2/3 and can be remedied by giving explicit result types.

[dwijnand]

A related question that I haven't seen explicitly addressed - if a library rolls back from Scala 3.1 to Scala 3.0, does that change preserve binary compatibility? i.e. will an application compiled against the Scala 3.1 version link against the 3.0 version? Hopefully @smarter can address this?

MiMa will tell you if it's not.

[smarter]

MiMa will tell you if it's not.

One tricky aspect here is if the compiler suddenly starts generating new public code when upgrading (so mima won't say anything) like in scala/scala3#11686, then simply downgrading will break your API. Hopefully you can still generate the equivalent code by hand but that might not be straight-forward.

[johnynek]

Just a note:

https://contributors.scala-lang.org/t/scala-3-1-0-and-3-1-1-rc1-released/5294

In the meantime, we recommend testing your library with Scala 3.1.0 and 3.0.2, but publishing it with 3.0.2. This will allow downstream users to use your library even if they can’t update to Scala 3.1.0. You can find an example of build that follows this recommendation here.

[scala-steward]

Superseded by #4123.

[smarter]

Scala 3.1.2-RC1 has been released with an experimental -Yscala-release flag (documentation: https://docs.scala-lang.org/scala3/reference/language-versions/binary-compatibility.html). This will need some careful testing before we can stabilize it so please try it out and give us feedback! Note that we do have one big integration test already which builds all the dependencies of cats-effect with -Yscala-release 3.0, then builds cats-effect itself with Scala 3.0.2 and run its test suite, but it'd be interesting to have people publish actual experimental artefacts with -Yscala-release on maven and see how it behaves in the wild (until sbt learns to interpret this flag, you'll need some amount of fiddling to generate correct .pom files as noted in https://docs.scala-lang.org/scala3/reference/language-versions/binary-compatibility.html).

[armanbilge]

@smarter Thanks! We can integrate that fiddling into sbt-typelevel so that projects can easily adopt by updating to 3.1.2-RC1 or greater.

Edit: created a discussion here:

• Support for -Yscala-release flag sbt-typelevel#138