update scalajs-dom to 2.0.0 #377

hugo-vrijswijk · 2021-10-26T17:13:43Z

Update scalajs-dom to 2.0.0. This version is not binary compatible with 1.2.0. I was running into an issue where some of my other dependencies had upgraded, but ST hadn't, resulting in an eviction error from sbt.

I've mostly followed the migration guide:

Change version number
Remove .for3Use2_13
Run script to migrate most imports
Run find-replace on dom.raw, dom.experimental, dom.crypto to replace with dom imports
Test on a local project

Two pre-release versions are used to be able to use scalajs-dom 2 on tests projects:

japgolly react 2.0.0-RC4. Find-replace when final version is released
slinky 0.6.8+1-b1dd3f4c. Find-replace when final version is released

I realize the changelog for this is massive. But because of the changed imports there's not really a way around it. When someone reviews this, perhaps there's a way to filter out certain diffs (like anything that starts with scalajs.dom.?

oyvindberg · 2021-10-26T21:11:06Z

...s/src/main/scala/org/scalablytyped/converter/internal/scalajs/flavours/ScalaJsDomNames.scala

+      CastConversion(stdNames.lib + Name("ChannelSplitterNode"), QualifiedName("org.scalajs.dom.ChannelSplitterNode")),
+      CastConversion(stdNames.lib + Name("CharacterData"), QualifiedName("org.scalajs.dom.CharacterData")),
+      CastConversion(stdNames.lib + Name("ClientQueryOptions"), QualifiedName("org.scalajs.dom.ClientQueryOptions")),
+      CastConversion(stdNames.lib + Name("DOMRect"), QualifiedName("org.scalajs.dom.DOMRect")),


This file is my biggest concern actually, What you're looking at is a table of named DOM types in typescript and in scalajs-dom used for translation. I generated this at some point.

I'd like to confirm that all the types referenced on the right hand side, for instance org.scalajs.dom.DOMRect actually exist. Do you have any script foo to do that? otherwise I'll get to it eventually :)

Maybe the auto-generated API reports can be of some use for this? https://github.com/scala-js/scala-js-dom/blob/master/api-reports/2_13.txt

oyvindberg · 2021-10-26T21:13:43Z

Thank you, most of the work is done now! :)

Gotta love the scala churn, it never stops. This change is entangled with the scalajs-react major version, which means this PR is entangled with #326 . I rebased that on top of your PR, and all the tests seem to pass, which is good.

That PR is setup to publish snapshots, so we'll publish snapshots of the two PRs combined until both slinky and scalajs-react have stable releases. Hopefully that will be soon.

diffs

The diff is huge because this project uses snapshot tests - all of those files under tests/ are generated, and enforced by CI that they are up to date for each build. just running sbt test will update all of them.

Likewise the huge files underneath import-scalajs-definitions/ are also generated, and can be regenerated with sbt import-scalajs-definitions/run

oyvindberg · 2021-10-26T21:14:58Z

Snapshot published as 1.0.0-beta36+27-dc516c4d-SNAPSHOT, it needs this resolver to work:
resolvers += MavenRepository("sonatype-s01-snapshots", "https://s01.oss.sonatype.org/content/repositories/snapshots")

armanbilge · 2021-10-26T22:19:46Z

Thank you both so much for taking this on, much much appreciated and sincere apologies for the churn :( let me know if there's something I can help with.

armanbilge · 2021-10-27T02:28:08Z

Ok, so I used this regex to extract the scala-js-dom types:

QualifiedName\("org\.scalajs\.dom\.(.*)"\)

which I made into a list of urls which I checked the HTTP status of:

while read LINE; do
  curl -o /dev/null --silent --head --write-out "%{http_code} $LINE\n" "$LINE"
done

giving me this list: https://gist.githubusercontent.com/armanbilge/0637566f9babf52986e9dea6dd67b630/raw/30b4f9eb557fbae5393827423f6c2eedb7b38a63/gistfile1.txt.

There were some 404s in there, but many of those were files moved to scala-2/3 split sources: https://gist.githubusercontent.com/armanbilge/0637566f9babf52986e9dea6dd67b630/raw/943426f38c7249da596d5d261a43467f3141bcca/gistfile2.txt

So in the end, we are left with:

I think these are type aliases or various other things, so I'm going to track them down and make suggestions on the PR if any need fixing.

...s/src/main/scala/org/scalablytyped/converter/internal/scalajs/flavours/ScalaJsDomNames.scala

oyvindberg · 2021-10-27T06:08:07Z

Thank you @armanbilge , exactly what I had in mind we would check :)

rpiaggio · 2021-10-30T17:09:41Z

Snapshot published as 1.0.0-beta36+27-dc516c4d-SNAPSHOT, it needs this resolver to work: resolvers += MavenRepository("sonatype-s01-snapshots", "https://s01.oss.sonatype.org/content/repositories/snapshots")

Would it be straightforward to release a snapshot depending on scalajs-react 2.0.0-RC5 ?

My deepest thanks to everyone involved.

rpiaggio · 2021-10-30T17:19:04Z

Hmm. I realize we would need a snapshot that also integrates this PR in order to be able to migrate to scalajs-dom 2.

armanbilge · 2021-10-30T17:22:00Z

@rpiaggio #326 is based on this PR, so the snapshot published from there against scalajs-react 2.0.0-RC4 works with sjs-dom v2. In fact, I would have expected it to work for RC5 unless there are breaking changes?

rpiaggio · 2021-10-30T17:43:12Z

Thank you for the quick response!

@rpiaggio #326 is based on this PR, so the snapshot published from there against scalajs-react 2.0.0-RC4 works with sjs-dom v2. In fact, I would have expected it to work for RC5 unless there are breaking changes?

I'm doing ThisBuild / libraryDependencySchemes += "com.github.japgolly.scalajs-react" %% "core_sjs1" % "always" to avoid the version conflict error betweekn RC4 and RC5; but then I get an error when compiling this ST generated class:

std/ParentNode.scala:305:17: class HTMLCollection takes type parameters
  val children: HTMLCollection = js.native

HTMLCollection is org.scalajs.dom.HTMLCollection.

We have stUseScalaJsDom := true, but setting it to false makes no difference.

armanbilge · 2021-10-30T17:46:26Z

Thanks for reporting, that's because of the changes in scala-js/scala-js-dom#593. It looks like the generator in ST will have to be adjusted to accommodate for that.

I am surprised that setting stUseScalaJsDom := false ~~doesn't work~~ doesn't at least generate a different error, though.

Edit: ah, it's probably to due with the flavour setting. The ST docs say that

Also note that if you use a flavour which translates types to scala-js-dom types, only types found in the chosen stdlib will be translated.

oyvindberg · 2021-10-30T22:33:30Z

I wrote a test to enforce that all right hand sides in that conversion tables exist, and that the number of type parameters matches. Then I fixed the few things which showed up. The biggest surprise was probably that IDBRequest has a type parameter which TS doesn't.

For reference, here is a list of types in scala-js-dom which ST will not translate to currently. I don't intend to add any of those to the conversion list now, but anyone can certainly feel free:

somewhat long list

org.scalajs.dom.AppendMode
org.scalajs.dom.BigInteger
org.scalajs.dom.BufferSource
org.scalajs.dom.CanvasProxy
org.scalajs.dom.Chunk
org.scalajs.dom.Client
org.scalajs.dom.ClientRect
org.scalajs.dom.ClientRectList
org.scalajs.dom.ClientType
org.scalajs.dom.Clients
org.scalajs.dom.Clipboard
org.scalajs.dom.CompositionEventInit
org.scalajs.dom.ConvertToBlobOptions
org.scalajs.dom.CreateImageBitmapOptions
org.scalajs.dom.CustomEventInit
org.scalajs.dom.DOMList
org.scalajs.dom.DedicatedWorkerGlobalScope
org.scalajs.dom.DocumentReadyState
org.scalajs.dom.EndOfStreamError
org.scalajs.dom.EventException
org.scalajs.dom.EventInit
org.scalajs.dom.ExtendableEvent
org.scalajs.dom.ExtendableEventInit
org.scalajs.dom.ExtendableMessageEvent
org.scalajs.dom.ExtendableMessageEventInit
org.scalajs.dom.FetchEvent
org.scalajs.dom.FetchEventInit
org.scalajs.dom.FileReaderSync
org.scalajs.dom.FocusEventInit
org.scalajs.dom.FrameType
org.scalajs.dom.FullscreenOptions
org.scalajs.dom.GetSVGDocument
org.scalajs.dom.HTMLFormControlsCollection
org.scalajs.dom.HTMLOptionsCollection
org.scalajs.dom.HTMLTableAlignment
org.scalajs.dom.HashAlgorithm
org.scalajs.dom.HeadersInit
org.scalajs.dom.HttpMethod
org.scalajs.dom.IDBCreateIndexOptions
org.scalajs.dom.IDBCreateObjectStoreOptions
org.scalajs.dom.IDBCursorDirection
org.scalajs.dom.IDBCursorReadOnly
org.scalajs.dom.IDBDatabaseInfo
org.scalajs.dom.IDBEvent
org.scalajs.dom.IDBEventTarget
org.scalajs.dom.IDBRequestReadyState
org.scalajs.dom.IDBStoreLike
org.scalajs.dom.IDBTransactionDurability
org.scalajs.dom.IDBTransactionMode
org.scalajs.dom.IDBTransactionOptions
org.scalajs.dom.IDBVersionChangeEventInit
org.scalajs.dom.ImageBitmap
org.scalajs.dom.ImageCapture
org.scalajs.dom.MediaQueryListListener
org.scalajs.dom.MediaSource
org.scalajs.dom.MediaSourceReadyState
org.scalajs.dom.MediaStreamEventInit
org.scalajs.dom.MessageEventInit
org.scalajs.dom.ModifierKeyEvent
org.scalajs.dom.ModifierKeyEventInit
org.scalajs.dom.MouseEventInit
org.scalajs.dom.NavigatorGeolocation
org.scalajs.dom.NavigatorStorageUtils
org.scalajs.dom.NavigatorVibration
org.scalajs.dom.NodeListOf
org.scalajs.dom.NodeSelector
org.scalajs.dom.OffscreenCanvas
org.scalajs.dom.OpenEndedDictionary
org.scalajs.dom.PageTransitionEvent
org.scalajs.dom.PageVisibility
org.scalajs.dom.PermissionDescriptor
org.scalajs.dom.PermissionName
org.scalajs.dom.PermissionState
org.scalajs.dom.PermissionStatus
org.scalajs.dom.Permissions
org.scalajs.dom.PushEvent
org.scalajs.dom.PushMessageData
org.scalajs.dom.PushPermissionDescriptor
org.scalajs.dom.RadioNodeList
org.scalajs.dom.ReadableStreamController
org.scalajs.dom.RequestType
org.scalajs.dom.SVGAnimatedPathData
org.scalajs.dom.SVGException
org.scalajs.dom.SVGExternalResourcesRequired
org.scalajs.dom.SVGLangSpace
org.scalajs.dom.SVGLocatable
org.scalajs.dom.SVGStylable
org.scalajs.dom.SVGTransformable
org.scalajs.dom.SVGZoomEventInit
org.scalajs.dom.Sequence
org.scalajs.dom.ServiceWorkerGlobalScope
org.scalajs.dom.SharedWorker
org.scalajs.dom.SharedWorkerGlobalScope
org.scalajs.dom.SourceBuffer
org.scalajs.dom.SourceBufferList
org.scalajs.dom.StorageEstimate
org.scalajs.dom.StorageEventInit
org.scalajs.dom.StorageManager
org.scalajs.dom.TextEventInit
org.scalajs.dom.TextTrackMode
org.scalajs.dom.TouchEventInit
org.scalajs.dom.Transferable
org.scalajs.dom.TwoDContextAttributes
org.scalajs.dom.UIEventInit
org.scalajs.dom.URLSearchParams
org.scalajs.dom.VideoTrack
org.scalajs.dom.VideoTrackList
org.scalajs.dom.WheelEventInit
org.scalajs.dom.WindowClient
org.scalajs.dom.WindowOrWorkerGlobalScope
org.scalajs.dom.WorkerGlobalScope
org.scalajs.dom.WorkerLocation
org.scalajs.dom.WorkerNavigator
org.scalajs.dom.WriteableState
org.scalajs.dom.WriteableStream
org.scalajs.dom.XPathNSResolver
org.scalajs.dom.webgl.ActiveInfo
org.scalajs.dom.webgl.Buffer
org.scalajs.dom.webgl.ContextAttributes
org.scalajs.dom.webgl.Framebuffer
org.scalajs.dom.webgl.Program
org.scalajs.dom.webgl.Renderbuffer
org.scalajs.dom.webgl.RenderingContext
org.scalajs.dom.webgl.Shader
org.scalajs.dom.webgl.ShaderPrecisionFormat
org.scalajs.dom.webgl.Texture
org.scalajs.dom.webgl.UniformLocation
org.scalajs.dom.webgl.extensions.ANGLEInstancedArrays
org.scalajs.dom.webgl.extensions.EXTBlendMinmax
org.scalajs.dom.webgl.extensions.EXTColorBufferHalfFloat
org.scalajs.dom.webgl.extensions.EXTDisjointTimerQuery
org.scalajs.dom.webgl.extensions.EXTFragDepth
org.scalajs.dom.webgl.extensions.EXTShaderTextureLOD
org.scalajs.dom.webgl.extensions.EXTTextureFilterAnisotropic
org.scalajs.dom.webgl.extensions.EXTsRGB
org.scalajs.dom.webgl.extensions.OESElementIndexUint
org.scalajs.dom.webgl.extensions.OESStandardDerivatives
org.scalajs.dom.webgl.extensions.OESTextureFloat
org.scalajs.dom.webgl.extensions.OESTextureFloatLinear
org.scalajs.dom.webgl.extensions.OESTextureHalfFloat
org.scalajs.dom.webgl.extensions.OESTextureHalfFloatLinear
org.scalajs.dom.webgl.extensions.OESVertexArrayObject
org.scalajs.dom.webgl.extensions.WEBGLColorBufferFloat
org.scalajs.dom.webgl.extensions.WEBGLCompressedTextureETC1
org.scalajs.dom.webgl.extensions.WEBGLDrawBuffers
org.scalajs.dom.webgl.extensions.WebGLCompressedTextureATC
org.scalajs.dom.webgl.extensions.WebGLCompressedTexturePVRTC
org.scalajs.dom.webgl.extensions.WebGLCompressedTextureS3TC
org.scalajs.dom.webgl.extensions.WebGLDebugRendererInfo
org.scalajs.dom.webgl.extensions.WebGLDebugShaders
org.scalajs.dom.webgl.extensions.WebGLDepthTexture
org.scalajs.dom.webgl.extensions.WebGLLoseContext
org.scalajs.dom.webgl.extensions.WebGLTimerQueryEXT
org.scalajs.dom.webgl.extensions.WebGLVertexArrayObjectOES
org.scalajs.dom.webworkers.AbstractWorker
org.scalajs.dom.webworkers.DedicatedWorkerGlobalScope
org.scalajs.dom.webworkers.Worker
org.scalajs.dom.webworkers.WorkerGlobalScope
org.scalajs.dom.webworkers.WorkerLocation
org.scalajs.dom.webworkers.WorkerNavigator

I also bumped #326 to use 2.0.0-M5, a snapshot if hopefully on the way

armanbilge · 2021-10-30T22:38:30Z

The biggest surprise was probably that IDBRequest has a type parameter which TS doesn't.

Yes, I think that was done in scala-js/scala-js-dom#569. Hope it didn't cause too much trouble 😅

Quite a list there!

oyvindberg · 2021-10-30T22:59:41Z

No, not a problem at all @armanbilge :)

snapshot published with version ~~1.0.0-beta36+29-f3b7dd30-SNAPSHOT~~ edit: use 1.0.0-beta36+41-906a0cad-SNAPSHOT

hugo-vrijswijk · 2021-10-31T14:17:43Z

Thanks for the extensive review and comments. I might have bitten off more than I can chew with this PR, as my SJS and ST experience are limited, and this PR grew a lot. I see the build is still red, but I'm not sure what the exact reason for it is and what still needs to be done for this PR to be merged 😅

armanbilge · 2021-10-31T14:20:24Z

@hugo-vrijswijk as previously mentioned this PR is entangled with #326 which IIUC means they need to be merged at the same time (this one cannot standalone). Given that #326 is rebased against this, the fact that it is green there is a good sign :)

rpiaggio · 2021-10-31T15:27:00Z

snapshot published with version 1.0.0-beta36+29-f3b7dd30-SNAPSHOT

Awesome, our library compiles with this version. Thank you very much!

ngbinh · 2021-11-07T09:23:12Z

scala.js react 2.0.0 final has been released with not much changes https://github.com/japgolly/scalajs-react/blob/master/doc/changelog/2.0.0-RCs.md#changes-in-between-rc5-and-200-final

oyvindberg · 2021-11-07T16:50:24Z

Now we only need a stable version of slinky :)

Co-authored-by: Arman Bilge <armanbilge@gmail.com>

…erify number of type parameters.

oyvindberg · 2021-12-15T22:09:32Z

Ok, tired of being blocked. This is going in anyway, and I'll release when possible. with a slinky snapshot version if necessary

hugo-vrijswijk marked this pull request as draft October 26, 2021 17:52

oyvindberg reviewed Oct 26, 2021

View reviewed changes

oyvindberg mentioned this pull request Oct 26, 2021

Build/test against sjs-dom 2.0.0-RC1 #373

Closed

armanbilge mentioned this pull request Oct 26, 2021

Establish series/0.2 against sjs-dom 2.x http4s/http4s-dom#28

Merged

armanbilge suggested changes Oct 27, 2021

View reviewed changes

hugo-vrijswijk marked this pull request as ready for review November 2, 2021 08:51

hugo-vrijswijk and others added 5 commits November 9, 2021 23:38

update scala-js-dom to 2.0.0

913b965

slinky update

e42dca3

Apply suggestions from code review

b167086

Co-authored-by: Arman Bilge <armanbilge@gmail.com>

regen definitions

796d608

Check that conversions to scalajs-dom refers to existing types, and v…

05fde79

…erify number of type parameters.

oyvindberg merged commit b3efa41 into ScalablyTyped:master Dec 15, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update scalajs-dom to 2.0.0 #377

update scalajs-dom to 2.0.0 #377

hugo-vrijswijk commented Oct 26, 2021 •

edited

Loading

oyvindberg Oct 26, 2021

armanbilge Oct 26, 2021

oyvindberg commented Oct 26, 2021

oyvindberg commented Oct 26, 2021

armanbilge commented Oct 26, 2021

armanbilge commented Oct 27, 2021 •

edited

Loading

oyvindberg commented Oct 27, 2021

rpiaggio commented Oct 30, 2021

rpiaggio commented Oct 30, 2021

armanbilge commented Oct 30, 2021

rpiaggio commented Oct 30, 2021

armanbilge commented Oct 30, 2021 •

edited

Loading

oyvindberg commented Oct 30, 2021 •

edited

Loading

armanbilge commented Oct 30, 2021

oyvindberg commented Oct 30, 2021 •

edited

Loading

hugo-vrijswijk commented Oct 31, 2021

armanbilge commented Oct 31, 2021

rpiaggio commented Oct 31, 2021

ngbinh commented Nov 7, 2021

oyvindberg commented Nov 7, 2021

oyvindberg commented Dec 15, 2021

update scalajs-dom to 2.0.0 #377

update scalajs-dom to 2.0.0 #377

Conversation

hugo-vrijswijk commented Oct 26, 2021 • edited Loading

oyvindberg Oct 26, 2021

Choose a reason for hiding this comment

armanbilge Oct 26, 2021

Choose a reason for hiding this comment

oyvindberg commented Oct 26, 2021

diffs

oyvindberg commented Oct 26, 2021

armanbilge commented Oct 26, 2021

armanbilge commented Oct 27, 2021 • edited Loading

oyvindberg commented Oct 27, 2021

rpiaggio commented Oct 30, 2021

rpiaggio commented Oct 30, 2021

armanbilge commented Oct 30, 2021

rpiaggio commented Oct 30, 2021

armanbilge commented Oct 30, 2021 • edited Loading

oyvindberg commented Oct 30, 2021 • edited Loading

armanbilge commented Oct 30, 2021

oyvindberg commented Oct 30, 2021 • edited Loading

hugo-vrijswijk commented Oct 31, 2021

armanbilge commented Oct 31, 2021

rpiaggio commented Oct 31, 2021

ngbinh commented Nov 7, 2021

oyvindberg commented Nov 7, 2021

oyvindberg commented Dec 15, 2021

hugo-vrijswijk commented Oct 26, 2021 •

edited

Loading

armanbilge commented Oct 27, 2021 •

edited

Loading

armanbilge commented Oct 30, 2021 •

edited

Loading

oyvindberg commented Oct 30, 2021 •

edited

Loading

oyvindberg commented Oct 30, 2021 •

edited

Loading