Proposal: remove the need to rely on webpack for es5 worker generation #1420

peaBerberian · 2024-04-03T10:34:28Z

A move toward a single bundler (esbuild for now)

For purposes where we need to generate a bundle (the demo, the worker files, the RxPlayer bundle builds), we're progressively removing our reliance on webpack toward esbuild.

It began with the demo page originally for performance reasons, but we found out that the esbuild behavior and API was much simpler to understand than webpack's, and so we ended up also prefering to rely on it for our worker build.

Considering that we since the v4 only rely on TypeScript with no bundling step for our "regular" build (and our home-made scripts in the ./scripts directory), we thus ended up having no reliance on webpack for most of our builds.

Issue: es5 generation and babeljs

However at #1400, we noticed that we might also need to provide an ES5 version of our worker file (at least for the PlayStation 4), and I found that having ES5 support through esbuild is not straightforward (evanw/esbuild#297).

Historically, babeljs was used to translate to ES5 - and that's the one I know, so I tried using it on the bundle made by esbuild.
However it turns out that there's no resource for how to use babel like this - not integrated as a bundler's plugin (here we just wanted to translate an ES2017 already-made bundle to es5) so I just gave up and relied on webpack for specifically that ES5 build.

However this both added some perceptible delay in our build, and we now relied on two different bundlers in our npm run build main build script, which made me un-comfortable in terms of maintainance and understanding of how our build step worked.

swc may do the trick

So I re-looked around and read about swc, which seems to be some kind of babel competitor with some credibility (though they advertises themselves as a """platform""", I would have preferred a clearer term but ok), with some differences, including speed, but what is most interesting here is that it can just be run as a standalone transpiler (what I mean: not embedded in a bundler).

By using it, I've been able to remove reliance on webpack (and babel) in our regular build steps (produced by npm run build) which should be realistically used by 99.9999% of applications (approximately!). The only exception where both webpack and babel are still used is to produce our bundles attached to each release notes (they expose the RxPlayer through window, like our grandfathers did) - which I guess are rarely relied upon.
Still, we should probably remove reliance on webpack there in the future.

Swc is maintained apparently by volunteers, and is less known than babeljs, so there's still that. Also, the bundler and transpiler JavaScript landscape is VERY active (even in JavaScript terms) right now with projects like rolldown and oxc so sadly it may be not the last update of this part of our build process.

For purposes where we need to generate a bundle (the demo, the worker files, the RxPlayer bundle builds), we're progressively removing our reliance on webpack toward esbuild. It began with the demo page originally for performance reasons, but we found out that the esbuild behavior and API was much simpler to understand than webpack's, and so we ended up also prefering to rely on it for our worker build. Considering that we since the v4 only rely on TypeScript with no bundling step for our "regular" build (and our home-made scripts in the `./scripts` directory), we thus ended up having no reliance on webpack for most of our builds. However at #1400, we noticed that we might also need to provide an ES5 version of our worker file (at least for the PlayStation 4), and I found that having ES5 support through esbuild is not straightforward (evanw/esbuild#297). Historically, [babeljs](https://babeljs.io/) was used to translate to ES5 - and that's the one I know, so I tried using it on the bundle made by esbuild. However it turns out that there's no resource for how to use babel like this - not integrated as a bundler's plugin (here we just wanted to translate an ES2017 already-made bundle to es5) so I just gave up and relied on webpack for specifically that ES5 build. However this both added some perceptible delay in our build, and we now relied on two different bundlers in our `npm run build` main build script, which made me un-comfortable in terms of maintainance and understanding of how our build step worked. So I re-looked around and read about [`swc`](https://swc.rs/), which seems to be some kind of babel competitor with some credibility (though they advertises themselves as a """platform""", I would have preferred a clearer term but ok), with some differences, including speed, but what is most interesting here is that it can just be run as a standalone transpiler (what I mean: not embedded in a bundler). By using it, I've been able to remove reliance on webpack (and babel) in our regular build steps (produced by `npm run build`) which should be realistically used by 99.9999% of applications (approximately!). The only exception where both webpack and babel are still used is to produce our bundles attached to each release notes (they expose the RxPlayer through `window`, like our grandfathers did) - which I guess are rarely relied upon. Still, we should probably remove reliance on webpack there in the future. Swc is maintained apparently by volunteers, and is less known than babeljs, so there's still that. Also, the bundler and transpiler JavaScript landscape is __VERY__ active (even in JavaScript terms) right now with projects like [rolldown](https://rolldown.rs/) and [oxc](https://oxc-project.github.io/) so sadly it may be not the last update of this part of our build process.

As a follow-up effort to #1420 (on which this work is based on), I attempt here to stop relying on webpack for the generation of our "legacy bundles", which are rarely used RxPlayer bundles that are communicated through release notes and which exposes the `RxPlayer` class through the global scope (`window`). The idea, the same than for #1420, was to simplify our bundling process by relying only on esbuild for this (instead of both webpack and esbuild depending on the situation). I chose esbuild over webpack here mainly because I better understand its role, behavior and most-of-all its API than webpack's - though its (much) faster bundling speed is a also a very good argument for this. Like for #1420, the last remaining issue was our usage of the babeljs transpiler to transpile to ES5 (our legacy bundles are ES5), but that's been fixed by switching to swc for that part. Our legacy bundle was purely configured by the `webpack.config.js` file at the root of our project that I here removed. To replace it, I re-purposed our `scripts/bundle_worker.mjs` script into a more generic `scripts/run_bundler.mjs` script for which we're now supposed to indicate the wanted input and output files. -- Still, I encountered an important issue while doing that in that I did not understand how we're supposed to indicate to esbuild that our entry file's export has to be exported through the global scope (all my introduction about the esbuild API being simpler to understand now loses all credibility!). I tried playing with its [`globalName`](https://esbuild.github.io/api/#global-name) and [`format`](https://esbuild.github.io/api/#format) configuration options which seem linked to that but couldn't make it work. For now, I gave up, adding a `__GLOBAL_SCOPE__` compile-time constant (boolean) to `src/index.ts` which has to be set to `true` before bundling if you want to export the RxPlayer through the global scope.

peaBerberian · 2024-04-09T16:15:34Z

Well, it turns out that SWC rely on Object.getOwnPropertyDescriptors being present which is an ES2017 feature and thus the playstation 4 cannot use the outputed script.

I don't understand how I'm supposed to tell swc to just provide ES5-compatible polyfills by itself.
I could not make sense of their API documentation for this (target: "es5" is not sufficient and could not arrive to make "env.mode" working as intended) and I spent way too much time (in my defense I'm not that much used to relying on those application-side tools) for something that is """just""" planned for potential experience improvement on the playstation 4.

I'll propose to abandon multithread support for the playstation 4 instead through another PR instead.
If an application does want that much to obtain an ES5-compatible worker file, they can rely on the worker.js file we provide in release notes and do the transpiling themselves, so it's still a possibility (though nothing that I think that will realistically be done due to how complex that would be to maintain).

@Florent-Bouisset

This is built on top of #1425 itself built on top of #1420, which removes most dependencies to webpack (besides tests, that we'll need to also change anyway considering the fact that `karma-webpack` has an old reported issue on macOS with our build). We planned to release in our future 4.1.0 an ES5 version of our worker file and as a lesser feature an ES5 "legacy" build (the one linked to release notes - not the one published on npm). Providing an ES5 version of our worker file was especially added to support the `MULTI_THREAD` feature on PlayStation 4 devices. However I sadly propose here that we roll back that attempt of support (just of the `MULTI_THREAD` feature, the PlayStation 4 stay officially supported and tested). --- After multiple attempts (myself and then @Florent-Bouisset), we realized the complexities and costs of maintaining a supplementary ES5 version of builds. We initially tried to switch from a webpack+babel mix to swc on a bundle already produced by esbuild. The global idea was to simplify our bundling process by designing an architecturally simple and low-maintainance process made of a separate bundling step then transpiling step on the bundling result. But we realized that this philosophy doesn't seem to be compatible to those tools. When combined with the fact that their documentation is often either lacking or too complex, than some of our questionning to maintainers led to poor answers and that the JS tooling ecosystem appears to be still changing very fast, I'm not sure that I want to guarantee support of ES5 for our hopefully long-lived v4 right now.

@Florent-Bouisset

This is built on top of #1425 itself built on top of #1420, which removes most dependencies to webpack (besides tests, that we'll need to also change anyway considering the fact that `karma-webpack` has an old reported issue on macOS with our build). We planned to release in our future 4.1.0 an ES5 version of our worker file and as a lesser feature an ES5 "legacy" build (the one linked to release notes - not the one published on npm). Providing an ES5 version of our worker file was especially added to support the `MULTI_THREAD` feature on PlayStation 4 devices. However I sadly propose here that we roll back that attempt of support (just of the `MULTI_THREAD` feature, the PlayStation 4 stay officially supported and tested). --- After multiple attempts (myself and then @Florent-Bouisset), we realized the complexities and costs of maintaining a supplementary ES5 version of builds. We initially tried to switch from a webpack+babel mix to swc on a bundle already produced by esbuild. The global idea was to simplify our bundling process by designing an architecturally simple and low-maintainance process made of a separate bundling step then transpiling step on the bundling result. But we realized that this philosophy doesn't seem to be compatible to those tools. When combined with the fact that their documentation is often either lacking or too complex, than some of our questionning to maintainers led to poor answers and that the JS tooling ecosystem appears to be still changing very fast, I'm not sure that I want to guarantee support of ES5 for our hopefully long-lived v4 right now.

@Florent-Bouisset

This is built on top of #1425 itself built on top of #1420, which removes most dependencies to webpack (besides tests, that we'll need to also change anyway considering the fact that `karma-webpack` has an old reported issue on macOS with our build). We planned to release in our future 4.1.0 an ES5 version of our worker file and as a lesser feature an ES5 "legacy" build (the one linked to release notes - not the one published on npm). Providing an ES5 version of our worker file was especially added to support the `MULTI_THREAD` feature on PlayStation 4 devices. However I sadly propose here that we roll back that attempt of support (just of the `MULTI_THREAD` feature, the PlayStation 4 stay officially supported and tested). --- After multiple attempts (myself and then @Florent-Bouisset), we realized the complexities and costs of maintaining a supplementary ES5 version of builds. We initially tried to switch from a webpack+babel mix to swc on a bundle already produced by esbuild. The global idea was to simplify our bundling process by designing an architecturally simple and low-maintainance process made of a separate bundling step then transpiling step on the bundling result. But we realized that this philosophy doesn't seem to be compatible to those tools. When combined with the fact that their documentation is often either lacking or too complex, than some of our questionning to maintainers led to poor answers and that the JS tooling ecosystem appears to be still changing very fast, I'm not sure that I want to guarantee support of ES5 for our hopefully long-lived v4 right now.

@Florent-Bouisset

This is built on top of #1425 itself built on top of #1420, which removes most dependencies to webpack (besides tests, that we'll need to also change anyway considering the fact that `karma-webpack` has an old reported issue on macOS with our build). We planned to release in our future 4.1.0 an ES5 version of our worker file and as a lesser feature an ES5 "legacy" build (the one linked to release notes - not the one published on npm). Providing an ES5 version of our worker file was especially added to support the `MULTI_THREAD` feature on PlayStation 4 devices. However I sadly propose here that we roll back that attempt of support (just of the `MULTI_THREAD` feature, the PlayStation 4 stay officially supported and tested). --- After multiple attempts (myself and then @Florent-Bouisset), we realized the complexities and costs of maintaining a supplementary ES5 version of builds. We initially tried to switch from a webpack+babel mix to swc on a bundle already produced by esbuild. The global idea was to simplify our bundling process by designing an architecturally simple and low-maintainance process made of a separate bundling step then transpiling step on the bundling result. But we realized that this philosophy doesn't seem to be compatible to those tools. When combined with the fact that their documentation is often either lacking or too complex, than some of our questionning to maintainers led to poor answers and that the JS tooling ecosystem appears to be still changing very fast, I'm not sure that I want to guarantee support of ES5 for our hopefully long-lived v4 right now.

peaBerberian · 2024-05-06T15:17:43Z

For now closed in profit of #1435

As a follow-up effort to #1420 (on which this work is based on), I attempt here to stop relying on webpack for the generation of our "legacy bundles", which are rarely used RxPlayer bundles that are communicated through release notes and which exposes the `RxPlayer` class through the global scope (`window`). The idea, the same than for #1420, was to simplify our bundling process by relying only on esbuild for this (instead of both webpack and esbuild depending on the situation). I chose esbuild over webpack here mainly because I better understand its role, behavior and most-of-all its API than webpack's - though its (much) faster bundling speed is a also a very good argument for this. Like for #1420, the last remaining issue was our usage of the babeljs transpiler to transpile to ES5 (our legacy bundles are ES5), but that's been fixed by switching to swc for that part. Our legacy bundle was purely configured by the `webpack.config.js` file at the root of our project that I here removed. To replace it, I re-purposed our `scripts/bundle_worker.mjs` script into a more generic `scripts/run_bundler.mjs` script for which we're now supposed to indicate the wanted input and output files. -- Still, I encountered an important issue while doing that in that I did not understand how we're supposed to indicate to esbuild that our entry file's export has to be exported through the global scope (all my introduction about the esbuild API being simpler to understand now loses all credibility!). I tried playing with its [`globalName`](https://esbuild.github.io/api/#global-name) and [`format`](https://esbuild.github.io/api/#format) configuration options which seem linked to that but couldn't make it work. For now, I gave up, adding a `__GLOBAL_SCOPE__` compile-time constant (boolean) to `src/index.ts` which has to be set to `true` before bundling if you want to export the RxPlayer through the global scope.

@Florent-Bouisset

This is built on top of #1425 itself built on top of #1420, which removes most dependencies to webpack (besides tests, that we'll need to also change anyway considering the fact that `karma-webpack` has an old reported issue on macOS with our build). We planned to release in our future 4.1.0 an ES5 version of our worker file and as a lesser feature an ES5 "legacy" build (the one linked to release notes - not the one published on npm). Providing an ES5 version of our worker file was especially added to support the `MULTI_THREAD` feature on PlayStation 4 devices. However I sadly propose here that we roll back that attempt of support (just of the `MULTI_THREAD` feature, the PlayStation 4 stay officially supported and tested). --- After multiple attempts (myself and then @Florent-Bouisset), we realized the complexities and costs of maintaining a supplementary ES5 version of builds. We initially tried to switch from a webpack+babel mix to swc on a bundle already produced by esbuild. The global idea was to simplify our bundling process by designing an architecturally simple and low-maintainance process made of a separate bundling step then transpiling step on the bundling result. But we realized that this philosophy doesn't seem to be compatible to those tools. When combined with the fact that their documentation is often either lacking or too complex, than some of our questionning to maintainers led to poor answers and that the JS tooling ecosystem appears to be still changing very fast, I'm not sure that I want to guarantee support of ES5 for our hopefully long-lived v4 right now.

As a follow-up effort to #1420 (on which this work is based on), I attempt here to stop relying on webpack for the generation of our "legacy bundles", which are rarely used RxPlayer bundles that are communicated through release notes and which exposes the `RxPlayer` class through the global scope (`window`). The idea, the same than for #1420, was to simplify our bundling process by relying only on esbuild for this (instead of both webpack and esbuild depending on the situation). I chose esbuild over webpack here mainly because I better understand its role, behavior and most-of-all its API than webpack's - though its (much) faster bundling speed is a also a very good argument for this. Like for #1420, the last remaining issue was our usage of the babeljs transpiler to transpile to ES5 (our legacy bundles are ES5), but that's been fixed by switching to swc for that part. Our legacy bundle was purely configured by the `webpack.config.js` file at the root of our project that I here removed. To replace it, I re-purposed our `scripts/bundle_worker.mjs` script into a more generic `scripts/run_bundler.mjs` script for which we're now supposed to indicate the wanted input and output files. -- Still, I encountered an important issue while doing that in that I did not understand how we're supposed to indicate to esbuild that our entry file's export has to be exported through the global scope (all my introduction about the esbuild API being simpler to understand now loses all credibility!). I tried playing with its [`globalName`](https://esbuild.github.io/api/#global-name) and [`format`](https://esbuild.github.io/api/#format) configuration options which seem linked to that but couldn't make it work. For now, I gave up, adding a `__GLOBAL_SCOPE__` compile-time constant (boolean) to `src/index.ts` which has to be set to `true` before bundling if you want to export the RxPlayer through the global scope.

@Florent-Bouisset

This is built on top of #1425 itself built on top of #1420, which removes most dependencies to webpack (besides tests, that we'll need to also change anyway considering the fact that `karma-webpack` has an old reported issue on macOS with our build). We planned to release in our future 4.1.0 an ES5 version of our worker file and as a lesser feature an ES5 "legacy" build (the one linked to release notes - not the one published on npm). Providing an ES5 version of our worker file was especially added to support the `MULTI_THREAD` feature on PlayStation 4 devices. However I sadly propose here that we roll back that attempt of support (just of the `MULTI_THREAD` feature, the PlayStation 4 stay officially supported and tested). --- After multiple attempts (myself and then @Florent-Bouisset), we realized the complexities and costs of maintaining a supplementary ES5 version of builds. We initially tried to switch from a webpack+babel mix to swc on a bundle already produced by esbuild. The global idea was to simplify our bundling process by designing an architecturally simple and low-maintainance process made of a separate bundling step then transpiling step on the bundling result. But we realized that this philosophy doesn't seem to be compatible to those tools. When combined with the fact that their documentation is often either lacking or too complex, than some of our questionning to maintainers led to poor answers and that the JS tooling ecosystem appears to be still changing very fast, I'm not sure that I want to guarantee support of ES5 for our hopefully long-lived v4 right now.

peaBerberian added the proposal This Pull Request or Issue is only a proposal for a change with the expectation of a debate on it label Apr 3, 2024

peaBerberian mentioned this pull request Apr 4, 2024

Proposal: Stop relying on webpack for legacy bundles generation #1425

Closed

peaBerberian added work-in-progress This Pull Request or issue is not finished yet Priority: 3 (Low) This issue or PR has a low priority. labels Apr 10, 2024

peaBerberian mentioned this pull request Apr 30, 2024

[Proposal] Stop transpiling some builds to ES5 #1435

Merged

peaBerberian closed this May 6, 2024

peaBerberian mentioned this pull request Jun 13, 2024

Release/v4.1.0 #1456

Merged

peaBerberian deleted the misc/remove-webpack-from-worker-bundling branch July 26, 2024 16:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Proposal: remove the need to rely on webpack for es5 worker generation #1420

Proposal: remove the need to rely on webpack for es5 worker generation #1420

peaBerberian commented Apr 3, 2024

peaBerberian commented Apr 9, 2024

peaBerberian commented May 6, 2024

Proposal: remove the need to rely on webpack for es5 worker generation #1420

Proposal: remove the need to rely on webpack for es5 worker generation #1420

Conversation

peaBerberian commented Apr 3, 2024

A move toward a single bundler (esbuild for now)

Issue: es5 generation and babeljs

swc may do the trick

peaBerberian commented Apr 9, 2024

peaBerberian commented May 6, 2024