WebWorker support?

[ebrensi]

Hi I just discovered esbuild, coming from Parcel. One thing I like about parcel is that if I instaniate a WebWorker with the string literal filename, like

const ww = new Worker ('./myWorker.js')
ww.postMessage('work!')

Parcel will recognize the Worker constructor and create another bundle starting at ./myWorker.js. It also handles cache busting for the filename. So the bundle would be called ./myWorker8a68r8912q.js or something and that string would be updated in the code above.

Does ESBuild do something like that? If not, where would I look to implement that?

[evanw]

Oh, interesting. That's pretty cool. I didn't know about that feature of Parcel. I assume this also works for the SharedWorker constructor. You should be able to get this to work with the current version of esbuild by just adding myWorker.js as an additional entry point in the list of entry points passed to esbuild.

However, the cache busting part won't work yet. None of the entry points for esbuild are cache-busted right now. I'm working on enabling this as part of general code splitting support (see #16) but it's still a work in progress. Introducing cache busting in entry point names is also a breaking change and I've been waiting to do this until the next batch of breaking changes. Once that's in, I think it should be relatively straightforward to make something like this work.

[ggoodman]

This sort of feature is sorely needed in Rollup and can be achieved in Webpack with something like worker-plugin.

Having this built into esbuild as an opt-in would be absolutely fantastic! I think deferring to plugins would likely not do it 'right' as these would need to parse an AST, figure out bindings, etc.. (or just RegEx and yolo).

[ggoodman]

There are other web apis that would benefit from this form of non require, non import()-based code splitting like audio worklets.

[rtsao]

I think this would be a really nice feature.

As a stopgap, I was thinking this could be implemented as an esbuild plugin. This would impose some rather unfortunate syntax, but in a pinch, something like this would work:

import workerUrl from "workerUrl(./worker.js)";

const worker = new Worker(workerUrl)

import path from "path";
import { build } from "esbuild";

let workerLoader = (plugin) => {
  plugin.setName("worker-loader");
  plugin.addResolver({ filter: /^workerUrl\((.+)\)/ }, (args) => {
    return { path: args.path, namespace: "workerUrl" };
  });
  plugin.addLoader(
    { filter: /^workerUrl\((.+)\)/, namespace: "workerUrl" },
    async (args) => {
      let match = /^workerUrl\((.+)\)/.exec(args.path),
        workerPath = match[1];
      let outfile = path.join("dist", path.basename(workerPath));
      try {
        // bundle worker entry in a sub-process
        await build({
          entryPoints: [workerPath],
          outfile,
          minify: true,
          bundle: true,
        });

        // return the bundled path
        return { contents: `export default ${JSON.stringify(workerPath)};` };
      } catch (e) {
        // ...
      }
    }
  );
};

[blixt]

Besides path resolution within new Worker(…) etc, it would also be nice to enable an entrypoint to be generated as a worker script, which means it will not use import * as x from "./x.js" but rather importScripts("./x.js") – at least until all browsers implement new Worker(…, { type: "module" }) (whatwg/html#550).

[gkjohnson]

As of version 5.x webpack can bundle web workers out of the box with the following syntax:

new Worker( new URL( "./worker", import.meta.url ), { type: "module" } )

import.meta.url is needed so the worker file is explicitly loaded relative to the current file. Otherwise it's loaded relative to the href of the page. And of course type: module is needed in order to optionally support modules. IMO it's the right way to support bundling Workers that because it promotes practices that work in browsers, as well. Parcel doesn't support this yet but I'm still hoping they will in v2.

This bundler looks great and I'd really look forward to seeing this feature added! It's been too hard to use WebWorkers generically for too long...

[endreymarcell]

Update: I created a minimal demonstration for how this plugin would work at https://github.com/endreymarcell/esbuild-plugin-webworker

An important difference to Webpack is that this one does not inline the code of the web worker script into the main bundle. Instead, it is built as a separate JS file and has to be shipped next to the main bundle.

[RReverser]

An important difference to Webpack is that this one does not inline the code of the web worker script into the main bundle. Instead, it is built as a separate JS file and has to be shipped next to the main bundle.

Webpack does not inline it either, it's emitted as a separate file too.

[endreymarcell]

@RReverser oh OK, thanks for the correction. I updated the original comment to not mislead anyone.
Also created a little example at https://github.com/endreymarcell/esbuild-plugin-webworker for anyone who might find it useful.

[dy]

Side note: it's possible to (partially) minify worker code with the inline-worker technique:

let worker = new Worker(URL.createObjectURL(new Blob([
  (function(){

  // ...worker code goes here

  }).toString().slice(11,-1) ], { type: "text/javascript" })
));

so as a workaround, workers can be stored in usual js files and included as follows:

// worker.js
export default URL.createObjectURL(new Blob([(function(){
  // ...worker code
}).toString().slice(11,-1)], {type:'text/javascript'}))

// main.js
import workerUrl from './worker.js'
let worker = new Worker(workerUrl)

[evanw]

It works but uses a rather ugly hack to pass both the importer path and the imported file's path to be onLoad handler which makes me think there must be a better way...

The pluginData feature might make this easier.

Side note: it's possible to (partially) minify worker code with the inline-worker technique:

This is fragile and can easily break with esbuild. For example, setting the language target to es6 converts async functions into calls to a helper called __async that implements the required state machine using a generator function. However, that function is defined in the top-level scope and won't be available in the worker. So that code will crash.

As of version 5.x webpack can bundle web workers out of the box with the following syntax:

new Worker( new URL( "./worker", import.meta.url ), { type: "module" } )

This is my current plan to support this feature. This approach is sufficiently general and doesn't need to know anything about web workers. That would make this issue a duplicate of #795. It also looks like Parcel might drop support for the syntax that was originally proposed in this thread: parcel-bundler/parcel#5430 (comment). So that's another count against the original approach.

[RReverser]

That would make this issue a duplicate of #795.

Note that while there is some overlap, those are not really duplicates, because general asset handling and Worker handling need to differ.

In particular, with general asset handling Worker JS file would be treated as raw binary, and wouldn't get minified, the imports wouldn't resolve and so on - which is not what user normally wants. For this reason other bundlers have separate implementation paths for these two features.

[evanw]

I was assuming that the new URL('./some/file.js', import.meta.url) syntax should cause a new JavaScript entry point to be created independent of whether there is new Worker nearby or not. The .js file extension is normally associated with the js loader so it wouldn't be interpreted as binary.

[RReverser]

The .js file extension is normally associated with the js loader so it wouldn't be interpreted as binary.

Fair enough. That's not how it's handled in any of those bundlers AFAIK, but seems to be a sensible route too (probably also needs to include mjs, ts etc. that can result in JavaScript code as well).

[blixt]

new Worker( new URL( "./worker", import.meta.url ), { type: "module" } )

This is my current plan to support this feature.

This would also support the version without { type: "module" } right? So the bundle would use importScripts(…) instead of ES Module import syntax?

While I would love for this to all be ES Modules, neither Firefox nor Safari have any support for ES Module workers, which would make esbuild unsuitable for Worker bundling in most cases if it only emits ES Module-based Worker scripts.

Firefox issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1247687
WebKit issue: https://bugs.webkit.org/show_bug.cgi?id=164860 (but it turns out it was very recently "FIXED" so it might hit technical preview soon! 🎉)

[RReverser]

While I would love for this to all be ES Modules, neither Firefox nor Safari have any support for ES Module workers, which would make esbuild unsuitable for Worker bundling in most cases if it only emits ES Module-based Worker scripts.

Not necessarily, the point of type: "module" is to tell bundler that it contains ESM, so that it can bundle Worker code as a separate entry point. It's not meant to preserve imports in the final bundle anyway.

[blixt]

Not necessarily, the point of type: "module" is to tell bundler that it contains ESM, so that it can bundle Worker code as a separate entry point. It's not meant to preserve imports in the final bundle anyway.

I guess that makes sense, assuming it always bundles down into "old school" Workers. I'd expect the flag to be kept in the output though, as the choice can have pretty big effects on other things, for example <link rel="modulepreload"> tags in the HTML will only work for entrypoints bundled for an ES Module worker, so if the bundler gets to choose freely between the two outputs it wouldn't be possible to set up an HTML file with the right preloading logic, for example.

Basically, an author should be able to always pick "old school" workers for cross-browser support, but also optionally use ES Module workers for Chrome (which has supported this for a long time).

[RReverser]

I'd expect the flag to be kept in the output though, as the choice can have pretty big effects on other things

Usually it has to be removed precisely because it has effect on other things.

E.g. if bundler supports code splitting, then chunk loading usually has to be transformed down to importScripts, which is only available in regular Workers, and not when type:"module" is used. For this reason other bundlers remove this property.

[endreymarcell]

It works but uses a rather ugly hack to pass both the importer path and the imported file's path to be onLoad handler which makes me think there must be a better way...

The pluginData feature might make this easier.

Oh right. Thanks! I updated my implementation at https://github.com/endreymarcell/esbuild-plugin-webworker to use pluginData instead of the hack.

[NullVoxPopuli]

for me, ESM works as a work build target for the most part when combined with 'bundle' except, I need to remove all export keywords.
Is there a way to do that?

---

I submitted a PR to microsoft/vscode#123739
so if that's resolved, I don't need a way to strip exports from workers 🙃

[pierre10101]

I managed to resolve this after following the docs at link.

[lgarron]

```js
new Worker( new URL( "./worker", import.meta.url ), { type: "module" } )

This is my current plan to support this feature. This approach is sufficiently general and doesn't need to know anything about web workers. That would make this issue a duplicate of #795. It also looks like Parcel might drop support for the syntax that was originally proposed in this thread: parcel-bundler/parcel#5430 (comment). So that's another count against the original approach.

It seems Firefox and Safari are newly making progress on module workers. Given the hours I've wasted on CJS workarounds, I'd love for esbuild to be ready for this ahead of their releases so we can live in an ESM-only world. And for what it's worth, such behaviour would already be very useful right now, since it already works in node and one major browser (Chrome) — enough to use for local development and try out things ahead of other browsers being ready. 😃

But I more importantly for me, I would like to note that that it would be most useful if this was not specifically tied to the implicit/explicit global Worker constructor. For those of us writing libraries that also work in node, workarounds like like web-worker may be necessary. In that case, call sites may look something like:

new CustomWorkerConstructor( new URL( "./worker", import.meta.url ), { type: "module" } )

(It's possible to assign CustomWorkerConstructor to a variable named Worker in the current scope, so that the AST locally resembles a direct browser worker construction. But that's not always desirable.)

Handling this at the new URL() level instead of the new Worker() level would avoid the issue.

[RReverser]

Another update to this issue is that Emscripten on C++ side and wasm-bindgen-rayon on Rust side both now emit new Worker(new URL('...', import.meta.url)) pattern for Workers internally used for WebAssembly threads, so supporting this syntax would automatically make bundling Wasm easier, too.

[diachedelic]

It appears that Chrome has very recently implemented the sync import.meta.resolve(specifier) function, which is pretty much identical to new URL(specifier, import.meta.url).href. import.meta.resolve seems to be a more permanent solution. Here is a good writeup explaining it: https://gist.github.com/domenic/f2a0a9cb62d499bcc4d12aebd1c255ab.

[remcohaszing]

It would be nice if the esbuild uses the worker exports condition in package.json by default.

[fabiospampinato]

IMO it'd be cool if something like the following got implemented, which as far as I know is pretty much the cleanest web worker abstraction possible:

import {foo, bar} from './foo?worker';

await foo ( 123 );

Where every exported function is an async function. Now the type checker doesn't need to be aware that those functions will be executed in web worker, your code calling those functions doesn't need to know that either (unless in some cases if you are passing on transferrable objects), and neither the functions themselves really need to be changed, other than being marked as async.

Something like this can be implemented as a plugin, but something more tightly integrated with the bundler would work better.

[peterpeterparker]

I tried the various approach of this thread - thanks for all the great ideas - but the only solution that worked out for me once I embedded the lib that contains the worker within my apps was loaded the worker as base64 as displayed in the gist https://gist.github.com/manzt/689e4937f5ae998c56af72efc9217ef0 of @manzt

[AliMD]

@evanw
I love ESBuild and I'm waiting for you to officially support ESM capability in worker.
I really don't like having to migrate from ESBuild to another tools like Parcel.
Could you please give a timeline for this feature?
Thank you in advance.

[firien]

I do not advocate this approach, but if you are desperate, new-url-v0.18.13 , has new URL("./path/to/script.js", import.meta.url) support. This is based on work with @lgarron from #795. These new URLs are treated as dynamic imports so you need splitting option - which is still marked as WIP - You should have some good browser tests in your project if you want to use this.

I have included a devcontainer in the branch to make it simple to get going. If you don't know how to get a container going in vscode, um, read tutorial. Fire up the container and vscode should prompt you to install any additional go stuff you need. The follow assumes you are building for linux-x64 platform - change if need be.

make clean
make platform-linux-x64

new binary will be at ./npm/@esbuild/linux-x64/bin/esbuild. With ESBUILD_BINARY_PATH you can use this new binary with the esbuild from npm. The versions of npm esbuild and this new binary HAVE TO match, new-url-v0.18.13 is currently pinned to v0.18.13 so your package.json line needs to be "esbuild": "0.18.13"

If you copy ./npm/@esbuild/linux-x64/bin/esbuild out of container, and into your project at bin/esbuild - just add ESBUILD_BINARY_PATH=bin/esbuild to your current esbuild script and good luck!


---

note that I use on a project with only js. typescript is untested.

[lgarron]

I do not advocate this approach, but if you are desperate, new-url-v0.18.13 , has new URL("./path/to/script.js", import.meta.url) support. This is based on work with @lgarron from #795. These new URLs are treated as dynamic imports so you need splitting option - which is still marked as WIP - You should have some good browser tests in your project if you want to use this.

Glad to see that the code from last year can still be adapted! 🤓
(Even if I can't use it. 😭)

It looks like you're trying to support bare package names in firien@196a279
This is an issue for new URL(…, import.meta.url) (because bare names are semantically identical to relative file paths for the URL constructor). But it's a great use case for import.meta.resolve(…), which is now supported in all major browsers and node (experimental)/deno/bun: #2866

[AliMD]

any news on this?

[noseratio]

I'm late to this thread, having faced this issue with an audio worklet. I eventually resorted to the following (rather primitive) workaround:

In esbuild settings:

    entryPoints: [
      './src/www/components/audioRecorder/index.mts',
      './src/www/components/audioRecorder/audioRecorderWorklet.mts'
    ]

In the audio worklet audioRecorderWorklet.mts:

export function getWorkletUrl() {
  return import.meta.url;
}

if (globalThis.AudioWorkletProcessor) {  
  const AudioRecorderWorklet = class extends AudioWorkletProcessor {
    // ... audio worklet class
  }
  
  registerProcessor('audio-recorder-worklet', AudioRecorderWorklet);
}

In the main file index.mts:

import { getWorkletUrl } from './audioRecorderWorklet.mjs'

//...

await audioContext.audioWorklet.addModule(getWorkletUrl());

This works, but the esbuild-generated audioRecorderWorklet.mjs looks like this:

import {
  getWorkletUrl
} from "../../chunk-GRFBZUB4.mjs";
import "../../chunk-4OBMG7SZ.mjs";
export {
  getWorkletUrl
};

And the URL returned by getWorkletUrl() actually points to ../../chunk-GRFBZUB4.mjs, while audioRecorderWorklet.mjs essentially becomes a redundant stub.

I hope someone (including my future self) may find this useful, but I'm still looking for a better way of doing this.

PS. Huge thank to Evan for creating and maintaining ESBuild. It's an immensely useful tool, which keeps on delivering for cases where similar tools often fall short.

[fasiha]

I saw this issue was still open and thought esbuild didn't support the new Worker approach with webworkers but #2439 makes me think esbuild does support this?

[firien]

@fasiha, I believe #2439 is for allowing esbuild to preserve the comment, so it can make it to webpack. (https://github.com/privatenumber/esbuild-loader)

[andrevenancio]

Any updates?

[mcharytoniuk]

This issue can also be worked around with explicitly passing some kind of build id to esbuild, for example:

# can be replaced with anything, build id, random hash, commit id, date
BUILD_ID = $(date +"%s%N")

esbuild \
    --asset-names="./[name]_$(BUILD_ID)" \
    --entry-names="./[name]_$(BUILD_ID)" \
    --bundle \
    --define:__BUILD_ID=\"$(BUILD_ID)\" \
    --define:__STATIC_FILES_PATH="http://mycdn" \ # replace with your path to static files
    controller_mycontroller.ts \
    worker_myworker.ts 

It will produce a worker file in the output directory named exactly like worker_myworker_$(BUILD_ID). Then other files
will know the exact filename of the worker, they can include it with:

new Worker(`${__STATIC_FILES_PATH}/worker_myworker_${__BUILD_ID}.js`)