Skip to content

Commit

Permalink
feat: automatically provide imports object using import map (#11)
Browse files Browse the repository at this point in the history
  • Loading branch information
pi0 authored Jan 4, 2024
1 parent b06f6f3 commit 4e2ae1a
Show file tree
Hide file tree
Showing 13 changed files with 113 additions and 31 deletions.
30 changes: 26 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,15 @@ The development will be split into multiple stages.
- [ ] ESM Loader ([unjs/unwasm#5](https://github.com/unjs/unwasm/issues/5))
- [ ] Integration with [Wasmer](https://github.com/wasmerio) ([unjs/unwasm#6](https://github.com/unjs/unwasm/issues/6))
- [ ] Convention for library authors exporting wasm modules ([unjs/unwasm#7](https://github.com/unjs/unwasm/issues/7))
- [x] Auto resolve imports from the imports map

## Bindings API

When importing a `.wasm` module, unwasm resolves, reads, and then parses the module during build process to get the information about imports and exports and generate appropriate code bindings.
When importing a `.wasm` module, unwasm resolves, reads, and then parses the module during the build process to get the information about imports and exports and even tries to [automatically resolve imports](#auto-imports) and generate appropriate code bindings for the bundler.

If the target environment supports [top level `await`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/await#top_level_await) and also the wasm module requires no imports object (auto-detected after parsing), unwasm generates bindings to allow importing wasm module like any other ESM import.
If the target environment supports [top level `await`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/await#top_level_await) and also the wasm module requires no imports object (or they are auto resolvable), unwasm generates bindings to allow importing wasm module like any other ESM import.

If the target environment lacks support for top level `await` or the wasm module requires imports object or `lazy` plugin option is set to `true`, unwasm will export a wrapped [Proxy](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy) object which can be called as a function to lazily evaluate the module with custom imports object. This way we still have a simple syntax as close as possible to ESM modules and also we can lazily initialize modules.
If the target environment lacks support for top-level `await` or the wasm module requires an imports object or `lazy` plugin option is set to `true`, unwasm will export a wrapped [Proxy](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy) object which can be called as a function to evaluate the module with custom imports object lazily. This way we still have a simple syntax as close as possible to ESM modules and also we can lazily initialize modules.

**Example:** Using static import

Expand All @@ -46,7 +47,7 @@ import { sum } from "unwasm/examples/sum.wasm";
const { sum } = await import("unwasm/examples/sum.wasm");
```

If your WebAssembly module requires an import object (which is likely!), the usage syntax would be slightly different as we need to initiate the module with an import object first.
If your WebAssembly module requires an import object (unwasm can [automatically infer them](#auto-imports)), the usage syntax would be slightly different as we need to initiate the module with an import object first.

**Example:** Using dynamic import with imports object

Expand Down Expand Up @@ -169,6 +170,27 @@ Example parsed result:
}
```
## Auto Imports
unwasm can automatically infer the imports object and bundle them using imports maps (read more: [MDN](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script/type/importmap), [Node.js](https://nodejs.org/api/packages.html#imports) and [WICG](https://github.com/WICG/import-maps)).
To hint to the bundler how to resolve imports needed by the `.wasm` file, you need to define them in a parent `package.json` file.
**Example:**
```js
{
"exports": {
"./rand.wasm": "./rand.wasm"
},
"imports": {
"env": "./env.mjs"
}
}
```
**Note:** The imports can also be prefixed with `#` like `#env` if you like to respect Node.js conventions.
## Development
- Clone this repository
Expand Down
1 change: 1 addition & 0 deletions examples/env.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export const seed = () => Math.random() * Date.now()
5 changes: 5 additions & 0 deletions examples/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"imports": {
"#env": "./env.mjs"
}
}
7 changes: 5 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,9 @@
"files": [
"dist",
"*.d.ts",
"examples/*.wasm"
"examples/*.wasm",
"examples/package.json",
"examples/*.mjs"
],
"scripts": {
"build": "unbuild && pnpm build:examples",
Expand All @@ -50,6 +52,7 @@
"magic-string": "^0.30.5",
"mlly": "^1.4.2",
"pathe": "^1.1.1",
"pkg-types": "^1.0.3",
"unplugin": "^1.6.0"
},
"devDependencies": {
Expand All @@ -70,4 +73,4 @@
"vitest": "^1.1.1"
},
"packageManager": "pnpm@8.13.1"
}
}
3 changes: 3 additions & 0 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions src/plugin/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ const unplugin = createUnplugin<UnwasmPluginOptions>((opts) => {
const buff = await fs.readFile(id);
return buff.toString("binary");
},
transform(code, id) {
async transform(code, id) {
if (!id.endsWith(".wasm")) {
return;
}
Expand All @@ -111,7 +111,7 @@ const unplugin = createUnplugin<UnwasmPluginOptions>((opts) => {
});

return {
code: getWasmBinding(asset, opts),
code: await getWasmBinding(asset, opts),
map: { mappings: "" },
};
},
Expand Down
65 changes: 60 additions & 5 deletions src/plugin/runtime.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { readPackageJSON } from "pkg-types";
import {
UMWASM_HELPERS_ID,
UNWASM_EXTERNAL_PREFIX,
Expand All @@ -8,27 +9,35 @@ import {
// https://marketplace.visualstudio.com/items?itemName=Tobermory.es6-string-html
const js = String.raw;

export function getWasmBinding(asset: WasmAsset, opts: UnwasmPluginOptions) {
export async function getWasmBinding(
asset: WasmAsset,
opts: UnwasmPluginOptions,
) {
// -- Auto load imports --
const autoImports = await getWasmImports(asset, opts);

// --- Environment dependent code to initialize the wasm module using inlined base 64 or dynamic import ---
const envCode: string = opts.esmImport
? js`
async function _instantiate(imports) {
${autoImports.code};
async function _instantiate(imports = _imports) {
const _mod = await import("${UNWASM_EXTERNAL_PREFIX}${asset.name}").then(r => r.default || r);
return WebAssembly.instantiate(_mod, imports)
}
`
: js`
import { base64ToUint8Array } from "${UMWASM_HELPERS_ID}";
${autoImports.code};
function _instantiate(imports) {
function _instantiate(imports = _imports) {
const _data = base64ToUint8Array("${asset.source.toString("base64")}")
return WebAssembly.instantiate(_data, imports)
}
`;

// --- Binding code to export the wasm module exports ---
const canTopAwait =
opts.lazy !== true && Object.keys(asset.imports).length === 0;
const canTopAwait = opts.lazy !== true && autoImports.resolved;

// eslint-disable-next-line unicorn/prefer-ternary
if (canTopAwait) {
Expand Down Expand Up @@ -136,3 +145,49 @@ export function createLazyWasmModule(_instantiator) {
}
`;
}

export async function getWasmImports(
asset: WasmAsset,
opts: UnwasmPluginOptions,

Check warning on line 151 in src/plugin/runtime.ts

View workflow job for this annotation

GitHub Actions / autofix

'opts' is defined but never used

Check warning on line 151 in src/plugin/runtime.ts

View workflow job for this annotation

GitHub Actions / ci (ubuntu-latest)

'opts' is defined but never used
) {
const importNames = Object.keys(asset.imports || {});
if (importNames.length === 0) {
return {
code: "const _imports = { /* no imports */ }",
resolved: true,
};
}

// Try to resolve from nearest package.json
const pkgJSON = await readPackageJSON(asset.id);

let code = "const _imports = {";

let resolved = true;

for (const moduleName of importNames) {
const importNames = asset.imports[moduleName];
const pkgImport =
pkgJSON.imports?.[moduleName] || pkgJSON.imports?.[`#${moduleName}`];

if (pkgImport) {
code = `import * as _imports_${moduleName} from "${pkgImport}";\n${code}`;
} else {
resolved = false;
}
code += `\n ${moduleName}: {`;
for (const name of importNames) {
code += pkgImport
? `\n ${name}: _imports_${moduleName}.${name},\n`
: `\n ${name}: () => { throw new Error("\`${moduleName}.${name}\` is not provided!")},\n`;
}
code += " },\n";
}

code += "};\n";

return {
code,
resolved,
};
}
6 changes: 0 additions & 6 deletions test/fixture/_shared.mjs

This file was deleted.

12 changes: 5 additions & 7 deletions test/fixture/dynamic-import.mjs
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
const { sum } = await import("@fixture/wasm/sum.wasm");
export async function test() {
// Avoid top-level await because of miniflare curren't limitation.
// https://github.com/cloudflare/miniflare/issues/753
const { rand } = await import("@fixture/wasm/rand.wasm");
const { sum } = await import("@fixture/wasm/sum.wasm");

const { imports } = await import("./_shared.mjs");
const { rand } = await import("@fixture/wasm/rand.wasm").then((r) =>
r.default(imports),
);

export function test() {
if (sum(1, 2) !== 3) {
return "FALED: sum";
}
Expand Down
5 changes: 1 addition & 4 deletions test/fixture/static-import.mjs
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
import { imports } from "./_shared.mjs";
import { sum } from "@fixture/wasm/sum.wasm";
import initRand, { rand } from "@fixture/wasm/rand.wasm";

await initRand(imports);
import { rand } from "@fixture/wasm/rand.wasm";

export function test() {
if (sum(1, 2) !== 3) {
Expand Down
1 change: 1 addition & 0 deletions test/node_modules/@fixture/wasm/env.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions test/node_modules/@fixture/wasm/package.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion test/plugin.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ async function _evalCloudflare(name: string) {
import { test } from "./index.mjs";
export default {
async fetch(request, env, ctx) {
return new Response(test());
return new Response(await test());
}
}
`,
Expand Down

0 comments on commit 4e2ae1a

Please sign in to comment.