Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support the wasm32-wasi target #84

Merged
merged 1 commit into from
Jun 11, 2024

Conversation

alicja-januszkiewicz
Copy link
Contributor

Hi, I'm interested in extending the functionality of dbase-rs to make it runnable on the web. My use case is that I'm working on a hobby project that targets the web and needs to be able to read in shapefiles, which is a multifile format with a dbase component. Would you be happy to accept a PR enabling this?

Below is a more involved explanation of what this PR attempts to achieve. I don't have much knowledge in this domain so I might have made some mistakes. I've included some links for reference.

Running dbase-rs on the web can be achieved with WebAssembly but there seem to be 2 different compilation targets, neither of which target the web specifically:

  • wasm32-unknown-unknown
  • wasm32-wasi / wasm32-wasip2

The former does not make any assumptions about the platform it will run on, and since dbase works with files it does not seem like the best fit. However, one could still use web_sys::File or similar to make it work on web.

The latter target assumes the target platform will provide the WebAssembly System Interface which includes a filesystem api. If my understanding is correct, this implies that dbase and std in general should work more or less out of the box. The WASI spec is still in the works and is not yet implemented by any browser, but there is a polyfill that makes wasi components executable on the web. One can also run them in a vm using a wasm runtime like wasmtime.

In this PR I've made some adjustments to make dbase-rs build on wasm32-wasip2 as it seemed like the simpler and cleaner option.

The only block preventing it from building was std::fs::File::try_clone. I've followed a workaround I found im some other repositories [1], [2] which is essentially to wrap std::fs::File in an Arc.

The filesystem api was added in WASI preview 2, which has only recently been stabilised and added as a rustc tier 3 target. As a result rustup does not yet support it, but there is a cargo-component tool that translates binaries built for wasm32-wasip1 into wasm32-wasip2 components.

To test out whether this PR works, I've installed both cargo-component and wasmtime, and have managed to get the two dbase examples to run:

cargo component build --example file
wasmtime --mapdir .::. target/wasm32-wasi/debug/examples/file.wasm

The above will

  1. cargo build the file example with the wasm32-wasip1 target
  2. transform the wasm-wasip1 binary into a wasm-wasip2 component
  3. run it on the wasmtime vm, mapping directories between the virtual and the host filesystems.

@tmontaigu
Copy link
Owner

What are you using to build your web wasm package ? I think the correct thing to do use use wasm-pack

@alicja-januszkiewicz
Copy link
Contributor Author

Sorry for the long delay in getting back,

Using wasm-pack is an option for targeting wasm32-unknown-unknown as wasm-pack does not support wasm32-wasi. This could be done but it seems more complicated as it would require the use of web APIs to simulate filesystem calls. There are multiple ways to go about that and I'm not sure which one would be best.

On the other hand, wasm32-wasi works nearly out of the box, requiring only the changes in this PR. It is potentially less useful though as hardware rendering is not yet supported on wasi.

Is either solution something you'd be happy to accept as a PR or would you rather this sort of thing be separate? I'm currently using wasm-pack for my package but I might switch to wasi.

@tmontaigu
Copy link
Owner

The changes are simply enough so we can merge them is here

@tmontaigu tmontaigu merged commit d0a06a6 into tmontaigu:master Jun 11, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants