wasm support in fpm

[AbrarNitk]

Status: In Progress

Tasks

WASM Backend

• FPM.ftd Configuration changes
• fpm_utils crate for WASM types
• Finalize and prepare the code for the transport layer (.wit files)
• Incorporate with serve command

WASM processor

• FPM.ftd Configuration changes
• fpm_utils crate for WASM types
• .wit file creation and management
• incorporate the pluggable processor

---

We want to be able to create a special FPM package that ship with .wasm files, and these files are invoked during:

• processor / foreign variable
• HTTP backend handler

Special Package

We already have some special packages like font package. We will create two special packages, processor package and backend package.

-- fpm.package: foo.com/bar

-- fpm.processor: foo
wasm: foo.wasm

or we can follow a convention that the name of the processor and the name of the WASM file will have to match.

-- fpm.package: foo.com/bar
processor: f
processor: g

The processor WASM file has to expose a .process(section: ftd::p1::Section) -> ftd::Value method. There is a possibility we can not pass and return ftd::* stuff, and we have to pass String and another such primitive type only.

Processor namespacing

If a package is named say foo.com/bar, and it has defined a processor named f. We will have to use the full path of the processor:

-- import: foo.com/bar

-- string f:
$processor$: bar.f

Here we have used the bar because the foo.com/bar import creates bar in the global namespace, and bar has .f.

Backend Package

The file backend.wasm is a special file which is to be used for the backend creation. Any package definition which wants to expose the backend is to set the attribute backend: true in the fpm.package definition.

-- fpm.package: foo.com/bar
backend: true

;; Allow the dependency package to expose the backend.

-- fpm.dependency: baz.com
backend: true

Backend is the lowest priority catch all for all the requests.

The available backends would be

foo.com/bar/* and foo.com/bar/-/baz.com/*

When a request to foo.com/bar/f/ comes, foo.com/bar/backend.wasm would be loaded and .handle(request: fpm_utils::Request) -> fpm_utils::Response would be called. Similarly, foo.com/bar/-/baz.com/f/ comes, foo.com/bar/.packages/baz.com/backend.wasm will be loaded and the handle function would be invoked for the same.

Reusability

Both of the special packages should be reusable applications. We want to be able to reuse the packages across the ecosystem.

FPM Utils Crate

The WASM code will have access to some functions defined in the fpm-utils crate. And also a corresponding fpm-utils npm/assembly script package.

Decorators

Since we can only pass primitives over WASM boundary, we will actually create a macro that will let you write with better signature:

[fpm_utils::processor]
fn process(s: &fpm_utils::Section) -> fpm_utils::Value {
    todo!()
}

[fpm_utils::http_request]
fn serve(s: &fpm_utils::HttpRequest) -> fpm_utils::HttpResponse {
    todo!()
}

A Note On Crossing Borders

When going from Rust to WASM and back, we can not share a memory, which means we can not pass say point to fpm::Config, from Rust to WASM and hope WASM to call methods using a pointer. WASM only has to access to its own/private memory, so if you want WASM to access something Rust has, you have to copy it to WASM's memory.

To avoid such copies, we have to ensure that as little as possible data crosses the Rust/WASM boundary. So if Config is a big struct, never share Config with WASM.

Plugins In Future

In future, we may create some sort of plugin mechanism.

• plugin mechanism (on every request this function is called, before request, after response, build: before building a file, after building the file)

Performance metrics

WASM Guest function execution: 277.5µs

First WASM Execution (loading + engine etc.): 316.876666ms
Subsequent WASM execution: 156.306208ms

[sharmashobhit]

Discussion with @amitu

Suggested enhancements

• Rename guest_backend to http_backend
• We can simplify the source code for the WASM guest by implementing the Procedural macros
• Embed the .wit files in the fpm-utils package using the include_str! macro

---

Thoughts

As WASM format is a compiled binary, it's impossible for us to peek into the source code. What if a package which distributes the WASM backend also contains the source code which can be compiled?

• Cookie sharing logic. Namespaced with SHA?
• The processor is written in WASM itself, so we can cut short the implementation.
  Right now the http processor invokes the request and processes the data, ideally we can just do the same in a processor written in wasm.