feat: WASM UDF MVP #10910

xxchan · 2023-07-12T09:17:38Z

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

To test it, just run:

./src/udf/wit_example/create.sh

TODO

user friendly SDK (proc macro)
- More crazily, what about let user upload Rust, and let us compile it ...?
consider a upgrade procedure if breaking changes happens
schema check
memory/CPU limitation

Checklist

I have written necessary rustdoc comments
I have added necessary unit tests and integration tests
I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
All checks passed in ./risedev check (or alias, ./risedev c)
My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

My PR contains user-facing changes.

Types of user-facing changes

SQL commands, functions, and operators

Release note

Support WebAssembly UDF.

Note

Powered by WebAssembly Component Model, which is still work in progress, although already usable. Therefore, RisingWave Wasm UDF is also highly experimental, and thus we cannot provide stability guarantee.

Usage

Users can create a WASM component with the WIT file src/udf/wit/udf.wit and Apache Arrow. They can use different programming languages (rust and golang are provided examples).

To create a function with the compiled WASM component,

-- $encoded is base64-encoded WASM component
CREATE FUNCTION foo(...) RETURNS ... LANGUAGE wasm_v1 USING BASE64 '$encoded';

See src/udf/wit_example for example code & required tools.

Configuration

One additional system parameter wasm_storage_url is added (defaults to fs://@/tmp/risingwave, which is used to store the user-uploaded WASM file, and intermediate compilation artifacts.

picked from #10910

Cargo.lock

xxchan · 2023-08-02T22:16:39Z

src/udf/src/wasm.rs

+        identifier: &str,
+    ) -> WasmUdfResult<InstantiatedComponent> {
+        let object_store = get_wasm_storage(wasm_storage_url).await?;
+        let serialized_component = object_store.read(&compiled_path(identifier), None).await?;


FIXME: This stuck forever when I use S3. WHY??? 🥵

update: this was because of using futures rt inside tokio block_on (or sth). Already fixed by tokio::task::block_in_place(|| { tokio::runtime::Handle::current().block_on

commit b52a004 Author: xxchan <xxchan22f@gmail.com> Date: Thu Oct 26 14:25:18 2023 +0800 update arrow-ipc commit e94feeb Author: xxchan <xxchan@users.noreply.github.com> Date: Thu Oct 26 06:21:34 2023 +0000 Fix "cargo-hakari" commit 08a5601 Merge: 56e6fc4 942e99d Author: xxchan <xxchan22f@gmail.com> Date: Thu Oct 26 14:19:34 2023 +0800 Merge branch 'main' into xxchan/wasm-udf commit 942e99d Author: Yufan Song <33971064+yufansong@users.noreply.github.com> Date: Wed Oct 25 22:10:31 2023 -0700 fix(nats-connector): change stream into optional string, add replace stream name logic (#13024) commit 90fb4a3 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu Oct 26 04:25:11 2023 +0000 chore(deps): Bump comfy-table from 7.0.1 to 7.1.0 (#13049) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit b724be7 Author: jinser <aimer@purejs.icu> Date: Thu Oct 26 00:26:15 2023 +0800 feat: add `comment on` clause support (#12849) Co-authored-by: Richard Chien <stdrc@outlook.com> Co-authored-by: August <pin@singularity-data.com> commit 7f791d6 Author: August <pin@singularity-data.com> Date: Wed Oct 25 20:29:16 2023 +0800 feat: move model_v2 and model_migration into a separate crates (#13058) commit 7f82929 Author: Noel Kwan <47273164+kwannoel@users.noreply.github.com> Date: Wed Oct 25 16:57:45 2023 +0800 fix(meta): persist internal tables of `CREATE TABLE` (#13039) commit 09a67ab Author: Noel Kwan <47273164+kwannoel@users.noreply.github.com> Date: Wed Oct 25 16:49:08 2023 +0800 fix: `WAIT` should return error if timeout (#13045) commit e48547d Author: Runji Wang <wangrunji0408@163.com> Date: Wed Oct 25 16:41:16 2023 +0800 refactor(type): switch jsonb to flat representation (#12952) Signed-off-by: Runji Wang <wangrunji0408@163.com> commit 56e6fc4 Author: xxchan <xxchan22f@gmail.com> Date: Wed Oct 25 15:33:36 2023 +0800 fix merge issue commit c644361 Merge: fcd6992 2d428b1 Author: xxchan <xxchan22f@gmail.com> Date: Wed Oct 25 15:23:44 2023 +0800 Merge remote-tracking branch 'origin/main' into xxchan/wasm-udf commit fcd6992 Author: xxchan <xxchan22f@gmail.com> Date: Wed Oct 25 14:28:53 2023 +0800 fix s3 stuck commit 21e9740 Author: xxchan <xxchan22f@gmail.com> Date: Wed Oct 25 12:47:24 2023 +0800 Revert "fix s3 stuck (why?)" This reverts commit f19a6b4. commit f19a6b4 Author: xxchan <xxchan22f@gmail.com> Date: Wed Sep 13 14:32:28 2023 +0800 fix s3 stuck (why?) commit 019f309 Author: xxchan <xxchan22f@gmail.com> Date: Tue Sep 12 15:29:52 2023 +0800 ON_ERROR_STOP=1 commit 6e4ee3c Author: xxchan <xxchan22f@gmail.com> Date: Tue Sep 12 15:09:58 2023 +0800 generate-config commit b63a1c3 Merge: 2b0cc96 53611bf Author: xxchan <xxchan22f@gmail.com> Date: Tue Sep 12 14:53:10 2023 +0800 Merge remote-tracking branch 'origin/main' into xxchan/wasm-udf commit 2b0cc96 Author: xxchan <xxchan22f@gmail.com> Date: Sat Sep 9 23:49:43 2023 +0800 fix conflicts commit 6b13fe3 Author: xxchan <xxchan22f@gmail.com> Date: Sat Sep 9 23:35:50 2023 +0800 update system param default commit a273943 Merge: cc34bfe f649aa6 Author: xxchan <xxchan22f@gmail.com> Date: Sat Sep 9 23:33:38 2023 +0800 Merge remote-tracking branch 'origin/main' into xxchan/wasm-udf commit cc34bfe Author: xxchan <xxchan22f@gmail.com> Date: Tue Aug 1 17:47:42 2023 +0200 use count_char as the example commit f913f63 Merge: 53bf8e0 2637dbd Author: xxchan <xxchan22f@gmail.com> Date: Tue Aug 1 17:22:13 2023 +0200 Merge branch 'main' into xxchan/wasm-udf commit 53bf8e0 Author: xxchan <xxchan22f@gmail.com> Date: Mon Jul 31 14:20:07 2023 +0200 minor update commit 70cee42 Author: xxchan <xxchan22f@gmail.com> Date: Mon Jul 17 14:53:29 2023 +0200 fix arrow_schema into -> try_into commit a7d172d Author: xxchan <xxchan22f@gmail.com> Date: Fri Jul 14 16:31:20 2023 +0200 buf format commit 43a3290 Author: xxchan <xxchan22f@gmail.com> Date: Thu Jul 13 23:04:16 2023 +0200 add tinygo example & turn on wasi support commit 61a4998 Author: xxchan <xxchan22f@gmail.com> Date: Wed Jul 12 11:40:56 2023 +0200 cleanup commit 165d4d9 Author: xxchan <xxchan22f@gmail.com> Date: Wed Jul 12 11:02:44 2023 +0200 use object store to store wasm commit 88979e4 Author: xxchan <xxchan22f@gmail.com> Date: Tue Jul 11 15:32:52 2023 +0200 add wasm_storage_url system param commit a897320 Author: xxchan <xxchan22f@gmail.com> Date: Thu Jul 6 20:04:45 2023 +0200 Load compiled wasm module in expr 🚀🚀🚀 commit 63b3523 Author: xxchan <xxchan22f@gmail.com> Date: Sun Jul 2 19:27:22 2023 +0200 it works (although very slow)

Signed-off-by: Runji Wang <wangrunji0408@163.com>

wangrunji0408

I have tested and it works! Great job! Next I'm going to review the code and begin to learn wasm. 🤡

wangrunji0408 · 2023-12-08T07:16:34Z

src/udf/Cargo.toml

 cfg-or-panic = "0.2"
 futures-util = "0.3.28"
+itertools = "0.11"
+risingwave_object_store = { workspace = true }


I think it would be better to decouple UDF from object store and put related code into the compute node.

src/udf/Cargo.toml

wangrunji0408 · 2023-12-08T07:30:11Z

src/storage/hummock_trace/Cargo.toml

@@ -25,7 +25,7 @@ tokio = { version = "0.2", package = "madsim-tokio" }
 tracing = "0.1"

 [dev-dependencies]
-itertools = "0.10.5"
+itertools = "0.11"


Suggested change

itertools = "0.11"

itertools = "0.12"

src/udf/wit/udf.wit

wangrunji0408 · 2023-12-08T07:56:17Z

src/udf/wit_example/create.sh

+# debug:   23557258
+# release: 12457072


The binary size looks much larger than I imagine. Is there a way to strip the binary? or is it possible to build on target wasm32-unknown-unknown?

is it possible to build on target wasm32-unknown-unknown?

This is not possible and there's a long story. But I forget the details now. 🤡

Is there a way to strip the binary?

I think so. I remember there are some tools for this purpose. And IIRC currently debuginfo is also included.

Please tell me when you recall the story. 🤡

Maybe my memory was wrong.

I remember:

We can definitely compile a wit module to wasm32-unknown-unknown and run it.

There are many things don't work in wasm32-unknown-unknown.

From the runtime side (host), API for wasi and non-wasi are different.

But I don't remember:

(Mainly) Whether the WIT UDF example (especially arrow-rs) works in wasm32-unknown-unknown

Whether a non-wasi module can be run with the wasi host API.

Maybe I was using wasi just because I can println 🤡

xxchan · 2023-12-08T07:56:50Z

Next I'm going to review the code and begin to learn wasm. 🤡

I will also need to relearn it 🤡

wangrunji0408 · 2023-12-08T08:14:05Z

src/expr/core/src/error.rs

+    WasmUdf(
+        #[from]
+        #[backtrace]
+        risingwave_udf::wasm::WasmUdfError,


Can this error be merged into risingwave_udf::Error?

wangrunji0408 · 2023-12-08T08:20:58Z

src/frontend/src/catalog/function_catalog.rs

+    // for backward compatibility, newly added fields should be optional
+    pub extra: Option<PbExtra>,


I think this struct can be reorganized without worrying about backward compatibility like protobuf messages.

src/udf/wit_example/rust/Cargo.toml

wangrunji0408 · 2023-12-08T08:31:03Z

src/udf/wit_example/rust/Cargo.toml

+[profile.release]
+debug = 1


Without this, the binary size in release mode could be reduced to 3MB.

Suggested change

[profile.release]

debug = 1

wangrunji0408 · 2023-12-08T08:33:50Z

src/udf/wit_example/rust/src/lib.rs

+}
+
+impl Udf for CountChar {
+    fn eval(batch: RecordBatch) -> Result<RecordBatch, EvalErrno> {


I'm planning to build another #[function] macro to generate these code. It would be very similar to our internal #[function] macro. The only difference is fitting Arrow arrays.

wangrunji0408 · 2023-12-08T09:32:15Z

src/udf/wit/udf.wit

+  // TODO: is schema needed? since record-batch already contains schema.
+  export input-schema: func() -> schema
+  export output-schema: func() -> schema


I think it's needed as a signature of the function.

wangrunji0408 · 2023-12-08T09:34:10Z

src/udf/wit/udf.wit

+  export input-schema: func() -> schema
+  export output-schema: func() -> schema
+
+  // export init: func(inputs: list<scalar>) -> result<_, init-errno>


What's this method for?

src/udf/wit/udf.wit

Signed-off-by: Runji Wang <wangrunji0408@163.com>

wangrunji0408 · 2023-12-14T05:25:54Z

src/frontend/src/handler/create_function.rs

@@ -48,7 +51,7 @@ pub async fn handle_create_function(
        Some(lang) => {
            let lang = lang.real_value().to_lowercase();
            match &*lang {
-                "python" | "java" => lang,
+                "python" | "java" | "wasm_v1" => lang,


The version can be encoded as metadata in wasm binaries so that the language can be always "wasm".

github-actions bot added the type/feature label Jul 12, 2023

xxchan added a commit that referenced this pull request Jul 12, 2023

chore: add some comments / move some code

3fc615d

picked from #10910

xxchan mentioned this pull request Jul 12, 2023

chore: add some comments / move some code #10912

Merged

8 tasks

xxchan force-pushed the xxchan/wasm-udf branch from 746c432 to b702d0a Compare July 12, 2023 20:53

xxchan mentioned this pull request Jul 12, 2023

refactor(pgwire): truncate sql in tracing #10923

Merged

8 tasks

xxchan force-pushed the xxchan/wasm-udf branch 5 times, most recently from 237d240 to 3039d04 Compare July 13, 2023 21:17

xxchan added the experimental label Jul 13, 2023

github-actions bot added the user-facing-changes Contains changes that are visible to users label Jul 13, 2023

xxchan force-pushed the xxchan/wasm-udf branch from 3039d04 to a7d172d Compare July 17, 2023 12:48

xxchan commented Jul 17, 2023

View reviewed changes

Cargo.lock Outdated Show resolved Hide resolved

wangrunji0408 self-requested a review August 2, 2023 10:40

xxchan commented Aug 2, 2023

View reviewed changes

xxchan mentioned this pull request Oct 9, 2023

Tracking: User-Defined Function(UDF) Support #7405

Closed

26 tasks

xxchan force-pushed the xxchan/wasm-udf branch from b52a004 to 26eaf7a Compare October 26, 2023 06:39

xxchan force-pushed the xxchan/wasm-udf branch from 26eaf7a to e111248 Compare October 31, 2023 02:11

wangrunji0408 force-pushed the xxchan/wasm-udf branch 2 times, most recently from 69a280b to 5b53a7c Compare December 8, 2023 06:52

Merge remote-tracking branch 'origin/main' into xxchan/wasm-udf

97f64e3

Signed-off-by: Runji Wang <wangrunji0408@163.com>

wangrunji0408 force-pushed the xxchan/wasm-udf branch from 5b53a7c to 97f64e3 Compare December 8, 2023 07:10

wangrunji0408 reviewed Dec 8, 2023

View reviewed changes

wangrunji0408 and others added 2 commits December 8, 2023 18:17

update wasmtime to v15

f65ed64

Signed-off-by: Runji Wang <wangrunji0408@163.com>

Fix "cargo-hakari"

f617cac

Merge remote-tracking branch 'origin/main' into xxchan/wasm-udf

d5db2b6

Signed-off-by: Runji Wang <wangrunji0408@163.com>

wangrunji0408 force-pushed the xxchan/wasm-udf branch from f697a5e to d5db2b6 Compare December 14, 2023 05:19

wangrunji0408 reviewed Dec 14, 2023

View reviewed changes

wangrunji0408 mentioned this pull request Dec 29, 2023

feat(udf): add initial support for WASM-based Rust UDF #14271

Merged

9 tasks

xxchan closed this Dec 29, 2023

xxchan deleted the xxchan/wasm-udf branch April 18, 2024 09:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: WASM UDF MVP #10910

feat: WASM UDF MVP #10910

xxchan commented Jul 12, 2023 •

edited

Loading

xxchan Aug 2, 2023

xxchan Dec 8, 2023 •

edited

Loading

wangrunji0408 left a comment •

edited

Loading

wangrunji0408 Dec 8, 2023

wangrunji0408 Dec 8, 2023

wangrunji0408 Dec 8, 2023 •

edited

Loading

xxchan Dec 8, 2023

wangrunji0408 Dec 8, 2023

xxchan Dec 8, 2023

xxchan commented Dec 8, 2023

wangrunji0408 Dec 8, 2023

wangrunji0408 Dec 8, 2023

wangrunji0408 Dec 8, 2023

wangrunji0408 Dec 8, 2023

wangrunji0408 Dec 8, 2023

wangrunji0408 Dec 8, 2023

wangrunji0408 Dec 14, 2023

		// for backward compatibility, newly added fields should be optional
		pub extra: Option<PbExtra>,

feat: WASM UDF MVP #10910

feat: WASM UDF MVP #10910

Conversation

xxchan commented Jul 12, 2023 • edited Loading

What's changed and what's your intention?

Checklist

Documentation

Types of user-facing changes

Release note

Usage

Configuration

Choose a reason for hiding this comment

xxchan Dec 8, 2023 • edited Loading

Choose a reason for hiding this comment

wangrunji0408 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wangrunji0408 Dec 8, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xxchan commented Dec 8, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xxchan commented Jul 12, 2023 •

edited

Loading

xxchan Dec 8, 2023 •

edited

Loading

wangrunji0408 left a comment •

edited

Loading

wangrunji0408 Dec 8, 2023 •

edited

Loading