Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support webidl namespaces. #678

Merged
merged 13 commits into from
Aug 13, 2018

Conversation

richard-uk1
Copy link
Contributor

I'm opening this early to get feedback.

This PR aims to get namespaces supported, so you can do

console::log("string")

So far it can generate code from the AST, which I reconstructed from the following code snippet which I checked compiles correctly.

extern crate wasm_bindgen;

#[no_mangle]
#[allow(non_snake_case)]
#[doc(hidden)]
#[cfg(all(target_arch = "wasm32", not(target_os = "emscripten")))]
pub extern "C" fn __wbindgen_describe___wbg_log_894f9ca062837e26() {
    use wasm_bindgen::describe::*;
    inform(FUNCTION);
    inform(1u32);
    <&str as WasmDescribe>::describe();
    inform(0);
}

mod console {
#[allow(bad_style)]
#[cfg(all(target_arch = "wasm32", not(target_os = "emscripten")))]
#[doc = ""]
pub fn log(s: &str) {
    ::wasm_bindgen::__rt::link_mem_intrinsics();
    #[link(wasm_import_module = "__wbindgen_placeholder__")]
    extern "C" {
        fn __wbg_log_894f9ca062837e26(s: <&str as ::wasm_bindgen::convert::IntoWasmAbi>::Abi)
            -> ();
    }
    unsafe {
        let _ret = {
            let mut __stack = ::wasm_bindgen::convert::GlobalStack::new();
            let s = <&str as ::wasm_bindgen::convert::IntoWasmAbi>::into_abi(s, &mut __stack);
            __wbg_log_894f9ca062837e26(s)
        };
        ()
    }
}
#[allow(bad_style, unused_variables)]
#[cfg(not(all(target_arch = "wasm32", not(target_os = "emscripten"))))]
#[doc = ""]
pub fn log(s: &str) {
    panic!(
        "cannot call wasm-bindgen imported functions on \
         non-wasm targets"
    );
}
}

#[allow(non_upper_case_globals)]
#[cfg(target_arch = "wasm32")]
#[link_section = "__wasm_bindgen_unstable"]
#[doc(hidden)]
pub static __WASM_BINDGEN_GENERATED_50eb08d47552231d :
[ u8 ; 283usize ] = *
b"\x17\x01\x00\x00{\"exports\":[],\"enums\":[],\"imports\":[{\"module\":null,\"js_namespace\":\"console\",\"kind\":{\"kind\":\"function\",\"shim\":\"__wbg_log_894f9ca062837e26\",\"catch\":false,\"method\":null,\"structural\":false,\"function\":{\"name\":\"log\"}}}],\"structs\":[],\"version\":\"0.2.15 (5b935526f)\",\"schema_version\":\"8\"}";

pub fn greet(name: &str) {
    console::log(&format!("Hello, {}!", name));
}
#[export_name = "greet"]
#[allow(non_snake_case)]
#[cfg(all(target_arch = "wasm32", not(target_os = "emscripten")))]
pub extern "C" fn __wasm_bindgen_generated_greet(
    arg0: <str as ::wasm_bindgen::convert::RefFromWasmAbi>::Abi,
) {
    ::wasm_bindgen::__rt::link_mem_intrinsics();
    let _ret = {
        let mut __stack = unsafe { ::wasm_bindgen::convert::GlobalStack::new() };
        let arg0 = unsafe {
            <str as ::wasm_bindgen::convert::RefFromWasmAbi>::ref_from_abi(arg0, &mut __stack)
        };
        let arg0 = &*arg0;
        greet(arg0)
    };
}
#[no_mangle]
#[allow(non_snake_case)]
#[doc(hidden)]
#[cfg(all(target_arch = "wasm32", not(target_os = "emscripten")))]
pub extern "C" fn __wbindgen_describe_greet() {
    use wasm_bindgen::describe::*;
    inform(FUNCTION);
    inform(1u32);
    <&str as WasmDescribe>::describe();
    inform(0);
}
#[allow(non_upper_case_globals)]
#[cfg(target_arch = "wasm32")]
#[link_section = "__wasm_bindgen_unstable"]
#[doc(hidden)]
pub static
__WASM_BINDGEN_GENERATED_bbc0315d281cc3c3 : [ u8 ; 214usize ] = *
b"\xd2\x00\x00\x00{\"exports\":[{\"class\":null,\"method\":false,\"consumed\":false,\"constructor\":null,\"function\":{\"name\":\"greet\"},\"comments\":[]}],\"enums\":[],\"imports\":[],\"structs\":[],\"version\":\"0.2.15 (5b935526f)\",\"schema_version\":\"8\"}"
;

Copy link
Member

@fitzgen fitzgen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @derekdreery! I think this is on the right track.

However, I'm unsure we want to land this without tests, which will be hard to write without integration in one of our two frontends (webidl and proc-macro). Maybe @alexcrichton has a stronger opinion?

General thoughts: If we add support for modules to the proc-macro frontend, and two different #[wasm_bindgen] extern { ... } blocks reference the same module, it will be emitted as two duplicate modules in the final code, which will cause errors. I don't see a way around this (other than putting the extern blocks in different modules), and so we would have to document the limitation very clearly.

With the webidl frontend, we can do some pre/post processing to ensure that there aren't duplicate modules, just the single canonical one.

@richard-uk1
Copy link
Contributor Author

I'm planning to write tests once I've done the webidl-to-backend bit (which I've just started), I wouldn't expect you to accept this without tests. However, I think it's hard to just test the codegen without manually creating AST in the test - once the webidl bit is working we can test both at the same time.

@fitzgen
Copy link
Member

fitzgen commented Aug 9, 2018

Yep, sounds good to me!

@fitzgen
Copy link
Member

fitzgen commented Aug 9, 2018

Let me/alex/ohanar know when you're ready for another round of review :)

@richard-uk1
Copy link
Contributor Author

richard-uk1 commented Aug 10, 2018

There is another option here:

The webidl suggests that the console methods are pseudo-static methods on the Console type. So instead of generating a module, we could generate a Console struct and add the methods there. Note sure which is better.

For example: the console webidl is

[Exposed=(Window,Worker,WorkerDebugger,Worklet,System),
 ClassString="Console",
 ProtoObjectHack]
namespace console {
    //...
}

So the operations are sort-of methods on a global singleton Console object at console.

So there are 3 options for generating bindings:

extern {
    pub static console: Console;
}

// later..
console.log("something");

or using static (js meaning) methods:

pub struct Console;

impl Console {
    pub fn log(val: JsValue);
}

// later..
Console::log("something")

or using a module and bare methods

pub mod console {
    pub fn log(val: JsValue);
}

// later..
console::log("something")

@fitzgen
Copy link
Member

fitzgen commented Aug 10, 2018

I had written out a comment about how I prefer modules, but then I discovered this in the webidl spec:

A namespace is a specification of a set of namespace members (matching NamespaceMembers), which are the regular operations and read only regular attributes that appear between the braces in the namespace declaration. These operations and attributes describe the behaviors packaged into the namespace.

[...]

Namespace ::
    namespace identifier { NamespaceMembers } ;

NamespaceMembers ::
    ExtendedAttributeList NamespaceMember NamespaceMembers
    ε

NamespaceMember ::
    RegularOperation
    readonly AttributeRest

and verified that our webidls only have operations and attributes within namespaces. That means we don't need to worry about nested types/interfaces/mixins. Which means that the first and second options are looking a lot more tempting, and require much less invasive changes to wasm-bindgen.

Additionally, if you carefully read these steps on creating an operation you'll find that namespace operations do not get the namespace object passed as a this value. This doesn't matter in the end because static_method_of = ... will bind the static method to its class anyways, and if we used a static object like static console; console.log(..); we would be invoking the method with a this receiver as well.

All of that said, I am partial to using static methods, the equivalent of

#[wasm_bindgen]
extern {
    type console;

    #[wasm_bindgen(static_method_of = console)]
    fn log(...);
}

console::log(...);

because it matches what js-sys does with Math.

@afdw
Copy link
Contributor

afdw commented Aug 10, 2018

There is a problem with naming here though: while Math is not a bad name for a Rust type, console is, as it starts with a lowercase letters. It is a bit unconventional.

@ohanar
Copy link
Member

ohanar commented Aug 10, 2018

The main problem with using static methods rather than modules (at least as I see it), is that you can't import a static method into your local namespace. I.e.

pub mod console_module {
    pub fn log(...) { ... }
}
// vs
pub struct ConsoleType;
impl ConsoleType {
    pub fn log(...) { ... }
}

use console_module::log; // this works
use ConsoleType::log; // this doesn't

@richard-uk1
Copy link
Contributor Author

richard-uk1 commented Aug 10, 2018

I've almost finished implementing this as per the module version, (finished modulo bugs/errors), and anyone who wants to change over to one of the other variants will find it much easier after this PR has merged (since you can just go in and edit the AST transformation from webidl to backend), so I would consider punting on the decision and seeing what people's experience of the current version is.

The reason I've stopped is I'm getting the following error

error[E0308]: mismatched types
   --> crates/webidl/src/lib.rs:895:32
    |
895 |             js_namespace: Some(ns_names.js_name.clone()),
    |                                ^^^^^^^^^^^^^^^^^^^^^^^^ expected struct `proc_macro2::Ident`, found &str
    |
    = note: expected type `proc_macro2::Ident`
               found type `&str`

error: aborting due to previous error

which is easy to solve, but I was wondering if an Ident is the right type for the js_namespace, since it is only used on the javascript side. Unless the valid ident rules are the same in javascript and rust (I don't know if they are or not), we might as well just use a &str, because Ident suggests that it has meaning in rust, which it doesn't.

As always, I love to find out I'm wrong, so tell me if I am :).

Also please don't merge this until I have added tests, and implemented partial namespaces (easy because I already did all the first_pass stuff).

I need to find something other than console for testing, as it uses variadic arguments which will continue to be unsupported (at least in this PR).

@richard-uk1
Copy link
Contributor Author

There is a problem with naming here though: while Math is not a bad name for a Rust type, console is, as it starts with a lowercase letters. It is a bit unconventional.

This problem also comes up with external C libraries - so maybe people would be comfortable with it. I don't know.

 - Tried `cargo doc` and seen methods generated.
 - Added test with a few method calls to the console operations.
@richard-uk1
Copy link
Contributor Author

richard-uk1 commented Aug 11, 2018

Ok @fitzgen @alexcrichton ready for review :).

EDIT The library definitely isn't working right now - when I try to run the functions I get an error

CompileError: wasm validation error: at offset 316: byte size mismatch in function section

I'm using an altered version of the hello_world example to test @ https://github.com/derekdreery/test_console_wasm .

I'm not sure how to make progress.

EDIT I think it's related to the fact that we are importing the module from another file - wasm-bindgen doesn't like this, but it's happy if we declare the module in the same file (see the bindings I manually constructed).

@richard-uk1 richard-uk1 changed the title [WIP] Add support for modules to the backend. Add support webidl namespaces. Aug 11, 2018
@ohanar
Copy link
Member

ohanar commented Aug 11, 2018

I have no issue running your altered hello_world example. I remember when I first started looking at this project, I was getting a bunch of validation errors, that eventually went away when I restarted my system. I never really figured out what was causing the issue.

Copy link
Member

@ohanar ohanar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally this looks really good to me. I would really like to see some more testing here though. You could add something to the webidl-tests crate if nothing is quite ready in web-sys.

.operations
.entry(identifier)
.and_modify(|operation_data| operation_data.overloaded = true)
.or_insert_with(Default::default)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could just use or_default here.

pub(crate) operations: BTreeMap<Option<&'src str>, OperationData<'src>>,
}

impl<'src> NamespaceData<'src> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just make this the implementation for Default?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason it's the way it currently is is that I felt implementing Default is misleading - there isn't an 'obvious' default value for the partial field. I thought that having separate methods helped convey meaning.

In fact, if I had followed my own plan I would not implement Default at all, I'd have 2 methods for partial and non-partial. I'll change to this so you can see what I mean, if you don't like it I can change it to just the Default :).

return Ok(());
}

let rust_name = rust_ident(self.identifier.0.to_snake_case().as_str());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is repeated a lot. Maybe make a snake_case_ident function in utils? (It would match the camel_case_ident function`.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the camel_case_ident function exists to handle cases like HTMLBR where the generic converter cannot deduce there should be a space, like HTML_BR.

@richard-uk1
Copy link
Contributor Author

richard-uk1 commented Aug 12, 2018

@ohanar I've tried updating everything (rust, npm, cargo) and building from scratch, and I still get the offest error. Very strange, and unfortunate since I can't debug what I've done.

Anyway I can make the changes you suggested. What tests would you like - it's hard to test the output of the log functions as they don't return anything?

@richard-uk1
Copy link
Contributor Author

I'm not sure what's causing the build error now - it's failing in chrome but not firefox.

@ohanar
Copy link
Member

ohanar commented Aug 12, 2018

So it seems like the issue you are running up against is some upstream bug (see webpack/webpack#7760 and xtuc/webassemblyjs#407).

As for tests, you could add something like this to webidl-test:

namespace Math {
    double pow(double base, double exponent);
}

And then assert some things with it in a corresponding rust file.

@richard-uk1
Copy link
Contributor Author

yep using webpack from git://github.com/webpack/webpack.git#b310b9b45c7e8a60c40d13dae967f74e712ec224 fixed it and I get the output I expect!

I'll work on some tests.

@richard-uk1
Copy link
Contributor Author

I've added some tests - I'm not sure they ran though because when I typed cargo test -p wasm-bindgen-webidl, it said "running 0 tests".

@afdw
Copy link
Contributor

afdw commented Aug 12, 2018

Try cargo test -p webidl-tests --target wasm32-unknown-unknown.

@richard-uk1
Copy link
Contributor Author

@afdw that worked (well it errored, but that's better than not running).

@richard-uk1
Copy link
Contributor Author

Should be ready for review again.

@ohanar ohanar merged commit 36fe4c2 into rustwasm:master Aug 13, 2018
@ohanar
Copy link
Member

ohanar commented Aug 13, 2018

Looks good!

@richard-uk1 richard-uk1 deleted the webidl_namespace_support branch August 13, 2018 08:49
@fitzgen
Copy link
Member

fitzgen commented Aug 13, 2018

Thanks for sticking with this PR @derekdreery, and thanks for the detailed reviews @ohanar!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants