From 7597defbf3ea5c94487107fc6673d63ecc524641 Mon Sep 17 00:00:00 2001 From: Mark Hammond Date: Fri, 31 Dec 2021 15:04:45 +1100 Subject: [PATCH 1/7] Add a doc comparing UniFFI with diplomat --- docs/diplomat-and-macros.md | 186 ++++++++++++++++++++++++++++++++++++ 1 file changed, 186 insertions(+) create mode 100644 docs/diplomat-and-macros.md diff --git a/docs/diplomat-and-macros.md b/docs/diplomat-and-macros.md new file mode 100644 index 0000000000..9709b68d8e --- /dev/null +++ b/docs/diplomat-and-macros.md @@ -0,0 +1,186 @@ +# Comparing UniFFI with Diplomat + +[Diplomat](https://github.com/rust-diplomat/diplomat/) and [UniFFI](https://github.com/mozilla/uniffi-rs/) +are both tools which expose a rust implemented API over an FFI. +At face value, these tools are solving the exact same problem, but their approach +is significantly different. + +This document attempts to describe these different approaches and discuss the pros and cons of each. +It's not going to try and declare one better than the other, but instead just note how they differ. +If you are reading this hoping to find an answer to "what one should I use?", then that's easy - +each tool currently supports a unique set of foreign language bindings, so the tool you should +use is the one that supports the languages you care about! + +(There may even be a future where these 2 tools converge - that seems like a lot of work, but +might also provide a large payoff - more on this later) + +Disclaimer: This document was written by one of the UniFFI developers, who has never used +diplomat in anger. Please feel free to open PRs if anything here misrepresents diplomat. + +# The type systems + +The key different between these 2 tools is the "type system". While both are exposing Rust +code (which obviously comes with its own type system), the foreign bindings need to know +lots of details about all the types expressed by the tool. + +For the sake of this document, we will use the term "type universe" to define the set of +all types known by each of the tools. Both of these tools build their own "type universe" then +use that to generate both Rust code and foreign bindings. + +## UniFFI's type universe +UniFFI's model is to parse an external ffi description from a `.udl` file which describes the +entire "type universe". This type universe is then used to generate both the Rust scaffolding +(on disk as a `.rs` file) and the foreign bindings. + +**What's good about this** is that the entire type system is known when generating both the rust code +and the foreign binding. + +**What's bad about this** is that the external UDL is very ugly and redundant in terms of the +implemented rust API. + +## Diplomat's type universe + +Diplomat defines its "type universe" (ie, the external ffi) using macros. + +**What's good about this** is that the "ffi module" defines the canonical API and it is defined in +terms of Rust types - the redundant UDL is removed. The Rust scaffolding can also be generated +by the macros, meaning there are no generated `.rs` files involved. + +Ryan even tried this for UniFFI in [#416](https://github.com/mozilla/uniffi-rs/pull/416) - but we +struck **what's bad about this**: the context in which the macro runs doesn't know about types defined +outside of that macro, which are what we need to expose. + +## Limitations in the macro approach + +Let's look at diplomat's simple example: + +```rust +#[diplomat::bridge] +mod ffi { + pub struct MyFFIType { + pub a: i32, + pub b: bool, + } + + impl MyFFIType { + pub fn create() -> MyFFIType { ... } + ... + } +} +``` + +This works fine, but starts to come unstuck if you want the types defined somewhere else. In this trivial example, something like: + +```Rust +pub struct MyFFIType { + pub a: i32, + pub b: bool, +} + +#[diplomat::bridge] +mod ffi { + impl MyFFIType { + pub fn create() -> MyFFIType { ... } + ... + } +} +``` + +fails - diplomat can't handle this scenario - in the same way and for the same reasons that Ryan's +[#416](https://github.com/mozilla/uniffi-rs/pull/416) can't - the contents of the struct aren't known. + +From the Rust side of the world, this is probably solvable by sprinkling more macros around - eg, something like: + +```Rust +#[uniffi::magic] +pub struct MyFFIType { + pub a: i32, + pub b: bool, +} +``` + +Might be enough for the generation of the Rust scaffolding. However, the problems are in the foreign bindings. + +## How the type universe is constructed for the macro approach. + +In both diplomat and [#416](https://github.com/mozilla/uniffi-rs/pull/416), the approach taken +is that the generation process wants a path to the Rust source file that contains the module in +question - in the example above, the `ffi` module annotated with `#[diplomat:bridge]`. They both +use the `syn` crate to parse the Rust code inside this module, build their type universe, then +generate the foreign bindings. + +In our problematic example above, this process never sees the layout of the `MyFFIType` struct, +and nor does it see any macros annotating them. + +For this approach to work, it would be necessary for this process to compile the entire crate, +including depedent crates - the actual definition of all the types might appear anywhere. +Not only would this be slow, it's not clear it could be made to work - it might be reasonable to +have constraints on what can appear in just the `ffi` mod, but if we started adding constraints +to the entire crate, the tool would become far less useful. + +This is the exact same problem which caused us to decide to stop working on +[#416](https://github.com/mozilla/uniffi-rs/pull/416) - the current world where the type universe +is described externally doesn't have this problem - only the UDL file needs to be parsed when +generating the foreign bindings - Rust code isn't considered. The application-services team has +concluded that none of our non-trival use-cases for UniFFI could be described using macros, +so supporting both mechanisms is pain for no gain. + +As noted in #416, `wasm-bindgen` has a similarly shaped problem, and solves it by having +the Rust macro arrange for the resulting library to have an extra data section with the +serialized "type universe" - foreign binding generation would then read this information from the +already built binary. This sounds more complex than the UniFFI team has appetite for at +the current time. + +## Is this a problem for users of diplomat? Will diplomat solve it? + +I couldn't find real examples using diplomat, so it's difficult to know if this +is a problem in practice. UniFFI came from a world where we had Rust crates and +a hand-written FFI that exposed types from all over the crate. If these tools +had started with the limitations from the macro approach in mind, it's possible +a different, acceptable design might have been made to work. Maybe duplicating +some structs and supplying suitable `Into` implementations might make things workable? + +Diplomat comes from a very smart team. They may well come up with a novel solution, so +UniFFI should track the progress of that project to see what we can gleefully steal +in the future. As discussed below, a kind of "hybrid" approach might even be possible. + +# Looking forward + +Before looking forward, let's step back a little - both UniFFI and diplomat are solving the exact +same use-cases, just using a different approach to defining the type universe. +But if we ignore that, the tools take the same basic approach - they all build the +type universe, then use the representation of this type universe to define both Rust +and foreign bindings. + +The type universe described by diplomat is somewhat "leaner" than that described by UniFFI - +Rust types are the first-class citizens in the universe. UniFFI defines an external type model - +for example, there's a `Type` enum where, for example, `Type::Record(Record)` represents a +Rust struct. In other words, diplomat's type world can not be divorced from Rust, +whereas UniFFI's already is. + +That said though, there might be a future where merging or otherwise creating some +interoperability between these type universes might make sense. You could imagine +a world where you can use diplomat to describe your type universe, but use UniFFI's foreign +generation code to generate the Kotlin bindings. Similarly, a world where you use UniFFI +and UDL files to describe your type universe, but then use diplomat to generate +the NodeJS bindings. + +Or to put it another way, you could imagine a world where both tools are split into a +"describe the type universe" portion and a "build the bindings" portion, and these tools +could be used together. + +Sadly, that looks like alot of work, so someone would probably need to find a compelling +actual use-case to perform this work. + +# Next steps for UniFFI + +As much as some of the UniFFI team dislike the external UDL file, there's no clear path to +moving away from it. The macro approach is too limiting, and no other promising opportunities +have presented themselves. There's no clear alternative to UDL which allows a complex +type universe to be described, and at this stage, any replacement would need to be +compelling enough to make a change worthwhile, which is hard to imagine. + +In the short term, the best we can probably do is to enumerate the perceived problems +with the UDL file and try to make them more ergonomic - for example, avoiding repetition of +`[Throws=SomeError]` would remove alot of noise, and some strategy for generating +documentation might go a long way. From 631dea032763c9c1b1aa224dc6b6b0725c0adce8 Mon Sep 17 00:00:00 2001 From: Mark Hammond Date: Mon, 3 Jan 2022 17:13:48 +1100 Subject: [PATCH 2/7] Address first round of feedback --- docs/diplomat-and-macros.md | 199 ++++++++++++++++++++++++------------ 1 file changed, 136 insertions(+), 63 deletions(-) diff --git a/docs/diplomat-and-macros.md b/docs/diplomat-and-macros.md index 9709b68d8e..52738c9a57 100644 --- a/docs/diplomat-and-macros.md +++ b/docs/diplomat-and-macros.md @@ -11,15 +11,17 @@ If you are reading this hoping to find an answer to "what one should I use?", th each tool currently supports a unique set of foreign language bindings, so the tool you should use is the one that supports the languages you care about! -(There may even be a future where these 2 tools converge - that seems like a lot of work, but -might also provide a large payoff - more on this later) - Disclaimer: This document was written by one of the UniFFI developers, who has never used diplomat in anger. Please feel free to open PRs if anything here misrepresents diplomat. +See also: This document was discussed in [this PR](https://github.com/mozilla/uniffi-rs/pull/1146), +which has some very interesting discussion - indeed, some of the content in this document has +been copy-pasted from that discussion, but there's further detail there which might be +of interest. + # The type systems -The key different between these 2 tools is the "type system". While both are exposing Rust +The key difference between these 2 tools is the "type system". While both are exposing Rust code (which obviously comes with its own type system), the foreign bindings need to know lots of details about all the types expressed by the tool. @@ -28,12 +30,14 @@ all types known by each of the tools. Both of these tools build their own "type use that to generate both Rust code and foreign bindings. ## UniFFI's type universe + UniFFI's model is to parse an external ffi description from a `.udl` file which describes the entire "type universe". This type universe is then used to generate both the Rust scaffolding (on disk as a `.rs` file) and the foreign bindings. **What's good about this** is that the entire type system is known when generating both the rust code -and the foreign binding. +and the foreign binding, and is known without parsing any Rust code. This is important because +things like field names and types in structs must be known on both sides of the FFI. **What's bad about this** is that the external UDL is very ugly and redundant in terms of the implemented rust API. @@ -42,15 +46,28 @@ implemented rust API. Diplomat defines its "type universe" (ie, the external ffi) using macros. -**What's good about this** is that the "ffi module" defines the canonical API and it is defined in -terms of Rust types - the redundant UDL is removed. The Rust scaffolding can also be generated -by the macros, meaning there are no generated `.rs` files involved. +**What's good about this** is that an "ffi module" (and there may be many) defines the canonical API +and it is defined in terms of Rust types - the redundant UDL is removed. +The Rust scaffolding can also be generated by the macros, meaning there are no generated `.rs` +files involved. + +It also offers better control over the stability of the API, because where the FFI is defined +is constrained. This is [an explicit design decision](https://github.com/rust-diplomat/diplomat/blob/main/docs/design_doc.md#requirements) of diplomat. -Ryan even tried this for UniFFI in [#416](https://github.com/mozilla/uniffi-rs/pull/416) - but we -struck **what's bad about this**: the context in which the macro runs doesn't know about types defined -outside of that macro, which are what we need to expose. +While the process for defining the type universe is different, the actual in-memory +representation of that type universe isn't radically different from UniFFI - for example, +here's the [definition of a Rust struct](https://github.com/rust-diplomat/diplomat/blob/main/core/src/ast/structs.rs), +and while it is built from a `syn` struct, the final representation is independent of `syn` +and its ast representation. -## Limitations in the macro approach +## UniFFI's experience with the macro approach. + +Ryan tried this same macro approach for UniFFI in [#416](https://github.com/mozilla/uniffi-rs/pull/416) - +but we struck **what's bad about this** - at least for UniFFI's use-cases: the context in which the +macro runs doesn't know about types defined outside of that macro, which are what we need to +expose. + +### Example of these limitations Let's look at diplomat's simple example: @@ -99,29 +116,40 @@ pub struct MyFFIType { } ``` -Might be enough for the generation of the Rust scaffolding. However, the problems are in the foreign bindings. +Might be enough for the generation of the Rust scaffolding. However, the problems are in the +foreign bindings, because, eg, those foreign bindings do not know the names and types of the +struct elements. + +### Why is this considered a limition for UniFFI but not diplomat? + +As mentioned above, diplomat considers the limitation described above as an intentional design +feature. By limiting where FFI types can be described, there's no risk of changes made "far away" +from the FFI to change the FFI. This was born of experience in tools like `cbindgen`. + +For Uniffi, all use-cases needed by Mozilla don't share this design goal, primarily because the +FFI is the primary consumer of the crate. The Rust API exists purely to service the FFI. It's not +really possible to accidentally change the API, because every API change made will be in service +of exposing that change over the FFI. The test suites written in the foreign languages are +considered canonical. ## How the type universe is constructed for the macro approach. In both diplomat and [#416](https://github.com/mozilla/uniffi-rs/pull/416), the approach taken -is that the generation process wants a path to the Rust source file that contains the module in -question - in the example above, the `ffi` module annotated with `#[diplomat:bridge]`. They both -use the `syn` crate to parse the Rust code inside this module, build their type universe, then -generate the foreign bindings. +is very similar - it takes a path to a the Rust source file/tree, and uses `syn` to locate the special modules (ie, ones annotated with `#[diplomat:bridge]` in the case of diplomat.) -In our problematic example above, this process never sees the layout of the `MyFFIType` struct, -and nor does it see any macros annotating them. +While some details differ, this is just a matter of implementation - #416 isn't quite as agressive +about consuming the entire crate to find multiple FFI modules (and even then, diplomat doesn't +actually *process* the entire crate, just modules tagged as a bridge), but could easily be +extended to do so. -For this approach to work, it would be necessary for this process to compile the entire crate, -including depedent crates - the actual definition of all the types might appear anywhere. -Not only would this be slow, it's not clear it could be made to work - it might be reasonable to -have constraints on what can appear in just the `ffi` mod, but if we started adding constraints -to the entire crate, the tool would become far less useful. +But in both cases, for our problematic example above, this process never sees the layout of the +`MyFFIType` struct, so that layout can't be communicated to the foreign bindings. +As noted above, this is considered a feature for diplomat, but a limitation for UniFFI. -This is the exact same problem which caused us to decide to stop working on +This is the problem which caused us to decide to stop working on [#416](https://github.com/mozilla/uniffi-rs/pull/416) - the current world where the type universe -is described externally doesn't have this problem - only the UDL file needs to be parsed when -generating the foreign bindings - Rust code isn't considered. The application-services team has +is described externally doesn't have this limitation - only the UDL file needs to be parsed when +generating the foreign bindings. The application-services team has concluded that none of our non-trival use-cases for UniFFI could be described using macros, so supporting both mechanisms is pain for no gain. @@ -131,54 +159,99 @@ serialized "type universe" - foreign binding generation would then read this inf already built binary. This sounds more complex than the UniFFI team has appetite for at the current time. -## Is this a problem for users of diplomat? Will diplomat solve it? +# Looking forward + +## Adapting the diplomat/[#416](https://github.com/mozilla/uniffi-rs/pull/416) model to process the entire crate? -I couldn't find real examples using diplomat, so it's difficult to know if this -is a problem in practice. UniFFI came from a world where we had Rust crates and -a hand-written FFI that exposed types from all over the crate. If these tools -had started with the limitations from the macro approach in mind, it's possible -a different, acceptable design might have been made to work. Maybe duplicating -some structs and supplying suitable `Into` implementations might make things workable? +We noted that diplomat intentionally restricts where the ffi is generated, +whereas UniFFI considers that a limitation - but what if we can teach UniFFI to process +more of the Rust crate? -Diplomat comes from a very smart team. They may well come up with a novel solution, so -UniFFI should track the progress of that project to see what we can gleefully steal -in the future. As discussed below, a kind of "hybrid" approach might even be possible. +It might be reasonable for the foreign bindings to know that Rust "paths" to modules which should +be processed, and inside those modules find structs "marked up" as being used by the FFI. -# Looking forward +In other words, borrowing the example above: + +```Rust +#[uniffi::magic] +pub struct MyFFIType { + pub a: i32, + pub b: bool, +} +``` + +maybe can made to work, so long as we are happy to help UniFFI discover where such annotations +may exist. A complication here is that currently UniFFI allows types defined in external crates, +but that might still be workable. + +## Duplicating structs inside Rust + +When reviewing the draft of this document, @rfk noted that we are already duplicating Rust structs +in UDL and in Rust. So instead of having: + +``` +// In a UDL file: +dictionary MyFFIType { + i32 a; + bool b; +}; +// Then in Rust: +pub struct MyFFIType { + pub a: i32, + pub b: bool, +} +``` + +we could have: +```rust +// In the Rust implementation, in some other module. +pub struct MyFFIType { + pub a: i32, + pub b: bool, +} + +// And to expose it over the FFI: +#[ffi::something] +mod ffi { + + #[ffi::magic_external_type_declaration] + pub struct MyFFIType { + pub a: i32, + pub b: bool, + } + + impl MyFFIType { + pub fn create() -> MyFFIType { ... } + ... + } +} +``` -Before looking forward, let's step back a little - both UniFFI and diplomat are solving the exact -same use-cases, just using a different approach to defining the type universe. -But if we ignore that, the tools take the same basic approach - they all build the -type universe, then use the representation of this type universe to define both Rust -and foreign bindings. +So while we haven't exactly reduced the duplication, we have removed the UDL. +We probably also haven't helped with documentation, because the natural location for +the documentation of `MyFFIType` is probably at the *actual* implementation. -The type universe described by diplomat is somewhat "leaner" than that described by UniFFI - -Rust types are the first-class citizens in the universe. UniFFI defines an external type model - -for example, there's a `Type` enum where, for example, `Type::Record(Record)` represents a -Rust struct. In other words, diplomat's type world can not be divorced from Rust, -whereas UniFFI's already is. +## Try and share some definitions with diplomat -That said though, there might be a future where merging or otherwise creating some -interoperability between these type universes might make sense. You could imagine -a world where you can use diplomat to describe your type universe, but use UniFFI's foreign -generation code to generate the Kotlin bindings. Similarly, a world where you use UniFFI -and UDL files to describe your type universe, but then use diplomat to generate -the NodeJS bindings. +We note above that the type universe described by diplomat is somewhat "leaner" than that +described by UniFFI, but in general they are very similar. Thus, there might be a future where +merging or otherwise creating some interoperability between these type universes might make +sense. -Or to put it another way, you could imagine a world where both tools are split into a -"describe the type universe" portion and a "build the bindings" portion, and these tools -could be used together. +It seems likely that this would start to add unwelcome constraints - eg, diplomat would not want +its ability to refactor type representations limited by what UniFFI needs. -Sadly, that looks like alot of work, so someone would probably need to find a compelling -actual use-case to perform this work. +However, what you could see happening in the future is UniFFI becoming a kind of higher-level +wrapper around Diplomat. You can imagine a Diplomat backend for UniFFI that converts a .udl file +into a bridge module and then uses the Diplomat toolchain to generate bindings from it, +keeping some of the additional affordances/conveniences UniFFI built for its specific use-cases. # Next steps for UniFFI As much as some of the UniFFI team dislike the external UDL file, there's no clear path to -moving away from it. The macro approach is too limiting, and no other promising opportunities -have presented themselves. There's no clear alternative to UDL which allows a complex -type universe to be described, and at this stage, any replacement would need to be -compelling enough to make a change worthwhile, which is hard to imagine. +moving away from it. We could experiment with some of the options above and see if they are +both viable and worth the investment for the UniFFI use-cases. That sounds like a long-term +goal. In the short term, the best we can probably do is to enumerate the perceived problems with the UDL file and try to make them more ergonomic - for example, avoiding repetition of From f024f03a674f4b42765106764bf38720ca0d8ced Mon Sep 17 00:00:00 2001 From: Mark Hammond Date: Mon, 3 Jan 2022 17:24:14 +1100 Subject: [PATCH 3/7] (it would help if I hit 'save' before commiting!) --- docs/diplomat-and-macros.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/docs/diplomat-and-macros.md b/docs/diplomat-and-macros.md index 52738c9a57..3da99b5719 100644 --- a/docs/diplomat-and-macros.md +++ b/docs/diplomat-and-macros.md @@ -49,7 +49,9 @@ Diplomat defines its "type universe" (ie, the external ffi) using macros. **What's good about this** is that an "ffi module" (and there may be many) defines the canonical API and it is defined in terms of Rust types - the redundant UDL is removed. The Rust scaffolding can also be generated by the macros, meaning there are no generated `.rs` -files involved. +files involved. Types can be shared among any of the ffi modules defined in the project - +for example, [this diplomat ffi module](https://github.com/unicode-org/icu4x/blob/7d9f89fcd7df4567e17ddd8c46810b0db287436a/ffi/diplomat/src/pluralrules.rs#L50-L51) +uses types from a [different ffi module](https://github.com/unicode-org/icu4x/blob/7d9f89fcd7df4567e17ddd8c46810b0db287436a/ffi/diplomat/src/locale.rs#L19). It also offers better control over the stability of the API, because where the FFI is defined is constrained. This is [an explicit design decision](https://github.com/rust-diplomat/diplomat/blob/main/docs/design_doc.md#requirements) of diplomat. @@ -181,8 +183,11 @@ pub struct MyFFIType { ``` maybe can made to work, so long as we are happy to help UniFFI discover where such annotations -may exist. A complication here is that currently UniFFI allows types defined in external crates, -but that might still be workable. +may exist. + +A complication here is that currently UniFFI allows types defined in external crates, +but that might still be workable - eg, +[diplomat has an issue open to support exactly this](https://github.com/rust-diplomat/diplomat/issues/34) ## Duplicating structs inside Rust From a86e9496623f9bbee77fb7b3b23e960ee47ebdc2 Mon Sep 17 00:00:00 2001 From: Mark Hammond Date: Wed, 5 Jan 2022 17:36:31 +1100 Subject: [PATCH 4/7] review comments from ben --- docs/diplomat-and-macros.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/docs/diplomat-and-macros.md b/docs/diplomat-and-macros.md index 3da99b5719..b563571743 100644 --- a/docs/diplomat-and-macros.md +++ b/docs/diplomat-and-macros.md @@ -118,9 +118,14 @@ pub struct MyFFIType { } ``` -Might be enough for the generation of the Rust scaffolding. However, the problems are in the -foreign bindings, because, eg, those foreign bindings do not know the names and types of the -struct elements. +Might be enough for the generation of the Rust scaffolding - in UniFFI's case, all we really need +is an implementation of `uniffi::RustBufferViaFfi` which is easy to derive, and UniFFI can +generate code which assumes that exists, much like it does now. +However, the problems are in the foreign bindings, because, eg, those foreign bindings do not know +the names and types of the struct elements without re-parsing every bit of Rust code with those +annotations. As discussed below, re-parsing this code might be an option if we help Uniffi to +find it, but asking UniFFI to parse this and all dependent crates to auto-discover them +probably is not going to be viable. ### Why is this considered a limition for UniFFI but not diplomat? @@ -236,6 +241,9 @@ So while we haven't exactly reduced the duplication, we have removed the UDL. We probably also haven't helped with documentation, because the natural location for the documentation of `MyFFIType` is probably at the *actual* implementation. +While it might not solve all our problems, it is worthy of serious consideration - fewer problems +is still a worthwhile goal, and needing a UDL file and parser seems like one worth removing. + ## Try and share some definitions with diplomat We note above that the type universe described by diplomat is somewhat "leaner" than that From eb044e2d45fa9915ef92fd9770276d54b816a2e5 Mon Sep 17 00:00:00 2001 From: Mark Hammond Date: Thu, 6 Jan 2022 13:40:12 +1100 Subject: [PATCH 5/7] Ryan's comments and a few other tweaks --- docs/diplomat-and-macros.md | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/docs/diplomat-and-macros.md b/docs/diplomat-and-macros.md index b563571743..049a81dcd1 100644 --- a/docs/diplomat-and-macros.md +++ b/docs/diplomat-and-macros.md @@ -53,8 +53,10 @@ files involved. Types can be shared among any of the ffi modules defined in the for example, [this diplomat ffi module](https://github.com/unicode-org/icu4x/blob/7d9f89fcd7df4567e17ddd8c46810b0db287436a/ffi/diplomat/src/pluralrules.rs#L50-L51) uses types from a [different ffi module](https://github.com/unicode-org/icu4x/blob/7d9f89fcd7df4567e17ddd8c46810b0db287436a/ffi/diplomat/src/locale.rs#L19). -It also offers better control over the stability of the API, because where the FFI is defined -is constrained. This is [an explicit design decision](https://github.com/rust-diplomat/diplomat/blob/main/docs/design_doc.md#requirements) of diplomat. +Restricting the definition of the FFI to a single module instead of allowing that definition +to appear in any Rust code in the crate also offers better control over the stability of the API, +because where the FFI is defined is constrained. This is +[an explicit design decision](https://github.com/rust-diplomat/diplomat/blob/main/docs/design_doc.md#requirements) of diplomat. While the process for defining the type universe is different, the actual in-memory representation of that type universe isn't radically different from UniFFI - for example, @@ -65,11 +67,11 @@ and its ast representation. ## UniFFI's experience with the macro approach. Ryan tried this same macro approach for UniFFI in [#416](https://github.com/mozilla/uniffi-rs/pull/416) - -but we struck **what's bad about this** - at least for UniFFI's use-cases: the context in which the +but we struck a limitation in this approach for UniFFI's use-cases - the context in which the macro runs doesn't know about types defined outside of that macro, which are what we need to expose. -### Example of these limitations +### Example of this limitation Let's look at diplomat's simple example: @@ -118,10 +120,10 @@ pub struct MyFFIType { } ``` -Might be enough for the generation of the Rust scaffolding - in UniFFI's case, all we really need +might be enough for the generation of the Rust scaffolding - in UniFFI's case, all we really need is an implementation of `uniffi::RustBufferViaFfi` which is easy to derive, and UniFFI can -generate code which assumes that exists, much like it does now. -However, the problems are in the foreign bindings, because, eg, those foreign bindings do not know +generate code which assumes that exists much like it does now. +However, the problems are in the foreign bindings, because those foreign bindings do not know the names and types of the struct elements without re-parsing every bit of Rust code with those annotations. As discussed below, re-parsing this code might be an option if we help Uniffi to find it, but asking UniFFI to parse this and all dependent crates to auto-discover them @@ -150,7 +152,8 @@ actually *process* the entire crate, just modules tagged as a bridge), but could extended to do so. But in both cases, for our problematic example above, this process never sees the layout of the -`MyFFIType` struct, so that layout can't be communicated to the foreign bindings. +`MyFFIType` struct because it's not inside the processed module, so that layout can't be +communicated to the foreign bindings. As noted above, this is considered a feature for diplomat, but a limitation for UniFFI. This is the problem which caused us to decide to stop working on @@ -187,7 +190,7 @@ pub struct MyFFIType { } ``` -maybe can made to work, so long as we are happy to help UniFFI discover where such annotations +maybe can be made to work, so long as we are happy to help UniFFI discover where such annotations may exist. A complication here is that currently UniFFI allows types defined in external crates, From 2d22d03f246c3e20662aabbfa1daa4c16da2da3f Mon Sep 17 00:00:00 2001 From: Mark Hammond Date: Thu, 6 Jan 2022 14:52:47 +1100 Subject: [PATCH 6/7] Add note to readme --- README.md | 40 +++++++++------------------------------- 1 file changed, 9 insertions(+), 31 deletions(-) diff --git a/README.md b/README.md index 54a7d0e809..9a021b019c 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,15 @@ A portmanteau word that also puns with "unify", to signify the joining of one co uni - [Latin ūni-, from ūnus, one] FFI - [Abbreviation, Foreign Function Interface] +## Alternative tools + +Other tools we know of which try and solve a similarly shaped problem are: + +* [Diplomat](https://github.com/rust-diplomat/diplomat/) - see our [writeup of + the different approach taken by that tool](https://github.com/mozilla/uniffi-rs/tree/main/docs/diplomat-and-macros.md) + +(Please open a PR if you think other tools should be listed!) + ## Contributing If this tool sounds interesting to you, please help us develop it! You can: @@ -47,34 +56,3 @@ If this tool sounds interesting to you, please help us develop it! You can: ## Code of Conduct This project is governed by Mozilla's [Community Participation Guidelines](./CODE_OF_CONDUCT.md). - ---- - -(Versions `v0.9.0` though `v0.11.0` include a deprecation notice that links to this README. Once those versions have -sufficiently aged out, this section can be removed from the top-level README.) - -### Thread Safety - -It is your responsibility to ensure the structs you expose via UniFFI are -all `Send+Sync`. This will be enforced by the Rust compiler, likely with an -inscrutable error from somewhere in UniFFI's generated Rust code. - -Early versions of this crate automatically wrapped rust structs in a mutex, -thus implicitly making the interfaces thread-safe and safe to be called -over the FFI by any thread. However, in practice we found this to be a -mis-feature, so version 0.7 first introduced the ability for the component -author to opt-out of this implicit wrapping and take care of thread-safety -themselves by adding a `[Threadsafe]` attribute to the interface. - -Version 0.9.0 took this further, and interfaces not marked as `[Threadsafe]` -started issuing a deprecation warning. If you are seeing these deprecation warnings, -you should upgrade your component as soon as possible. For an example of what kind of -effort is required to make your interfaces thread-safe, you might like to see -[this commit](https://github.com/mozilla/uniffi-rs/commit/454dfff6aa560dffad980a9258853108a44d5985) -where we made one the examples thread-safe. - -As of version 0.11.0, all interfaces will be required to be `Send+Sync`, and the -`[Threadsafe]` attribute will be deprecated and ignored. - -See also [adr-0004](https://github.com/mozilla/uniffi-rs/blob/main/docs/adr/0004-only-threadsafe-interfaces.md) -which outlines the reasoning behind this decision. \ No newline at end of file From ee6ee35e99cd6cf4e3965b4a22ead72cd0cd7992 Mon Sep 17 00:00:00 2001 From: Mark Hammond Date: Thu, 6 Jan 2022 14:53:44 +1100 Subject: [PATCH 7/7] use relative link in readme --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 9a021b019c..fce63e5e14 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,7 @@ FFI - [Abbreviation, Foreign Function Interface] Other tools we know of which try and solve a similarly shaped problem are: * [Diplomat](https://github.com/rust-diplomat/diplomat/) - see our [writeup of - the different approach taken by that tool](https://github.com/mozilla/uniffi-rs/tree/main/docs/diplomat-and-macros.md) + the different approach taken by that tool](docs/diplomat-and-macros.md) (Please open a PR if you think other tools should be listed!)