Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wasm support for client using grpc-web #23

Closed
wants to merge 10 commits into from

Conversation

shravanshetty1
Copy link

No description provided.

@shravanshetty1 shravanshetty1 changed the title Wasm support for client using grpc-web [WIP] Wasm support for client using grpc-web Jun 22, 2021
@shravanshetty1 shravanshetty1 changed the title [WIP] Wasm support for client using grpc-web Wasm support for client using grpc-web Jun 23, 2021
@shravanshetty1 shravanshetty1 changed the title Wasm support for client using grpc-web [WIP] Wasm support for client using grpc-web Jun 23, 2021
@shravanshetty1
Copy link
Author

TODO - allow user to give client, this way we can support both grpc-web and grpc and remove a dependency

@shravanshetty1 shravanshetty1 changed the title [WIP] Wasm support for client using grpc-web Wasm support for client using grpc-web Jun 23, 2021
@shravanshetty1
Copy link
Author

@manu0466 My implementation only uses grpc_web, ideally the user should be able to use grpc_web or grpc based on their situation. This can be accomplished by allowing them to inject a generic grpc client into the cosmos_client.

Currently my rust knowledge is not strong enough to implement it correctly.

@manu0466
Copy link

@manu0466 My implementation only uses grpc_web, ideally the user should be able to use grpc_web or grpc based on their situation. This can be accomplished by allowing them to inject a generic grpc client into the cosmos_client.

Currently my rust knowledge is not strong enough to implement it correctly.

Hi @shravanshetty1 thanks for your work 🙏.
I think that the user don't need to know what to inject, this can be automaticaly determinated at compile time based on the target.

To do that i suggest you to create 2 private function with the same name like grpc_client and then adding a conditional compilation so that only one of this 2 fuctions is compiled.

Here i provide you an actual example that should work

#[cfg(all(target_arch = "wasm32", target_os = "unknown"))]
use grpc_web_client::Client;
#[cfg(any(
    target_os = "linux",
    target_os = "windows",
    target_os = "macos",
    target_os = "android",
    target_os = "ios"
))]
use tonic::{
    codegen::http::Uri,
    transport::{Channel, Endpoint},
    Request,
};

impl CosmosClient {
....
   #[cfg(all(target_arch = "wasm32", target_os = "unknown"))]
    async fn grpc_client(&self) -> Result<Client, CosmosError> {
        Ok(Client::new(self.grpc_addr.clone()))
    }

    #[cfg(any(
        target_os = "linux",
        target_os = "windows",
        target_os = "macos",
        target_os = "android",
        target_os = "ios"
    ))]
    async fn grpc_client(&self) -> Result<Channel, CosmosError> {
        let grpc_uri = self.grpc_addr.parse::<Uri>()?;
        let grpc_channel = Channel::builder(grpc_uri);

        Ok(grpc_channel
            .connect()
            .await
            .map_err(|err| CosmosError::Grpc(err.to_string()))?)
    }
...
}

After adding this 2 fuctions you can simply call self.grpc_client().await?; that will provide you the right client for the current target.

After this modifications you must make sure that the right dependencies are available at compile time. To do so you should tell cargo to use grpc_web only for wasm and tonic for the other targets.
Here you can find a Cargo.toml file that you can use as reference.

@shravanshetty1
Copy link
Author

shravanshetty1 commented Jun 26, 2021

Made wasm compatibility into a feature,

wasm-pack build --target web -- --no-default-features --features wasm

default

cargo build

@shravanshetty1
Copy link
Author

@manu0466

@manu0466
Copy link

@shravanshetty1 Well done!! Only one thing I think that the feature wasm and no-wasm can be avoided since the dependencies and the code to compile can be determinated using target_os.
This will improve also the use of this crate from other projetc since the user don't need to enable the wasm feature when compiling to wasm.

@shravanshetty1
Copy link
Author

@shravanshetty1 Well done!! Only one thing I think that the feature wasm and no-wasm can be avoided since the dependencies and the code to compile can be determinated using target_os.
This will improve also the use of this crate from other projetc since the user don't need to enable the wasm feature when compiling to wasm.

I initially tried to conditionally compile tonic "transport" feature based on target but couldnt find any work around. But after doing research again I was able to find this rust-lang/cargo#7914. This will force us to add build arguement while compiling same as feature. So i dont see any reason to implement this either since we are just changing the build argument.

@manu0466
Copy link

manu0466 commented Jul 1, 2021

@shravanshetty1 Well done!! Only one thing I think that the feature wasm and no-wasm can be avoided since the dependencies and the code to compile can be determinated using target_os.
This will improve also the use of this crate from other projetc since the user don't need to enable the wasm feature when compiling to wasm.

I initially tried to conditionally compile tonic "transport" feature based on target but couldnt find any work around. But after doing research again I was able to find this rust-lang/cargo#7914. This will force us to add build arguement while compiling same as feature. So i dont see any reason to implement this either since we are just changing the build argument.

Seams that the problem was fixed with the new Feature Resolver 2.
Adding resolver = "2" to the root Cargo.toml file allowed me to correctly compile the project without the extra features.
This is a portion of the Cargo.toml that worked for me:

[dependencies]
cosmos-sdk-proto = { version = "0.5.0" }
prost = { version = "0.7.0"}
prost-types = { version = "0.7" }
reqwest = { version = "=0.10.10", features = ["json"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = { version = "1.0.62" }
thiserror = "1.0.24"
crw-wallet = { path = "../../packages/crw-wallet", version = "0.1.0" }

[target.'cfg(not(all(target_arch = "wasm32", target_os = "unknown")))'.dependencies]
tonic = { version = "0.4.1", features = ["transport"] }

[target.'cfg(all(target_arch = "wasm32", target_os = "unknown"))'.dependencies]
tonic = { version = "0.4.1", default-features = false }
grpc-web-client = { git = "https://github.com/titanous/grpc-web-client", branch = "main" }
getrandom = { version = "0.2", features = ["js"] }

@shravanshetty1
Copy link
Author

Doesnt seem to work for me, here is some info:
Cargo.toml -

[package]
name = "crw-client"
version = "0.1.0"
authors = ["bragaz <leo.braga95@gmail.com"]
edition = "2018"
description = "Client package of cosmos-rust-wallet to broadcast tx and query data from a cosmos based blockchain"
license = "Apache-2.0"
repository = "https://github.com/forbole/cosmos-rust-wallet"
keywords = ["blockchain", "cosmos", "cosmos-rust-wallet"]
resolver = "2"

[lib]
crate-type = ["cdylib", "lib"]

[dependencies]
cosmos-sdk-proto = { version = "0.5.0"}
prost = { version = "0.7.0"}
prost-types = { version = "0.7" }
reqwest = { version = "=0.10.10", features = ["json"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = { version = "1.0.62" }
crw-wallet = { path = "../../packages/crw-wallet", version = "0.1.0" }
thiserror = "1.0.24"

[target.'cfg(not(all(target_arch = "wasm32", target_os = "unknown")))'.dependencies]
tonic = { version = "0.4.1", features = ["transport"] }

[target.'cfg(all(target_arch = "wasm32", target_os = "unknown"))'.dependencies]
tonic = { version = "0.4.1", default-features = false }
grpc-web-client = { git = "https://github.com/titanous/grpc-web-client", branch = "main"}

[dev-dependencies]
wasm-bindgen-test = "0.3.20"

rustc

➜  crw-client git:(grpc-web) ✗ rustc --version
rustc 1.53.0 (53cb7b09b 2021-06-17)

wasm-pack

➜  crw-client git:(grpc-web) ✗ wasm-pack --version         
wasm-pack 0.9.1

wasm-pack command used

wasm-pack build --target web

@shravanshetty1
Copy link
Author

Logs

➜  crw-client git:(grpc-web) ✗ wasm-pack build --target web > out.txt                               
[INFO]: Checking for the Wasm target...
[INFO]: Compiling to Wasm...
warning: resolver for the non root package will be ignored, specify resolver at the workspace root:
package:   /home/anon/projects/cosmos/cosmos-rust-wallet/packages/crw-client/Cargo.toml
workspace: /home/anon/projects/cosmos/cosmos-rust-wallet/Cargo.toml
   Compiling socket2 v0.4.0
   Compiling bech32 v0.7.3
   Compiling byteorder v1.4.3
   Compiling rustc-hash v1.1.0
   Compiling mime v0.3.16
   Compiling bytes v0.5.6
   Compiling bech32 v0.8.1
   Compiling proc-macro2 v1.0.27
   Compiling syn v1.0.73
error[E0432]: unresolved import `crate::sys`
 --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/sockaddr.rs:5:12
  |
5 | use crate::sys::{
  |            ^^^ could not find `sys` in the crate root

error[E0432]: unresolved imports `crate::sys`, `crate::sys`
  --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/socket.rs:21:12
   |
21 | use crate::sys::{self, c_int, getsockopt, setsockopt, Bool};
   |            ^^^   ^^^^ no `sys` in the root
   |            |
   |            could not find `sys` in the crate root

error[E0432]: unresolved import `sys`
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:129:5
    |
129 | use sys::c_int;
    |     ^^^ use of undeclared crate or module `sys`

error[E0433]: failed to resolve: could not find `sys` in the crate root
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/sockaddr.rs:114:29
    |
114 |             let ip = crate::sys::from_in_addr(addr.sin_addr);
    |                             ^^^ could not find `sys` in the crate root

error[E0433]: failed to resolve: could not find `sys` in the crate root
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/sockaddr.rs:121:29
    |
121 |             let ip = crate::sys::from_in6_addr(addr.sin6_addr);
    |                             ^^^ could not find `sys` in the crate root

error[E0433]: failed to resolve: could not find `sys` in the crate root
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/sockaddr.rs:172:30
    |
172 |             sin_addr: crate::sys::to_in_addr(&addr.ip()),
    |                              ^^^ could not find `sys` in the crate root

error[E0433]: failed to resolve: could not find `sys` in the crate root
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/sockaddr.rs:207:31
    |
207 |             sin6_addr: crate::sys::to_in6_addr(addr.ip()),
    |                               ^^^ could not find `sys` in the crate root

error[E0433]: failed to resolve: use of undeclared crate or module `sys`
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:149:37
    |
149 |     pub const IPV4: Domain = Domain(sys::AF_INET);
    |                                     ^^^ use of undeclared crate or module `sys`

error[E0433]: failed to resolve: use of undeclared crate or module `sys`
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:152:37
    |
152 |     pub const IPV6: Domain = Domain(sys::AF_INET6);
    |                                     ^^^ use of undeclared crate or module `sys`

error[E0433]: failed to resolve: use of undeclared crate or module `sys`
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:191:35
    |
191 |     pub const STREAM: Type = Type(sys::SOCK_STREAM);
    |                                   ^^^ use of undeclared crate or module `sys`

error[E0433]: failed to resolve: use of undeclared crate or module `sys`
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:196:34
    |
196 |     pub const DGRAM: Type = Type(sys::SOCK_DGRAM);
    |                                  ^^^ use of undeclared crate or module `sys`

error[E0433]: failed to resolve: use of undeclared crate or module `sys`
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:231:43
    |
231 |     pub const ICMPV4: Protocol = Protocol(sys::IPPROTO_ICMP);
    |                                           ^^^ use of undeclared crate or module `sys`

error[E0433]: failed to resolve: use of undeclared crate or module `sys`
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:234:43
    |
234 |     pub const ICMPV6: Protocol = Protocol(sys::IPPROTO_ICMPV6);
    |                                           ^^^ use of undeclared crate or module `sys`

error[E0433]: failed to resolve: use of undeclared crate or module `sys`
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:237:40
    |
237 |     pub const TCP: Protocol = Protocol(sys::IPPROTO_TCP);
    |                                        ^^^ use of undeclared crate or module `sys`

error[E0433]: failed to resolve: use of undeclared crate or module `sys`
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:240:40
    |
240 |     pub const UDP: Protocol = Protocol(sys::IPPROTO_UDP);
    |                                        ^^^ use of undeclared crate or module `sys`

error[E0433]: failed to resolve: use of undeclared crate or module `sys`
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:272:18
    |
272 |         self.0 & sys::MSG_TRUNC != 0
    |                  ^^^ use of undeclared crate or module `sys`

error[E0433]: failed to resolve: use of undeclared crate or module `sys`
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:280:33
    |
280 | pub struct MaybeUninitSlice<'a>(sys::MaybeUninitSlice<'a>);
    |                                 ^^^ use of undeclared crate or module `sys`

error[E0433]: failed to resolve: use of undeclared crate or module `sys`
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:299:31
    |
299 |         MaybeUninitSlice(sys::MaybeUninitSlice::new(buf))
    |                               ^^^^^^^^^^^^^^^^ not found in `sys`
    |
help: consider importing this struct
    |
62  | use crate::socket::MaybeUninitSlice;
    |

   Compiling serde_derive v1.0.126
error[E0308]: mismatched types
    --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/socket.rs:1451:23
     |
1451 | from!(net::TcpStream, Socket);
     |                       ^^^^^^ expected struct `Socket`, found `()`
     | 
    ::: /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:104:16
     |
104  |             fn from(socket: $from) -> $for {
     |                ---- implicitly returns `()` as its body has no tail or `return` expression

error[E0308]: mismatched types
    --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/socket.rs:1452:25
     |
1452 | from!(net::TcpListener, Socket);
     |                         ^^^^^^ expected struct `Socket`, found `()`
     | 
    ::: /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:104:16
     |
104  |             fn from(socket: $from) -> $for {
     |                ---- implicitly returns `()` as its body has no tail or `return` expression

error[E0308]: mismatched types
    --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/socket.rs:1453:23
     |
1453 | from!(net::UdpSocket, Socket);
     |                       ^^^^^^ expected struct `Socket`, found `()`
     | 
    ::: /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:104:16
     |
104  |             fn from(socket: $from) -> $for {
     |                ---- implicitly returns `()` as its body has no tail or `return` expression

error[E0308]: mismatched types
    --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/socket.rs:1454:15
     |
1454 | from!(Socket, net::TcpStream);
     |               ^^^^^^^^^^^^^^ expected struct `TcpStream`, found `()`
     | 
    ::: /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:104:16
     |
104  |             fn from(socket: $from) -> $for {
     |                ---- implicitly returns `()` as its body has no tail or `return` expression

error[E0308]: mismatched types
    --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/socket.rs:1455:15
     |
1455 | from!(Socket, net::TcpListener);
     |               ^^^^^^^^^^^^^^^^ expected struct `TcpListener`, found `()`
     | 
    ::: /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:104:16
     |
104  |             fn from(socket: $from) -> $for {
     |                ---- implicitly returns `()` as its body has no tail or `return` expression

error[E0308]: mismatched types
    --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/socket.rs:1456:15
     |
1456 | from!(Socket, net::UdpSocket);
     |               ^^^^^^^^^^^^^^ expected struct `UdpSocket`, found `()`
     | 
    ::: /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/lib.rs:104:16
     |
104  |             fn from(socket: $from) -> $for {
     |                ---- implicitly returns `()` as its body has no tail or `return` expression

error[E0061]: this function takes 4 arguments but 3 arguments were supplied
   --> /home/anon/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/socket2-0.4.0/src/sockaddr.rs:123:33
    |
123 |             Some(SocketAddr::V6(SocketAddrV6::new(
    |                                 ^^^^^^^^^^^^^^^^^ expected 4 arguments
124 |                 ip,
    |                 --
125 |                 port,
    |                 ----
126 |                 addr.sin6_flowinfo,
    |                 ------------------ supplied 3 arguments
    |
note: associated function defined here
   --> /home/anon/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/net/addr.rs:371:12
    |
371 |     pub fn new(ip: Ipv6Addr, port: u16, flowinfo: u32, scope_id: u32) -> SocketAddrV6 {
    |            ^^^

   Compiling serde v1.0.126
error: aborting due to 25 previous errors

Some errors have detailed explanations: E0061, E0308, E0432, E0433.
For more information about an error, try `rustc --explain E0061`.
error: could not compile `socket2`

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: build failed
Error: Compiling your crate to WebAssembly failed
Caused by: failed to execute `cargo build`: exited with exit code: 101
  full command: "cargo" "build" "--lib" "--release" "--target" "wasm32-unknown-unknown"

@manu0466
Copy link

manu0466 commented Jul 3, 2021

I was having the same errors. You should add the resolver = "2" to the root Cargo.toml as suggested from the compiler:

warning: resolver for the non root package will be ignored, specify resolver at the workspace root:
package:   /home/anon/projects/cosmos/cosmos-rust-wallet/packages/crw-client/Cargo.toml
workspace: /home/anon/projects/cosmos/cosmos-rust-wallet/Cargo.toml

@shravanshetty1
Copy link
Author

shravanshetty1 commented Jul 3, 2021

@manu0466 Requested changes have been made

@shravanshetty1 shravanshetty1 changed the title Wasm support for client using grpc-web [WIP]Wasm support for client using grpc-web Jul 3, 2021
@shravanshetty1 shravanshetty1 changed the title [WIP]Wasm support for client using grpc-web Wasm support for client using grpc-web Jul 3, 2021
@shravanshetty1
Copy link
Author

@manu0466 after manual testing seems to work fine for both grpc and grpc-web

@@ -18,4 +19,4 @@ debug = false
debug-assertions = false

[patch.crates-io]
cosmos-sdk-proto = { git = "https://github.com/forbole/cosmos-rust", branch = "main"}
cosmos-sdk-proto = { git = "https://github.com/shravanshetty1/cosmos-rust", branch = "wasm2"}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use the upstream version instead of the your fork?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The upstream version is not wasm compatible

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prost = { version = "0.7.0"}
prost-types = { version = "0.7" }
reqwest = { version = "0.11.0", features = ["blocking", "json"]}
reqwest = { version = "=0.10.10", features = ["json"] }
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why exactly the 0.10.10 instead of the latest 0.11.4? Can we use the latest one?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0.11.0 onwards was not wasm compatible, cannot remember the exact issue - I can find out if you need to know

packages/crw-client/src/client.rs Outdated Show resolved Hide resolved
packages/crw-client/src/client.rs Outdated Show resolved Hide resolved
shravanshetty1 and others added 2 commits July 9, 2021 19:04
Co-authored-by: Manuel <manuel.turetta94@gmail.com>
Co-authored-by: Manuel <manuel.turetta94@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants