Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WASM support with grpc-web #97

Closed
wants to merge 43 commits into from
Closed

Conversation

shravanshetty1
Copy link

No description provided.

@shravanshetty1
Copy link
Author

@tony-iqlusion seems to work for grpc and grpc-web

This pull request introduces basic support for wasmd.

It includes compilation of the protobuf files and conversion of Msg types
into something that can be more easily put into Tx Body.
@tony-iqlusion
Copy link
Member

I'd like to bump prost and tonic before attempting this, but we're stuck waiting for tendermint-rs to update.

@romac
Copy link
Member

romac commented Jul 19, 2021

We can't update to prost 0.8 in tendermint-rs at the moment because Prost 0.8 includes a number of regressions which affect the generated code, notably tokio-rs/prost#502 / tokio-rs/prost#490.

dependabot bot and others added 24 commits July 26, 2021 10:03
Updates the requirements on [tendermint-proto](https://github.com/informalsystems/tendermint-rs) to permit the latest version.
- [Release notes](https://github.com/informalsystems/tendermint-rs/releases)
- [Changelog](https://github.com/informalsystems/tendermint-rs/blob/master/CHANGELOG.md)
- [Commits](informalsystems/tendermint-rs@v0.20.0...v0.21.0)

---
updated-dependencies:
- dependency-name: tendermint-proto
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
…v: v0.17.0)) (cosmos#109)

Co-authored-by: tony-iqlusion <tony-iqlusion@users.noreply.github.com>
…-proto-0.21

Update tendermint-proto requirement from 0.20 to 0.21
…0.17.0)) (cosmos#117)

Co-authored-by: zmanian <zmanian@users.noreply.github.com>
* Bump MSRV to 1.54

This is mainly to work around a very annoying bug in 1.51 which prevents
publishing any of the crates in this repo:

```
   Packaging cosmos_sdk v0.3.0 (src/cosmos-rust/cosmos-sdk-rs)
error: failed to prepare local package for uploading

Caused by:
  values must be emitted before tables
```

I'm not sure what the root cause is exactly, but whatever it is it
appears to be fixed in newer versions of Rust.

This commit bumps to 1.54, which is the latest stable release.

* clippy fixes
As noted on cosmos#82, the `amount` field of `MsgDelegate` is annotated as
non-nullable in the upstream cosmos-sdk:

https://github.com/cosmos/cosmos-sdk/blob/9fd866e3820b3510010ae172b682d71594cd8c14/proto/cosmos/staking/v1beta1/tx.proto#L89

```proto
message MsgDelegate {
  option (gogoproto.equal)           = false;
  option (gogoproto.goproto_getters) = false;

  string                   delegator_address = 1 [(gogoproto.moretags) = "yaml:\"delegator_address\""];
  string                   validator_address = 2 [(gogoproto.moretags) = "yaml:\"validator_address\""];
  cosmos.base.v1beta1.Coin amount            = 3 [(gogoproto.nullable) = false];
}
```

This commit changes the `amount` field of the domain type from
`Option<Coin>` to `Coin` to reflect that.
As discussed in cosmos#113, this renames "cosmos-sdk-rs" (i.e. the
`cosmos_sdk` crate) to `cosmrs`.
tony-iqlusion and others added 9 commits September 29, 2021 16:16
Adds support for the "Cosmos JSON" `PublicKey` serialization, e.g.:

    {"@type":"/cosmos.crypto.ed25519.PubKey","key":"sEEsVGkXvyewKLWMJbHVDRkBoerW0IIwmj1rHkabtHU="}

Encoding follows Protobub JSON conventions, with binary data encoded as
standard (i.e. non-URL safe) Base64. However, note that it's
structurally still a bit different from the Protobuf JSON encoding for
public keys, and thus entails a custom solution.

Also note that there is unfortunately not yet a general solution for
Protobuf JSON encoding for `prost::Message`:

tokio-rs/prost#75
…os#134)

...on the `Deserializer` and `Serializer` impls for `PublicKey`
Recently the cosmos team split the x/ibc module into its own
repo located at github.com/cosmos/ibc-go, these changes add
the ibc-go repo as a submodule, update cosmos-sdk-go to v0.44.1
and enable integration of ibc-go protos with the cosmos-sdk protos
In my effort to add support for the new cosmos/ibc-go repo,
jkilpatr discovered that the compile_*_proto_services() functions
recompile and overwrite files previously generated in the respective
compile_*_protos() functions. This change fixes cosmos-sdk proto
compilation by consolidating proto definitions and services into
one function: compile_sdk_protos_and_services().

Fixed compiled protos to follow.
See previous commit regarding the code fix leading to these updated
compiled proto files.
In v0.43.* the staking module introduced authz.proto, which has
an enum (oneof validators) and a struct (message Validators) both
defined under message StakeAuthorization. When this struct runs through
prost it creates a namespace conflict. This commit simply renames
those conflicts. Since this StakeAuthorization is new to cosmos-rust
no downstream issues should ocurr.
…te-sdk-v0.44.1

Update to use cosmos-sdk v0.44.1. An important consideration in this update is that the ibc module has been split off into it's own repo at https://github.com/cosmos/ibc-go.

In the compilation of the ibc-go protos, @jkilpatr discovered that the service compilation (in the `compile_*_services()` functions) recompiles and overwrites the regular definitions (in the respective `compile_*_protos()` functions), resulting in missing definitions. This is a result of combining all .proto definitions into a single file, e.g. ibc-go/proto/ibc/applications/transfer/v1 has genesis.proto, query.proto, transfer.proto, tx.proto. After running 2 stage compilation for the protos and then the services, the `GenesisState` struct from `ibc-go/proto/ibc/applications/transfer/v1/genesis.proto` was missing in `ibc.applications.transfer.v1.rs`. Cancelling the service compilation (aka the 2nd stage) resulted in the `GenesisState` struct appearing but all grpc client methods from the other files were not being generated. Applying the service compilation method (using `tonic_build`) to **all** ibc-go proto files results in everything we want compiled.

This 1 stage compilation approach was applied to the updated cosmos-sdk protos and several missing definitions were discovered (see the latest commit for which ones precisely).

NOTE: wasmd and cosmrs have not been touched and require updates 

NOTE: the last commit in this PR is a quick fix to resolve a namespace conflict introduced by prost's generation of an enum and a struct both named Validators in `cosmos-sdk-proto/src/prost/cosmos.staking.v1beta1.rs`'s `stake_authorization` mod. This is not a sustainable solution, but it does allow the module to compile. I renamed the enum to `IsStakeAuthorizationValidators` and the struct to `StakeAuthorizationValidators` to most closely match the generated go names [here](https://github.com/cosmos/cosmos-sdk/blob/8a73b266f52e08f24738c93ef519b529cd35bbd8/x/staking/types/authz.pb.go#L112-L123) and [here](https://github.com/cosmos/cosmos-sdk/blob/8a73b266f52e08f24738c93ef519b529cd35bbd8/x/staking/types/authz.pb.go#L171-L174)
@jkilpatr
Copy link
Contributor

Bumping this pr as #144 should unblock!

Release notes:

https://github.com/informalsystems/tendermint-rs/blob/master/CHANGELOG.md#v0230

This also includes updates to the `prost` v0.9 ecosystem, as well as
bumping `tonic` to v0.6.
@tony-iqlusion
Copy link
Member

Yep, sorry it's taken so long but if it isn't that much work to get it into the next release, I think this would be great to have

- Removes the former `Msg` newtype for `Any`; uses the `Any` type
  explicitly. The `Msg` newtype was providing little value and obscuring
  access to the `type_url`.
- Renames `from_msg` and `to_msg` methods to `from_any` and `to_any`.
- Renames the `MsgType` trait to `Msg`. This trait now has an associated
  `Proto` type bounded on `MsgProto`. This alows for default impls of
  `from_any`/`to_any`/`into_any` methods which replace the previous
  blanket impl of `MsgType` for `MsgProto`.

Altogether these changes simplify the API and eliminate some redundant
boilerplate impling the former `MsgType` trait.
Adds support for this key type when used as the `public_key` field of
`SignerInfo`.

To accomodate this, a new `SignerPublicKey` enum was added with variants
for a single `PublicKey` or a `LegacyAminoMultisig` key.

Decodes a test vector from `cosmoshub-4` (sdk@v0.42.10)
- Changes `edition` attribute to 2021 in Cargo.toml
- Adds a `rust-version` attribute set to 1.56
- Uses the new `include_str!` feature to import README.md as the
  toplevel rustdoc documentation for the crate
@shravanshetty1
Copy link
Author

was away for last few days - will try to get this done soon

@tony-iqlusion
Copy link
Member

If need be we can make another breaking release to get this in.

It would be nice to fix #151 but it requires a breaking change as well.

@shravanshetty1
Copy link
Author

@tony-iqlusion

I pulled the latest changes and rebuilt the proto files from current main branch and I am getting extra code that should not exist.

To generate the files, I navigated to the proto-build and ran the command cargo run.
Ideally no files should have been modified since the commit should have the latest changes but for me 4 files had changed.

➜  cosmos-sdk-proto git:(main) ✗ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   src/prost/cosmos.bank.v1beta1.rs
        modified:   src/prost/cosmos.base.store.v1beta1.rs
        modified:   src/prost/cosmos.evidence.v1beta1.rs
        modified:   src/prost/cosmos.staking.v1beta1.rs

I ran cargo check in cosmos-sdk-proto to check if the generated files are valid and they were not compiling.

➜  cosmos-sdk-proto git:(main) ✗ cargo check
    Checking cosmos-sdk-proto v0.8.0 (/home/anon/projects/cosmos/cosmos-rust/cosmos-sdk-proto)
error[E0428]: the name `Validators` is defined multiple times
   --> cosmos-sdk-proto/src/prost/cosmos.staking.v1beta1.rs:550:5
    |
544 |     pub struct Validators {
    |     --------------------- previous definition of the type `Validators` here
...
550 |     pub enum Validators {
    |     ^^^^^^^^^^^^^^^^^^^ `Validators` redefined here
    |
    = note: `Validators` must be defined only once in the type namespace of this module

error[E0119]: conflicting implementations of trait `std::clone::Clone` for type `cosmos::staking::v1beta1::stake_authorization::Validators`
   --> cosmos-sdk-proto/src/prost/cosmos.staking.v1beta1.rs:549:14
    |
543 |     #[derive(Clone, PartialEq, ::prost::Message)]
    |              ----- first implementation here
...
549 |     #[derive(Clone, PartialEq, ::prost::Oneof)]
    |              ^^^^^ conflicting implementation for `cosmos::staking::v1beta1::stake_authorization::Validators`
    |
    = note: this error originates in the derive macro `Clone` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0119]: conflicting implementations of trait `std::cmp::PartialEq` for type `cosmos::staking::v1beta1::stake_authorization::Validators`
   --> cosmos-sdk-proto/src/prost/cosmos.staking.v1beta1.rs:549:21
    |
543 |     #[derive(Clone, PartialEq, ::prost::Message)]
    |                     --------- first implementation here
...
549 |     #[derive(Clone, PartialEq, ::prost::Oneof)]
    |                     ^^^^^^^^^ conflicting implementation for `cosmos::staking::v1beta1::stake_authorization::Validators`
    |
    = note: this error originates in the derive macro `PartialEq` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0119]: conflicting implementations of trait `std::marker::StructuralPartialEq` for type `cosmos::staking::v1beta1::stake_authorization::Validators`
   --> cosmos-sdk-proto/src/prost/cosmos.staking.v1beta1.rs:549:21
    |
543 |     #[derive(Clone, PartialEq, ::prost::Message)]
    |                     --------- first implementation here
...
549 |     #[derive(Clone, PartialEq, ::prost::Oneof)]
    |                     ^^^^^^^^^ conflicting implementation for `cosmos::staking::v1beta1::stake_authorization::Validators`
    |
    = note: this error originates in the derive macro `PartialEq` (in Nightly builds, run with -Z macro-backtrace for more info)

error[E0119]: conflicting implementations of trait `std::fmt::Debug` for type `cosmos::staking::v1beta1::stake_authorization::Validators`
   --> cosmos-sdk-proto/src/prost/cosmos.staking.v1beta1.rs:549:32
    |
543 |     #[derive(Clone, PartialEq, ::prost::Message)]
    |                                ---------------- first implementation here
...
549 |     #[derive(Clone, PartialEq, ::prost::Oneof)]
    |                                ^^^^^^^^^^^^^^ conflicting implementation for `cosmos::staking::v1beta1::stake_authorization::Validators`
    |
    = note: this error originates in the derive macro `::prost::Oneof` (in Nightly builds, run with -Z macro-backtrace for more info)

Some errors have detailed explanations: E0119, E0428.
For more information about an error, try `rustc --explain E0119`.
error: could not compile `cosmos-sdk-proto` due to 5 previous errors

Have you faced this issue before, and if you have how did you deal with it?

@tony-iqlusion
Copy link
Member

@shravanshetty1 see #154. It's a naming conflict in the generated protos.

For now the only solution is hand-editing the generated code 😢

@shravanshetty1
Copy link
Author

Ok finished and tested it seems to work in grpc-web and normal grpc. Made new PR due to ugly commit history on this one - #157

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants