ADR 021 - Protocol Buffer Query Support

docs/architecture/README.md

@@ -46,3 +46,4 @@ Please add a entry below in your Pull Request for an ADR.
 - [ADR 018: Extendable Voting Periods](./adr-018-extendable-voting-period.md)
 - [ADR 019: Protocol Buffer State Encoding](./adr-019-protobuf-state-encoding.md)
 - [ADR 020: Protocol Buffer Transaction Encoding](./adr-020-protobuf-transaction-encoding.md)
+- [ADR 021: Protocol Buffer Query Encoding](./adr-021-protobuf-query-encoding.md)

docs/architecture/adr-021-protobuf-query-encoding.md

@@ -0,0 +1,258 @@
+# ADR 021: Protocol Buffer Query Encoding
+
+## Changelog
+
+- 2020 March 27: Initial Draft
+
+## Status
+
+Proposed
+
+## Context
+
+This ADR is a continuation of the motivation, design, and context established in
+[ADR 019](./adr-019-protobuf-state-encoding.md) and
+[ARD 020](./adr-019-protobuf-transaction-encoding.md), namely, we aim to design the
+Protocol Buffer migration path for the client-side of the Cosmos SDK.
+
+This ADR continues from [ARD 020](./adr-020-protobuf-transaction-encoding.md)
+to specify the encoding of queries.
+
+## Decision
+
+### Custom Query Definition
+
+Modules define custom queries through a protocol buffers `service` definition.
+These `service` definitions are generally associated with and used by the
+GRPC protocol. However, the protocol buffers specification indicates that
+they can be used more generically by any request/response protocol that uses
+protocol buffer encoding. Thus, we can use `service` definitions for specifying
+custom ABCI queries and even reuse a substantial amount of the GRPC infrastructure.
+
+Each module with custom queries should define a service canonically named `Query`:
+
+```proto
+// x/bank/types/types.proto
+
+service Query {
+  rpc QueryBalance(QueryBalanceParams) returns (cosmos_sdk.v1.Coin) { }
+  rpc QueryAllBalances(QueryAllBalancesParams) returns (QueryAllBalancesResponse) { }
+}
+```
+
+#### Handling of Interface Types
+
+Modules that use interface types and need true polymorphism generally force a
+`oneof` up to the app-level that provides the set of concrete implementations of
+that interface that the app supports. While app's are welcome to do the same for
+queries and implement an app-level query service, it is recommended that modules
+provide query methods that expose these interfaces via `google.protobuf.Any`.
+There is a concern on the transaction level that the overhead of `Any` is too
+high to justify its usage. However for queries this is not a concern, and
+providing generic module-level queries that use `Any` does not preclude apps
+from also providing app-level queries that return use the app-level `oneof`s.
+
+
+A hypothetical example for the `gov` module would look something like:
+
+```proto
+// x/gov/types/types.proto
+
+import "google/protobuf/any.proto";
+
+service Query {
+  rpc GetProposal(GetProposalParams) returns (AnyProposal) { }
+}
+
+message AnyProposal {
+  ProposalBase base = 1;
+  google.protobuf.Any content = 2;
+}
+```
+
+### Custom Query Implementation
+
+In order to implement the query service, we can reuse the existing [gogo protobuf](https://github.com/gogo/protobuf)
+grpc plugin, which for a service named `Query` generates an interface named
+`QueryServer` as below:
+
+```go
+type QueryServer interface {
+	QueryBalance(context.Context, *QueryBalanceParams) (*types.Coin, error)
+	QueryAllBalances(context.Context, *QueryAllBalancesParams) (*QueryAllBalancesResponse, error)
+}
+```
+
+The custom queries for our module are implemented by implementing this interface.
+
+The first parameter in this generated interface is a generic `context.Context`,
+whereas querier methods generally need an instance of `sdk.Context` to read
+from the store. Since arbitrary values can be attached to `context.Context`
+using the `WithValue` and `Value` methods, the SDK should provide a function
+`sdk.UnwrapSDKContext` to retrieve the `sdk.Context` from the provided
+`context.Context`.
+
+An example implementation of `QueryBalance` for the bank module as above would
+look something like:
+
+```go
+type Querier struct {
+	Keeper
+}
+
+func (q Querier) QueryBalance(ctx context.Context, params *types.QueryBalanceParams) (*sdk.Coin, error) {
+	balance := q.GetBalance(sdk.UnwrapSDKContext(ctx), params.Address, params.Denom)
+	return &balance, nil
+}
+```
+
+### Custom Query Registration and Routing
+
+Query server implementations as above would be registered with `AppModule`s using
+a new method `RegisterQueryServer(grpc.Server)` which could be implemented simply
+as below:
+
+```go
+// x/bank/module.go
+func (am AppModule) RegisterQueryServer(server grpc.Server) {
+	types.RegisterQueryServer(server, keeper.Querier{am.keeper})
+}
+```
+
+Underneath the hood, a new method `RegisterService(sd *grpc.ServiceDesc, handler interface{})`
+will be added to the existing `baseapp.QueryRouter` to add the queries to the custom
+query routing table (with the routing method being described below).
+The signature for this method matches the existing
+`RegisterServer` method on the GRPC `Server` type where `handler` is the custom
+query server implementation described above.
+
+
+GRPC-like requests are routed by the service name (ex. `cosmos_sdk.x.bank.v1.Query`)
+and method name (ex. `QueryBalance`) combined with `/`s to form a full
+method name (ex. `/cosmos_sdk.x.bank.v1.Query/QueryBalance`). This gets translated
+into an ABCI query as `custom/cosmos_sdk.x.bank.v1.Query/QueryBalance`. Service handlers
+registered with `QueryRouter.RegisterService` will be routed this way.
+
+Beyond the method name, GRPC requests carry a protobuf encoded payload, which maps naturally
+to `RequestQuery.Data`, and receive a protobuf encoded response or error. Thus
+there is a quite natural mapping of GRPC-like rpc methods to the existing
+`sdk.Query` and `QueryRouter` infrastructure.
+
+This basic specification allows us to reuse protocol buffer `service` definitions
+for ABCI custom queries substantially reducing the need for manual decoding and
+encoding in query methods. 
+
+### GRPC Protocol Support
+
+In addition to providing an ABCI query pathway, we can easily provide a GRPC
+proxy server that routes requests in the GRPC protocol to ABCI query requests
+under the hood. In this way, clients could use their host languages' existing
+GRPC implementations to make direct queries against Cosmos SDK app's using
+these `service` definitions. In order for this server to work, the `QueryRouter`
+on `BaseApp` will need to expose the service handlers registered with
+`QueryRouter.RegisterService` to the proxy server implementation. Nodes could
+launch the proxy server on a separate port in the same process as the ABCI app
+with a command-line flag.
+
+### REST Queries and Swagger Generation
+
+[grpc-gateway](https://github.com/grpc-ecosystem/grpc-gateway) is a project that
+translates REST calls into GRPC calls using special annotations on service
+methods. Modules that want to expose REST queries should add `google.api.http`
+annotations to their `rpc` methods as in this example below.
+
+```proto
+// x/bank/types/types.proto
+
+service Query {
+  rpc QueryBalance(QueryBalanceParams) returns (cosmos_sdk.v1.Coin) {
+    option (google.api.http) = {
+      get: "/x/bank/v1/balance/{address}/{denom}"
+    };
+  }
+  rpc QueryAllBalances(QueryAllBalancesParams) returns (QueryAllBalancesResponse) {
+    option (google.api.http) = {
+      get: "/x/bank/v1/balances/{address}"
+    };
+  }
+}
+```
+
+grpc-gateway will work direcly against the GRPC proxy described above which will 
+translate requests to ABCI queries under the hood. grpc-gateway can also
+generate Swagger definitions automatically.
+
+In the current implementation of REST queries, each module needs to implement
+REST queries manually in addition to ABCI querier methods. Using the grpc-gateway
+approach, there will be no need to generate separate REST query handlers, just
+query servers as described above as grpc-gateway handles the translation of protobuf
+to REST as well as Swagger definitions.
+
+The SDK should provide CLI commands for apps to start GRPC gateway either in
+a separate process or the same process as the ABCI app, as well as provide a
+command for generating grpc-gateway proxy `.proto` files and the `swagger.json`
+file.
+
+### Client Usage
+
+The gogo protobuf grpc plugin generates client interfaces in addition to server
+interfaces. For the `Query` service defined above we would get a `QueryClient`
+interface like:
+
+```go
+type QueryClient interface {
+	QueryBalance(ctx context.Context, in *QueryBalanceParams, opts ...grpc.CallOption) (*types.Coin, error)
+	QueryAllBalances(ctx context.Context, in *QueryAllBalancesParams, opts ...grpc.CallOption) (*QueryAllBalancesResponse, error)
+}
+```
+
+Via a small patch to gogo protobuf ([gogo/protobuf#675](https://github.com/gogo/protobuf/pull/675))
+we have tweaked the grpc codegen to use an interface rather than concrete type
+for the generated client struct. This allows us to also reuse the GRPC infrastructure
+for ABCI client queries.
+
+`CLIContext` will receive a new method `QueryConn` that returns a `ClientConn`
+that routes calls to ABCI queries
+
+Clients (such as CLI methods) will then be able to call query methods like this:
+
+```go
+cliCtx := context.NewCLIContext()
+queryClient := types.NewQueryClient(cliCtx.QueryConn())
+params := &types.QueryBalanceParams{addr, denom}
+result, err := queryClient.QueryBalance(gocontext.Background(), params)
+```
+
+### Testing
+
+Tests would be able to create a query client directly from keeper and `sdk.Context`
+references using a `QueryServerTestHelper` as below:
+
+```go
+queryHelper := baseapp.NewQueryServerTestHelper(ctx)
+types.RegisterQueryServer(queryHelper, keeper.Querier{app.BankKeeper})
+queryClient := types.NewQueryClient(queryHelper)
+```
+
+## Future Improvements
+
+## Consequences
+
+### Positive
+
+* greatly simplified querier implementation (no manual encoding/decoding)
+* easy query client generation (can use existing grpc and swagger tools)
+* no need for REST query implementations
+* type safe query methods (generated via grpc plugin)
+* going forward, there will be less breakage of query methods because of the
+backwards compatibility guarantees provided by buf
+
+### Negative
+
+* all clients using the existing ABCI/REST queries will need to be refactored
+for both the new GRPC/REST query paths as well as protobuf/proto-json encoded
+data, but this is more or less unavoidable in the protobuf refactoring
+
+### Neutral
+
+## References