Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: protoreflect based amino json encoder #14877

Merged
merged 146 commits into from
Feb 23, 2023
Merged

Conversation

kocubinski
Copy link
Member

@kocubinski kocubinski commented Feb 1, 2023

Description

Closes: #10993
Follows: #15019

This patch implements an Amino JSON encoder for proto.Message types using the protoreflect API. Therefore, protoreflect capable protoc generated types ("pulsar" types) can be serialized to bytes identical to an equivalent gogo generated type serialized through the legacy amino codec in go-amino.

For example, given the IDL specification for auth.ModuleAccount, the gogo type serialized through go-amino and the pulsar type serialized through the encoder in this patch produce the same bytes.


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

Copy link
Contributor

@amaury1093 amaury1093 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I'll approve once the last custom message encoder for LegacyAminoPubKey is added.

return jsonMarshal(w, "0")
}
return jsonMarshal(w, val)
default:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as Dec, I think cosmos.Int could be bytes too: https://github.com/cosmos/cosmos-sdk/blob/main/proto/cosmos/staking/v1beta1/genesis.proto#L18.

In practice I doubt that message will ever be marshalled with Amino, but I would still add it for completeness.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the only instance in of types.Int stored as bytes in a proto message. I'll make a test case to support it though.

x/tx/aminojson/json_marshal.go Outdated Show resolved Hide resolved
x/tx/aminojson/json_marshal.go Show resolved Hide resolved
Copy link
Contributor

@amaury1093 amaury1093 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -0,0 +1 @@
testdata
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem right

type FieldEncoder func(AminoJSON, protoreflect.Value, io.Writer) error

// AminoJSON is a JSON encoder that uses the Amino JSON encoding rules.
type AminoJSON struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
type AminoJSON struct {
type Encoder struct {

}

// NewAminoJSON returns a new AminoJSON encoder.
func NewAminoJSON() AminoJSON {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func NewAminoJSON() AminoJSON {
func NewEncoder() Encoder {

}

// MarshalAmino serializes a protobuf message to JSON.
func (aj AminoJSON) MarshalAmino(message proto.Message) ([]byte, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func (aj AminoJSON) MarshalAmino(message proto.Message) ([]byte, error) {
func (aj AminoJSON) Marshal(message proto.Message) ([]byte, error) {

Comment on lines 164 to 168
aj := Encoder{}.DefineMessageEncoding("key_field", keyFieldEncoder)
err = aj.marshalList(pubkeys, w)
if err != nil {
return err
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-looking at this PR, i'm wondering if it's worth changing the message encoder func signature to:

- type MessageEncoder func(protoreflect.Message, io.Writer) error
+ type MessageEncoder func(Encoder, protoreflect.Message, io.Writer) error

this way we don't need to create a new instance of Encoder{} here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be *Encoder in that case I think

Copy link
Member Author

@kocubinski kocubinski Feb 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with either. thresholdStringEncoder needs an Encoder instance to handle the Any types in the message.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be *Encoder in that case I think

Why do you say so?

Copy link
Member

@aaronc aaronc Feb 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Efficiency, so there's less copying

@kocubinski kocubinski enabled auto-merge (squash) February 23, 2023 21:21
@kocubinski kocubinski merged commit 976ecd4 into main Feb 23, 2023
@kocubinski kocubinski deleted the kocubinski/amino-json branch February 23, 2023 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement protoreflect based amino JSON encoder
4 participants