-
Notifications
You must be signed in to change notification settings - Fork 9.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Protobuf: cleanup both golang/protobuf and gogo/protobuf #14533
Comments
This seems like a big change, please let me know if anyone has any comments or concerns. |
@ahrtr - Has it been decided to make the change? If so, can I take a swing at this? |
@ajityagaty Thanks for your interest on this. There is no any feedback from other maintainers yet, but I think we must do this because both Since it's a big change, so please evaluate the impact and provide a detailed plan firstly. Afterwards, we can update the functional_test in the first step. |
any updates on this? @ajityagaty |
@ahrtr - Apologies! I have not been able to look at this yet as I have been overwhelmed at work. When I briefly looked at the work that is required, I realized that I am lacking context of how all components are dependent on this change. So, I think I would be better off taking a smaller piece of work. |
Thanks for the feedback. No worries. |
Unfortunately, I do not get time to work on this so far, so unassigned to me for now. Notes:
|
@ahrtr I can look at this problem.
I have started an initial analysis in this document. I will update it as I add findings. |
I'm trying to upgrade the grpc-gateway version right now. Still working on the task right now and will file pr in this week. Thanks Updated: NOTE: The grpc-gateway change is related to pb update. |
Thanks @fuweid for the update. Please feel free to let other contributors know if you need any help. |
Based on findings/discussions in #17892 the conclusion was basically that we need to migrate away from these deprecated packages to google.golang.org/protobuf to be able to bump certain dependencies and be able to make use of standards they provide. At the same time there might not be bandwidth available to work on this. To kick things of at least, I took some time looking into how etcd are using gogo/protobuf as a first start to get a better understanding what's involved making this change, what it might affect and as a way to get some discussions going. First I did a search to see if others have documented their migration and found https://thoughts.8-p.info/en/containerd-gogo-migration which includes some interesting details and links to PRs of their work. gogo/protobuf usage investigation:Extensionshttps://github.com/gogo/protobuf/blob/master/extensions.md option (gogoproto.marshaler_all) = true;Generates Marshal and MarshalTo method for each message Usage of Marshal:contrib/raftexample/raft.go:169 Usage of MarshalTo:server/etcdserver/api/rafthttp/msgappv2_codec.go option (gogoproto.unmarshaler_all) = true;The unmarshal plugin generates a Unmarshal method for each message. Usage of Unmarshal:contrib/raftexample/raft.go option (gogoproto.sizer_all) = true;generates a Size or ProtoSize method for each message. This is useful with the MarshalTo method generated by the marshalto plugin and the gogoproto.marshaler and gogoproto.marshaler_all extensions Usage of Size:/Users/marcus/go/src/github.com/marefr/etcd/server/etcdserver/bootstrap.go option (gogoproto.goproto_getters_all) = false;if false, the message is generated without get methods, this is useful when you would rather want to use face Should behave as standard protobuf generation based on my knowledge. option (gogoproto.goproto_enum_prefix_all) = false;if false, generates the enum constant names without the messagetype prefix Should behave as standard protobuf generation based on my knowledge. gogoproto.nullableif false, a field is generated without a pointer (see warning below). Migrating from gogo/protobuf Marshal, MarshalTo, Unmarshal, Size:By switching to google.golang.org/protobuf you don't get automatic generation of these methods in generated code for messages (Go structs). Seems like server/etcdserver/api/v3rpc/grpc.go uses a custom codec, server/etcdserver/api/v3rpc/codec.go, which The google.golang.org/protobuf/proto package provides Marshal, Unmarshal and Size functions, however The pbutil package mentioned above seems like it's used quite a lot so should be quite straightforward change that to use proto package instead for Marshal/Unmarshal. Any direct thoughts/insights whether it's plausible switching to google.golang.org/protobuf/proto package and use reflection during runtime for Marshal/Unmarshal and/or what affect it might have given that server/etcdserver/api/v3rpc/grpc.go doesn't seem to use it for request/response encode/decoding? One idea I had was as a first step try migrating to use github.com/google/protobuf/proto where possible and see what breaks/performance degrades. Thoughts? |
What would you like to be added?
Both
github.com/golang/protobuf
andgh.neting.cc/gogo/protobuf
are deprecated.github.com/golang/protobuf
with extra code generation features. Deprecated.In short, google.golang.org/protobuf is the future, so we should cleanup both
github.com/golang/protobuf
andgh.neting.cc/gogo/protobuf
, and only usegoogle.golang.org/protobuf
.Why is this needed?
Same as above
cc @ptabor @serathius @spzala @dims @liggitt
The text was updated successfully, but these errors were encountered: