Question: Deep Protobuf dependency hierarchy in Go service

[pjthepooh]

I have a Go service with big dependency graph of protobuf. There is only one top-level schema I need to register for the topic, but it depends on many protobuf messages in different files and packages. I couldn't find a Go client that can leverage an instance of a protobuf class to register all its references like Java client does with KafkaProtobufSerializer. I wonder what would be a reasonable approach to register and manage schemas in this case.

[rayokota]

You could try the Schema Registry Maven Plugin: https://docs.confluent.io/platform/current/schema-registry/develop/maven-plugin.html

[pjthepooh]

Thanks @rayokota this seems to be a significant overhead for our use case, but will look into that.

A follow up question I have is how can one register a protobuf schema from external package (without Java client)? e.g. If I have this

import "google/protobuf/wrappers.proto";

message ExampleSchema {
  int64 foo_count = 1;
  google.protobuf.StringValue string_value = 2;
}

I'd get error

ERROR io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Either the input schema or one its references is invalid; error code: 42201

[rayokota]

@pjthepooh , if you use CP 7.0+, the google types do not need to be registered separately, they are built-in.

[rayokota]

See confluentinc/confluent-kafka-go#776