Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add schema logic in producer and consumer #368

Merged
merged 19 commits into from
Oct 23, 2020

Conversation

wolfstudy
Copy link
Member

@wolfstudy wolfstudy commented Sep 21, 2020

Signed-off-by: xiaolong.ran rxl@apache.org

Fixes #344

Motivation

Type safety is extremely important in any application built around a message bus like Pulsar. Currently, Apache Pulsar supports the function of Schema Registry. And Java, CPP and Python clients already support schema registry related functions, In order to further improve the function of Go Client, we need to support the function of Schema Registry in Go Client.

Modifications

  • Add schema logic in producer
  • Add schema logic in consumer
  • Package basic tool functions for schema logic

Currente state

  • Encapsulates all basic types of decode and encode logic
  • Encapsulates the use of all schema types
  • A new interface has been added for producer and consumer to be compatible with schema functions

Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
@wolfstudy wolfstudy changed the title [WIP] Add schema logic for Go client Package basic tool functions for schema logic Oct 11, 2020
@wolfstudy wolfstudy self-assigned this Oct 11, 2020
@wolfstudy wolfstudy added this to the 0.3.0 milestone Oct 11, 2020
Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
@wolfstudy
Copy link
Member Author

@merlimat @cckellogg PTAL thanks.

pulsar/schemaDef_test.go Outdated Show resolved Hide resolved
pulsar/schemaDef_test.go Outdated Show resolved Hide resolved
pulsar/schema.go Show resolved Hide resolved
pulsar/primitiveSerDe.go Show resolved Hide resolved
pulsar/schema.go Show resolved Hide resolved
pulsar/message.go Show resolved Hide resolved
Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
@wolfstudy wolfstudy changed the title Package basic tool functions for schema logic Add schema logic in producer and consumer Oct 14, 2020
Copy link
Contributor

@cckellogg cckellogg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this PR should be split into 2. First add the structs/interfaces for the schema and then add a follow up PR to implement the schema in the client. This will keep the PRs more self contained and easier to review.

pulsar/schemaDef_test.go Outdated Show resolved Hide resolved
Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
Signed-off-by: xiaolong.ran <rxl@apache.org>
@wolfstudy wolfstudy requested a review from sijie October 16, 2020 02:02
Signed-off-by: xiaolong.ran <rxl@apache.org>
@skyrocknroll
Copy link

skyrocknroll commented Oct 21, 2020

@wolfstudy is there any specific reason why we are using v1.0.5 of goavro ? According to README "github.com/linkedin/goavro/v2" has better support for namespaces and performance improvements. Can we please switch to v2 ?

@wolfstudy
Copy link
Member Author

wolfstudy commented Oct 22, 2020

@wolfstudy is there any specific reason why we are using v1.0.5 of goavro ? According to README "github.com/linkedin/goavro/v2" has better support for namespaces and performance improvements. Can we please switch to v2 ?

Cool, I think we can update the goavro to v2, this looks good to me

Signed-off-by: xiaolong.ran <rxl@apache.org>
@wolfstudy
Copy link
Member Author

@skyrocknroll done PTAL again, thanks.

ping @sijie @massakam If time is convenient, please review it, thanks.

Copy link
Member

@jiazhai jiazhai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@wolfstudy wolfstudy merged commit 45c522d into apache:master Oct 23, 2020
@ZiyaoWei ZiyaoWei mentioned this pull request Mar 7, 2022
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support Schema Registry feature in Go Client
4 participants