Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebase the grpc branch to master branch #771

Merged
merged 28 commits into from
Feb 21, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
888e18e
[Issue #417] Grpc Transport Protocol support (#710)
jinrongluo Jan 12, 2022
5ad7d95
[Issue #417] Create getting started instructions for Grpc transport p…
jinrongluo Jan 12, 2022
7b7716d
[Issue #417] update Grpc Message Model name to SimpleMessage
jinrongluo Jan 18, 2022
da6b4fc
[Issue #417] more update Grpc Message Model name to SimpleMessage
jinrongluo Jan 18, 2022
c0f4ce9
[Issue #718] Fix readme file and protobuf file based on review comments
jinrongluo Jan 25, 2022
21aaa89
[Issue #745] fix the ack bugs and cloudevent message resolver
jinrongluo Jan 25, 2022
7c090a6
[Issue #744] update SDK API message model
jinrongluo Jan 27, 2022
519f379
[Issue #744] fix the gRPC Consumer SubscribeStream Message handler
jinrongluo Jan 27, 2022
4e7fe96
[Issue #744] Grpc Request-Reply API support
jinrongluo Jan 28, 2022
8ffb34a
[Issue #744] Bug fix for Grpc Request-Reply API support
jinrongluo Jan 29, 2022
e950126
[Issue #744] minor fix for Grpc request-Reply API
jinrongluo Jan 31, 2022
cd50721
[Issue #744] fix infinte message loop in Grpc CloudEvent request-Repl…
jinrongluo Jan 31, 2022
5156d06
[Issue #744] Fix Grpc subscribe-unsubscribe bug
jinrongluo Feb 1, 2022
e2e859d
[Issue #744] Fix Data models in Grpc Request-Reply API
jinrongluo Feb 2, 2022
5ad4db4
[Issue #744] Code optimization for Grpc Request-Reply API
jinrongluo Feb 2, 2022
651850f
[Issue #417] support Grpc broadcast async publish
jinrongluo Feb 3, 2022
c22bd1e
[Issue #718] add synchronized calls for grpc streamObserver
jinrongluo Feb 9, 2022
d6e17a3
supply apache header
xwm1992 Feb 16, 2022
0cd3709
add checkstyle ignore for grpc
xwm1992 Feb 17, 2022
3681ef7
Merge branch 'master' into master
xwm1992 Feb 17, 2022
fd608d3
Merge branch 'master' of https://github.com/apache/incubator-eventmes…
xwm1992 Feb 17, 2022
4c9efc6
fix checkstyle error
xwm1992 Feb 17, 2022
f254b6b
Merge branch 'master' of https://github.com/xwm1992/EventMesh into mi…
xwm1992 Feb 17, 2022
51b80bc
fix javax.annotation.generated compile error
xwm1992 Feb 18, 2022
cd7c332
fix javax.annotation.generated compile error
xwm1992 Feb 21, 2022
b1345f2
supply dependencies licenses
xwm1992 Feb 21, 2022
b766cd0
update known-dependencies.txt
xwm1992 Feb 21, 2022
c139a9e
update known-dependencies.txt
xwm1992 Feb 21, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ allprojects {
configFile = new File("${rootDir}/style/checkStyle.xml")
}

checkstyleMain.exclude '**/org/apache/eventmesh/client/grpc/protos**'

dependencies {
testImplementation "junit:junit"
}
Expand Down Expand Up @@ -464,6 +466,13 @@ subprojects {
dependency "io.cloudevents:cloudevents-core:2.2.0"
dependency "io.cloudevents:cloudevents-json-jackson:2.2.0"

dependency "io.grpc:grpc-protobuf:1.15.0"
dependency "io.grpc:grpc-stub:1.15.0"
dependency "io.grpc:grpc-netty:1.15.0"
dependency "io.grpc:grpc-netty-shaded:1.15.0"

dependency "javax.annotation:javax.annotation-api:1.3.2"

dependency "com.github.seancfoley:ipaddress:5.3.3"
}
}
Expand Down
161 changes: 161 additions & 0 deletions docs/cn/instructions/eventmesh-runtime-protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,3 +258,164 @@ public class LiteMessage {
| 场景 | Server向Client发送消息请求码 | Client回复Server消息响应码 | 说明 |
| ------------------ | ---------------------------- | -------------------------- | ---------------------- |
| 客户端接收异步事件 | HTTP_PUSH_CLIENT_ASYNC(105) | retCode | retCode值为0时代表成功 |


## gRPC 协议文档

#### 1. protobuf

在 `eventmesh-protocol-gprc` 模块有 Eventmesh gRPC 客户端的 protobuf 文件. the protobuf 文件路径是 `/src/main/proto/eventmesh-client.proto`.

用gradle build 生成 gRPC 代码在 `/build/generated/source/proto/main`. 生成代码用于 `eventmesh-sdk-java` 模块.

#### 2. gRPC 数据模型

- 消息

以下消息数据模型用于 `publish()`, `requestReply()` 和 `broadcast()` APIs.

```
message RequestHeader {
string env = 1;
string region = 2;
string idc = 3;
string ip = 4;
string pid = 5;
string sys = 6;
string username = 7;
string password = 8;
string language = 9;
string protocolType = 10;
string protocolVersion = 11;
string protocolDesc = 12;
}

message SimpleMessage {
RequestHeader header = 1;
string producerGroup = 2;
string topic = 3;
string content = 4;
string ttl = 5;
string uniqueId = 6;
string seqNum = 7;
string tag = 8;
map<string, string> properties = 9;
}

message BatchMessage {
RequestHeader header = 1;
string producerGroup = 2;
string topic = 3;

message MessageItem {
string content = 1;
string ttl = 2;
string uniqueId = 3;
string seqNum = 4;
string tag = 5;
map<string, string> properties = 6;
}

repeated MessageItem messageItem = 4;
}

message Response {
string respCode = 1;
string respMsg = 2;
string respTime = 3;
}
```

- 订阅

以下订阅数据模型用于 `subscribe()` 和 `unsubscribe()` APIs.

```
message Subscription {
RequestHeader header = 1;
string consumerGroup = 2;

message SubscriptionItem {
enum SubscriptionMode {
CLUSTERING = 0;
BROADCASTING = 1;
}

enum SubscriptionType {
ASYNC = 0;
SYNC = 1;
}

string topic = 1;
SubscriptionMode mode = 2;
SubscriptionType type = 3;
}

repeated SubscriptionItem subscriptionItems = 3;
string url = 4;
}
```

- 心跳

以下心跳数据模型用于 `heartbeat()` API.

```
message Heartbeat {
enum ClientType {
PUB = 0;
SUB = 1;
}

RequestHeader header = 1;
ClientType clientType = 2;
string producerGroup = 3;
string consumerGroup = 4;

message HeartbeatItem {
string topic = 1;
string url = 2;
}

repeated HeartbeatItem heartbeatItems = 5;
}
```

#### 3. gRPC 服务接口

- 事件生产端服务 APIs

```
service PublisherService {
# 异步事件生产
rpc publish(SimpleMessage) returns (Response);

# 同步事件生产
rpc requestReply(SimpleMessage) returns (Response);

# 批量事件生产
rpc batchPublish(BatchMessage) returns (Response);
}
```

- 事件消费端服务 APIs

```
service ConsumerService {
# 所消费事件通过 HTTP Webhook推送事件
rpc subscribe(Subscription) returns (Response);

# 所消费事件通过 TCP stream推送事件
rpc subscribeStream(Subscription) returns (stream SimpleMessage);

rpc unsubscribe(Subscription) returns (Response);
}
```

- 客户端心跳服务 API

```
service HeartbeatService {
rpc heartbeat(Heartbeat) returns (Response);
}
```
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ sudo vi eventmesh.properties
|----------------------------|-------|----------------------------|
| eventMesh.server.http.port | 10105 | EventMesh http server port |
| eventMesh.server.tcp.port | 10000 | EventMesh tcp server port |

| eventMesh.server.grpc.port | 10205 | EventMesh grpc server port |


### 配置 rocketmq-client.properties
Expand Down
1 change: 1 addition & 0 deletions docs/cn/instructions/eventmesh-runtime-quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,7 @@ eventMesh.connector.plugin.type=rocketmq
| ---------------------- | ------ | ----------------------- |
| eventMesh.server.http.port | 10105 | EventMesh http 服务端口 |
| eventMesh.server.tcp.port | 10000 | EventMesh tcp 服务端口 |
| eventMesh.server.grpc.port | 10205 | EventMesh grpc 服务端口 |

**rocketmq-client.properties**

Expand Down
53 changes: 51 additions & 2 deletions docs/cn/instructions/eventmesh-sdk-java-quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@
>
> EventMesh-sdk-java支持异步消息和广播消息。异步消息表示生产者只发送消息,不关心回复消息。广播消息表示生产者发送一次消息,所有订阅广播主题的消费者都将收到消息
>
> EventMesh-sdk-java支持HTTP和TCP协议
> EventMesh-sdk-java支持HTTP,TCP 和 GRPC 协议

TCP 和 HTTP 示例都在**eventmesh-example**模块下
TCP, HTTP 和 GRPC 示例都在**eventmesh-example**模块下

### 1. TCP DEMO

Expand Down Expand Up @@ -70,5 +70,54 @@ TCP 和 HTTP 示例都在**eventmesh-example**模块下
运行org.apache.eventmesh.http.demo.pub.eventmeshmessage.AsyncPublishInstance的主要方法
```

### 3. GRPC 演示

> eventmesh-sdk-java 实现了 gRPC 协议. 它能异步和同步发送事件到 eventmesh-runtime.
> 它可以通过webhook和事件流方式订阅消费事件, 同时也支持 CNCF CloudEvents 协议.

<h4> 异步事件发送 和 webhook订阅 </h4>

> Async生产者 异步发送事件到 eventmesh-runtime, 不需要等待事件储存到 `event-store`
> 在webhook 消费者, 事件推送到消费者的http endpoint url。这个URL在消费者的 `Subscription` 模型定于. 这方法跟前面的Http eventmsh client类似。

- 在rocketmq 创建主题 TEST-TOPIC-GRPC-ASYNC
- 启动 publisher 发送事件

```
运行 org.apache.eventmesh.grpc.pub.eventmeshmessage.AsyncPublishInstance 的主要方法
```

- 启动 webhook 消费者

```
运行 org.apache.eventmesh.grpc.sub.app.SpringBootDemoApplication 的主要方法
```

<h4> 同步事件发送和事件流订阅 </h4>

> 同步生产者 发送事件到 eventmesh-runtime, 同时等待事件储存到 `event-store`
> 在事件流消费者,事件以流的形式推送到 `ReceiveMsgHook` 客户端。 这方法类似 eventmesh client.

- 在rocketmq 创建主题 TEST-TOPIC-GRPC-RR
- 启动 Request-Reply publisher 发送事件

```
运行 org.apache.eventmesh.grpc.pub.eventmeshmessage.RequestReplyInstance 的主要方法
```

- 启动 stream subscriber

```
运行 org.apache.eventmesh.grpc.sub.EventmeshAsyncSubscribe 的主要方法
```

<h4> 批量事件发布 </h4>

> 批量发布多个事件到 eventmesh-runtime. 这是异步操作

- 在rocketmq 创建主题 TEST-TOPIC-GRPC-ASYNC
- 启动 publisher 来批量发布事件

```
运行 org.apache.eventmesh.grpc.pub.eventmeshmessage.BatchPublishInstance 的主要方法
```
Loading