Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support listen http2 with plaintext #3547

Merged
merged 13 commits into from
Feb 19, 2021
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .travis/linux_openresty_common_runner.sh
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,16 @@ script() {
-crt ./t/certs/apisix.crt -key ./t/certs/apisix.key \
&

# listen 9081 for http2 with plaintext
echo '
apisix:
node_listen:
- port: 9080
enable_http2: false
- port: 9081
enable_http2: true
' > conf/config.yaml

./bin/apisix help
./bin/apisix init
./bin/apisix init_etcd
Expand Down
8 changes: 4 additions & 4 deletions apisix/cli/ngx_tpl.lua
Original file line number Diff line number Diff line change
Expand Up @@ -362,8 +362,8 @@ http {
{% end %}

server {
{% for _, port in ipairs(node_listen) do %}
listen {* port *} {% if enable_reuseport then %} reuseport {% end %};
{% for _, item in ipairs(node_listen) do %}
listen {* item.port *} {% if enable_reuseport then %} reuseport {% end %} {% if item.enable_http2 then %} http2 {% end %};
{% end %}
{% if ssl.enable then %}
{% for _, port in ipairs(ssl.listen_port) do %}
Expand All @@ -378,8 +378,8 @@ http {
{% end %}

{% if enable_ipv6 then %}
{% for _, port in ipairs(node_listen) do %}
listen [::]:{* port *} {% if enable_reuseport then %} reuseport {% end %};
{% for _, item in ipairs(node_listen) do %}
listen [::]:{* item.port *} {% if enable_reuseport then %} reuseport {% end %} {% if item.enable_http2 then %} http2 {% end %};
{% end %}
{% if ssl.enable then %}
{% for _, port in ipairs(ssl.listen_port) do %}
Expand Down
14 changes: 12 additions & 2 deletions apisix/cli/ops.lua
Original file line number Diff line number Diff line change
Expand Up @@ -238,9 +238,19 @@ Please modify "admin_key" in conf/config.yaml .
util.die("missing apisix.proxy_cache for plugin proxy-cache\n")
end

--support multiple ports listen, compatible with the original style
-- support multiple ports listen, compatible with the original style
if type(yaml_conf.apisix.node_listen) == "number" then
local node_listen = {yaml_conf.apisix.node_listen}
local node_listen = {{port = yaml_conf.apisix.node_listen}}
yaml_conf.apisix.node_listen = node_listen
elseif type(yaml_conf.apisix.node_listen) == "table" then
local node_listen = {}
for index, value in ipairs(yaml_conf.apisix.node_listen) do
if type(value) == "number" then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the number type really needed? I think the following way is enough.

node_listen:
  - port: 9080
  - port: 9180
    enable_http2: true

Copy link
Member

@spacewander spacewander Feb 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to be backward compatibility unless it is a bugfix.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean the element type inside node_listen, not the node_listen itself.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can also be compatible with this format:

node_listen:
    - 9080
    - 9081

table_insert(node_listen, index, {port = value})
elseif type(value) == "table" then
table_insert(node_listen, index, value)
end
end
yaml_conf.apisix.node_listen = node_listen
end

Expand Down
27 changes: 26 additions & 1 deletion doc/grpc-proxy.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ Here's an example, to proxying gRPC service by specified route:

* attention: the `scheme` of the route's upstream must be `grpc` or `grpcs`.
* attention: APISIX use TLS‑encrypted HTTP/2 to expose gRPC service, so need to [config SSL certificate](https.md)
* attention: APISIX also support to expose gRPC service with plaintext HTTP/2, which do not need to config SSL certificate, usually used to proxy gRPC service in intranet environment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which do not need to config SSL certificate => which doesn't rely on ALPN/NPN extensions in TLS/SSL protocol.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is a little hard to understand for user...? Maybe it's enough to mention in doc that this feature is supported, and show how to configure it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about "which doesn't rely on TLS/SSL"?

Copy link
Contributor Author

@sysulq sysulq Feb 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, done :-)

* the grpc server example:[grpc_server_example](https://github.com/iresty/grpc_server_example)

```shell
Expand All @@ -54,7 +55,7 @@ curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f13
}'
```

#### testing
#### testing HTTP/2 with TLS‑encrypted

Invoking the route created before:

Expand All @@ -67,6 +68,30 @@ $ grpcurl -insecure -import-path /pathtoprotos -proto helloworld.proto -d '{"n

This means that the proxying is working.

#### testing HTTP/2 with plaintext

By default, the APISIX only listens to `9443` for TLS‑encrypted HTTP/2. You can support HTTP/2 with plaintext via the `node_listen` section under `apisix` in `conf/config.yaml`:

```yaml
apisix:
node_listen:
- port: 9080
enable_http2: false
- port: 9081
enable_http2: true
```

Invoking the route created before:

```shell
$ grpcurl -plaintext -import-path /pathtoprotos -proto helloworld.proto -d '{"name":"apisix"}' 127.0.0.1:9081 helloworld.Greeter.SayHello
{
"message": "Hello apisix"
}
```

This means that the proxying is working.

### gRPCS

If your gRPC service encrypts with TLS by itself (so called `gPRCS`, gPRC + TLS), you need to change the `scheme` to `grpcs`. The example above runs gRPCS service on port 50052, to proxy gRPC request, we need to use the configuration below:
Expand Down
28 changes: 27 additions & 1 deletion doc/zh-cn/grpc-proxy.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@

* 注意:这个 Route 对应的 Upstream 的 `scheme` 必须设置为 `grpc` 或者 `grpcs`。
* 注意: APISIX 使用 TLS 加密的 HTTP/2 暴露 gRPC 服务, 所以需要先 [配置 SSL 证书](https.md);
* 注意: APISIX 也支持通过纯文本的 HTTP/2 暴露 gRPC 服务,这不需要配置 SSL 证书,通常用于内网环境代理gRPC服务
* 下面例子所代理的 gRPC 服务可供参考:[grpc_server_example](https://github.com/api7/grpc_server_example)。

```shell
Expand All @@ -53,7 +54,7 @@ curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f13
}'
```

### 测试
### 测试 TLS 加密的 HTTP/2

访问上面配置的 Route:

Expand All @@ -67,6 +68,31 @@ grpcurl -insecure -import-path /pathtoprotos -proto helloworld.proto \

这表示已成功代理。

### 测试纯文本的 HTTP/2

默认情况下,APISIX只在 `9443` 端口支持 TLS 加密的 HTTP/2。你也可以支持纯本文的 HTTP/2,只需要修改 `conf/config.yaml` 文件中的 `node_listen` 配置即可。

```yaml
apisix:
node_listen:
- port: 9080
enable_http2: false
- port: 9081
enable_http2: true
```

访问上面配置的 Route:

```shell
grpcurl -plaintext -import-path /pathtoprotos -proto helloworld.proto \
-d '{"name":"apisix"}' 127.0.0.1:9081 helloworld.Greeter.SayHello
{
"message": "Hello apisix"
}
```

这表示已成功代理。

### gRPCS

如果你的 gRPC 服务使用了自己的 TLS 加密,即所谓的 `gPRCS` (gRPC + TLS),那么需要修改 scheme 为 `grpcs`。继续上面的例子,50052 端口上跑的是 gPRCS 的服务,这时候应该这么配置:
Expand Down
1 change: 1 addition & 0 deletions grpc_server_example
Submodule grpc_server_example added at d93635
50 changes: 50 additions & 0 deletions t/core/config-default.t
Original file line number Diff line number Diff line change
Expand Up @@ -91,3 +91,53 @@ apisix:
GET /t
--- response_body
admin_key: null



=== TEST 5: support listen multiple ports with array
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to add tests in t/grpc-proxy-test.sh.
And need to update the doc/grpc-proxy.md

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

--- yaml_config
apisix:
node_listen:
- 1985
- 1986
--- config
location /t {
content_by_lua_block {
local encode_json = require("toolkit.json").encode
local config = require("apisix.core").config.local_conf()

ngx.say("node_listen: ", encode_json(config.apisix.node_listen))
}
}
--- request
GET /t
--- response_body
node_listen: [1985,1986]
--- no_error_log
[error]



=== TEST 6: support listen multiple ports with array table
--- yaml_config
apisix:
node_listen:
- port: 1985
enable_http2: true
- port: 1986
enable_http2: true
--- config
location /t {
content_by_lua_block {
local encode_json = require("toolkit.json").encode
local config = require("apisix.core").config.local_conf()

ngx.say("node_listen: ", encode_json(config.apisix.node_listen))
}
}
--- request
GET /t
--- response_body
node_listen: [{"enable_http2":true,"port":1985},{"enable_http2":true,"port":1986}]
--- no_error_log
[error]
4 changes: 4 additions & 0 deletions t/grpc-proxy-test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f1
}
}'

# test grpc proxy with plaintext
./build-cache/grpcurl -plaintext -import-path ./build-cache/proto -proto helloworld.proto -d '{"name":"apisix"}' 127.0.0.1:9081 helloworld.Greeter.SayHello | grep 'Hello apisix'

# test grpc proxy with ssl
./build-cache/grpcurl -insecure -import-path ./build-cache/proto -proto helloworld.proto -d '{"name":"apisix"}' test.com:9443 helloworld.Greeter.SayHello | grep 'Hello apisix'

# the old way
Expand Down