Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update nginx template for TLS passthrough #2166

Merged
Show file tree
Hide file tree
Changes from 41 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
67fd5fc
Update nginx template for TLS passthrough
sarthyparty Jun 20, 2024
0ff40ac
add https server support and manifests made
sarthyparty Jun 21, 2024
7fd449e
add hash directives to stream block
sarthyparty Jun 21, 2024
ffce60f
Update comment in generator.go
sarthyparty Jun 21, 2024
a491af2
remove pointer for map slice and fix tests
sarthyparty Jun 21, 2024
621b552
rewrite string logic for creating Map
sarthyparty Jun 21, 2024
46b8838
refactor loop variables to be named server
sarthyparty Jun 21, 2024
0cdcd78
add helper function to create unix socket name
sarthyparty Jun 21, 2024
d11c72f
add preallocated space for slices
sarthyparty Jun 21, 2024
24ad258
remove local testing setup
sarthyparty Jun 24, 2024
31ae435
switch map from bool to struct
sarthyparty Jun 24, 2024
5085790
refactor tests
sarthyparty Jun 24, 2024
a221edd
Update gotemplate name
sarthyparty Jun 24, 2024
fcf2588
swap star character for :s
sarthyparty Jun 24, 2024
efcdc5d
refactor Server type definition
sarthyparty Jun 24, 2024
a3fe0c4
rename types
sarthyparty Jun 24, 2024
d2ee81c
add stream upstream functions
sarthyparty Jun 25, 2024
8238431
add https socket
sarthyparty Jun 26, 2024
ec359e5
fix tests
sarthyparty Jun 26, 2024
b42508c
add socket name test
sarthyparty Jun 26, 2024
ef01005
remove local testing setup
sarthyparty Jun 26, 2024
9c1babe
update generator test
sarthyparty Jun 26, 2024
4a0b72b
reset log to original
sarthyparty Jun 26, 2024
de308a5
move args to separate line
sarthyparty Jun 27, 2024
f109872
add test for stream conf content
sarthyparty Jun 27, 2024
73f9b08
refactor create stream maps to use map instead of slice
sarthyparty Jun 27, 2024
b331e6f
rename variable name gen func
sarthyparty Jun 27, 2024
c092f6a
add short circuit in createStreamMaps
sarthyparty Jun 27, 2024
b8f92d9
update map and base tests and pipeline
sarthyparty Jun 27, 2024
17bc73a
fix a bunch of nits
sarthyparty Jun 28, 2024
0fe630a
add connection closing server
sarthyparty Jul 1, 2024
63efd48
update invalid upstream name to remove socket
sarthyparty Jul 1, 2024
64a0cca
readd close connection server
sarthyparty Jul 1, 2024
bca9a5b
fix lint checks and manifests
sarthyparty Jul 2, 2024
0315e77
fix tests and update plus config
sarthyparty Jul 2, 2024
449c59c
revert gateway api tests version
sarthyparty Jul 2, 2024
0eb7092
update go sum
sarthyparty Jul 2, 2024
c28a566
update tests for stream maps and servers
sarthyparty Jul 2, 2024
bb7e505
add stream upstream tests, remove invalid upstream ref
sarthyparty Jul 2, 2024
e90646f
fix a code nit
sarthyparty Jul 2, 2024
83c0565
update server and map tests
sarthyparty Jul 2, 2024
2135010
rename tlsservers
sarthyparty Jul 3, 2024
82bf8c4
remove extra spaces in conf files
sarthyparty Jul 3, 2024
d023f28
remove escape slash and add stream folder to slice
sarthyparty Jul 5, 2024
fb857b0
update socket name creation and map template
sarthyparty Jul 5, 2024
b98951e
update upstream zone size and test
sarthyparty Jul 5, 2024
3f5ee8b
update comment for zone size upstream
sarthyparty Jul 8, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions charts/nginx-gateway-fabric/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,8 @@ spec:
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: nginx-stream-conf
sarthyparty marked this conversation as resolved.
Show resolved Hide resolved
mountPath: /etc/nginx/stream-conf.d
- name: module-includes
mountPath: /etc/nginx/module-includes
- name: nginx-secrets
Expand Down Expand Up @@ -153,6 +155,8 @@ spec:
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: nginx-stream-conf
mountPath: /etc/nginx/stream-conf.d
- name: module-includes
mountPath: /etc/nginx/module-includes
- name: nginx-secrets
Expand Down Expand Up @@ -187,6 +191,8 @@ spec:
volumes:
- name: nginx-conf
emptyDir: {}
- name: nginx-stream-conf
emptyDir: {}
- name: module-includes
emptyDir: {}
- name: nginx-secrets
Expand Down
6 changes: 6 additions & 0 deletions config/tests/static-deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ spec:
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: nginx-stream-conf
mountPath: /etc/nginx/stream-conf.d
- name: module-includes
mountPath: /etc/nginx/module-includes
- name: nginx-secrets
Expand Down Expand Up @@ -98,6 +100,8 @@ spec:
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: nginx-stream-conf
mountPath: /etc/nginx/stream-conf.d
- name: module-includes
mountPath: /etc/nginx/module-includes
- name: nginx-secrets
Expand All @@ -117,6 +121,8 @@ spec:
volumes:
- name: nginx-conf
emptyDir: {}
- name: nginx-stream-conf
emptyDir: {}
- name: module-includes
emptyDir: {}
- name: nginx-secrets
Expand Down
6 changes: 6 additions & 0 deletions deploy/manifests/nginx-gateway-experimental.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,8 @@ spec:
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: nginx-stream-conf
mountPath: /etc/nginx/stream-conf.d
- name: module-includes
mountPath: /etc/nginx/module-includes
- name: nginx-secrets
Expand Down Expand Up @@ -254,6 +256,8 @@ spec:
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: nginx-stream-conf
mountPath: /etc/nginx/stream-conf.d
- name: module-includes
mountPath: /etc/nginx/module-includes
- name: nginx-secrets
Expand All @@ -273,6 +277,8 @@ spec:
volumes:
- name: nginx-conf
emptyDir: {}
- name: nginx-stream-conf
emptyDir: {}
- name: module-includes
emptyDir: {}
- name: nginx-secrets
Expand Down
6 changes: 6 additions & 0 deletions deploy/manifests/nginx-gateway.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,8 @@ spec:
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: nginx-stream-conf
mountPath: /etc/nginx/stream-conf.d
- name: module-includes
mountPath: /etc/nginx/module-includes
- name: nginx-secrets
Expand Down Expand Up @@ -250,6 +252,8 @@ spec:
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: nginx-stream-conf
mountPath: /etc/nginx/stream-conf.d
- name: module-includes
mountPath: /etc/nginx/module-includes
- name: nginx-secrets
Expand All @@ -269,6 +273,8 @@ spec:
volumes:
- name: nginx-conf
emptyDir: {}
- name: nginx-stream-conf
emptyDir: {}
- name: module-includes
emptyDir: {}
- name: nginx-secrets
Expand Down
6 changes: 6 additions & 0 deletions deploy/manifests/nginx-plus-gateway-experimental.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,8 @@ spec:
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: nginx-stream-conf
mountPath: /etc/nginx/stream-conf.d
- name: module-includes
mountPath: /etc/nginx/module-includes
- name: nginx-secrets
Expand Down Expand Up @@ -261,6 +263,8 @@ spec:
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: nginx-stream-conf
mountPath: /etc/nginx/stream-conf.d
- name: module-includes
mountPath: /etc/nginx/module-includes
- name: nginx-secrets
Expand All @@ -280,6 +284,8 @@ spec:
volumes:
- name: nginx-conf
emptyDir: {}
- name: nginx-stream-conf
emptyDir: {}
- name: module-includes
emptyDir: {}
- name: nginx-secrets
Expand Down
6 changes: 6 additions & 0 deletions deploy/manifests/nginx-plus-gateway.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,8 @@ spec:
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: nginx-stream-conf
mountPath: /etc/nginx/stream-conf.d
- name: module-includes
mountPath: /etc/nginx/module-includes
- name: nginx-secrets
Expand Down Expand Up @@ -257,6 +259,8 @@ spec:
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: nginx-stream-conf
mountPath: /etc/nginx/stream-conf.d
- name: module-includes
mountPath: /etc/nginx/module-includes
- name: nginx-secrets
Expand All @@ -276,6 +280,8 @@ spec:
volumes:
- name: nginx-conf
emptyDir: {}
- name: nginx-stream-conf
emptyDir: {}
- name: module-includes
emptyDir: {}
- name: nginx-secrets
Expand Down
15 changes: 15 additions & 0 deletions internal/mode/static/nginx/conf/nginx-plus.conf
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,21 @@ http {
}
}

stream {
variables_hash_bucket_size 512;
variables_hash_max_size 1024;

map_hash_max_size 2048;
map_hash_bucket_size 256;

log_format stream-main '$remote_addr [$time_local] '
'$protocol $status $bytes_sent $bytes_received '
'$session_time "$ssl_preread_server_name"';
access_log /dev/stdout stream-main;
sarthyparty marked this conversation as resolved.
Show resolved Hide resolved

include /etc/nginx/stream-conf.d/*.conf;
}

mgmt {
usage_report interval=0s;
}
15 changes: 15 additions & 0 deletions internal/mode/static/nginx/conf/nginx.conf
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,18 @@ http {
}
}
}

stream {
variables_hash_bucket_size 512;
variables_hash_max_size 1024;

map_hash_max_size 2048;
map_hash_bucket_size 256;

log_format stream-main '$remote_addr [$time_local] '
'$protocol $status $bytes_sent $bytes_received '
'$session_time "$ssl_preread_server_name"';
access_log /dev/stdout stream-main;
sarthyparty marked this conversation as resolved.
Show resolved Hide resolved

include /etc/nginx/stream-conf.d/*.conf;
kate-osborn marked this conversation as resolved.
Show resolved Hide resolved
}
16 changes: 16 additions & 0 deletions internal/mode/static/nginx/config/base_http_config_template.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,20 @@ package config

const baseHTTPTemplateText = `
{{- if .HTTP2 }}http2 on;{{ end }}

# Set $gw_api_compliant_host variable to the value of $http_host unless $http_host is empty, then set it to the value
# of $host. We prefer $http_host because it contains the original value of the host header, which is required by the
# Gateway API. However, in an HTTP/1.0 request, it's possible that $http_host can be empty. In this case, we will use
# the value of $host. See http://nginx.org/en/docs/http/ngx_http_core_module.html#var_host.
map $http_host $gw_api_compliant_host {
'' $host;
default $http_host;
}

# Set $connection_header variable to upgrade when the $http_upgrade header is set, otherwise, set it to close. This
# allows support for websocket connections. See https://nginx.org/en/docs/http/websocket.html.
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
`
2 changes: 2 additions & 0 deletions internal/mode/static/nginx/config/base_http_config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,5 +47,7 @@ func TestExecuteBaseHttp(t *testing.T) {
res := executeBaseHTTPConfig(test.conf)
g.Expect(res).To(HaveLen(1))
g.Expect(test.expCount).To(Equal(strings.Count(string(res[0].data), expSubStr)))
g.Expect(strings.Count(string(res[0].data), "map $http_host $gw_api_compliant_host {")).To(Equal(1))
g.Expect(strings.Count(string(res[0].data), "map $http_upgrade $connection_upgrade {")).To(Equal(1))
}
}
9 changes: 9 additions & 0 deletions internal/mode/static/nginx/config/generator.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ const (
// httpFolder is the folder where NGINX HTTP configuration files are stored.
httpFolder = configFolder + "/conf.d"

// streamFolder is the folder where NGINX Stream configuration files are stored.
streamFolder = configFolder + "/stream-conf.d"
pleshakov marked this conversation as resolved.
Show resolved Hide resolved

// modulesIncludesFolder is the folder where the included "load_module" file is stored.
modulesIncludesFolder = configFolder + "/module-includes"

Expand All @@ -29,6 +32,9 @@ const (
// httpConfigFile is the path to the configuration file with HTTP configuration.
httpConfigFile = httpFolder + "/http.conf"

// streamConfigFile is the path to the configuration file with Stream configuration.
streamConfigFile = streamFolder + "/stream.conf"

// configVersionFile is the path to the config version configuration file.
configVersionFile = httpFolder + "/config-version.conf"

Expand Down Expand Up @@ -157,6 +163,9 @@ func (g GeneratorImpl) getExecuteFuncs() []executeFunc {
executeSplitClients,
executeMaps,
executeTelemetry,
executeStreamServers,
g.executeStreamUpstreams,
executeStreamMaps,
}
}

Expand Down
25 changes: 23 additions & 2 deletions internal/mode/static/nginx/config/generator_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,25 @@ func TestGenerate(t *testing.T) {
Port: 443,
},
},
TLSPassthroughServers: []dataplane.Layer4VirtualServer{
{
Hostname: "app.example.com",
Port: 443,
UpstreamName: "stream_up",
},
},
Upstreams: []dataplane.Upstream{
{
Name: "up",
Endpoints: nil,
},
},
StreamUpstreams: []dataplane.Upstream{
{
Name: "stream_up",
Endpoints: nil,
},
},
BackendGroups: []dataplane.BackendGroup{bg},
SSLKeyPairs: map[dataplane.SSLKeyPairID]dataplane.SSLKeyPair{
"test-keypair": {
Expand Down Expand Up @@ -81,7 +94,7 @@ func TestGenerate(t *testing.T) {

files := generator.Generate(conf)

g.Expect(files).To(HaveLen(6))
g.Expect(files).To(HaveLen(7))
pleshakov marked this conversation as resolved.
Show resolved Hide resolved
arrange := func(i, j int) bool {
return files[i].Path < files[j].Path
}
Expand All @@ -98,7 +111,7 @@ func TestGenerate(t *testing.T) {
// Note: this only verifies that Generate() returns a byte array with upstream, server, and split_client blocks.
// It does not test the correctness of those blocks. That functionality is covered by other tests in this package.
g.Expect(httpCfg).To(ContainSubstring("listen 80"))
g.Expect(httpCfg).To(ContainSubstring("listen 443"))
g.Expect(httpCfg).To(ContainSubstring("listen unix:/var/run/nginx/https443.sock"))
g.Expect(httpCfg).To(ContainSubstring("upstream"))
g.Expect(httpCfg).To(ContainSubstring("split_clients"))

Expand Down Expand Up @@ -127,4 +140,12 @@ func TestGenerate(t *testing.T) {
Path: "/etc/nginx/secrets/test-keypair.pem",
Content: []byte("test-cert\ntest-key"),
}))

g.Expect(files[6].Path).To(Equal("/etc/nginx/stream-conf.d/stream.conf"))
g.Expect(files[6].Type).To(Equal(file.TypeRegular))
streamCfg := string(files[6].Content)
g.Expect(streamCfg).To(ContainSubstring("listen unix:/var/run/nginx/app.example.com443.sock"))
g.Expect(streamCfg).To(ContainSubstring("listen 443"))
g.Expect(streamCfg).To(ContainSubstring("app.example.com unix:/var/run/nginx/app.example.com443.sock"))
g.Expect(streamCfg).To(ContainSubstring("example.com unix:/var/run/nginx/https443.sock"))
}
15 changes: 1 addition & 14 deletions internal/mode/static/nginx/config/http/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ package http
type Server struct {
SSL *SSL
ServerName string
Listen string
Locations []Location
Includes []string
Port int32
IsDefaultHTTP bool
IsDefaultSSL bool
GRPC bool
Expand Down Expand Up @@ -88,19 +88,6 @@ type SplitClientDistribution struct {
Value string
}

// Map defines an NGINX map.
type Map struct {
Source string
Variable string
Parameters []MapParameter
}

// Parameter defines a Value and Result pair in a Map.
type MapParameter struct {
Value string
Result string
}

// ProxySSLVerify holds the proxied HTTPS server verification configuration.
type ProxySSLVerify struct {
TrustedCertificate string
Expand Down
Loading
Loading