Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(*) enable IPv6 support #1734

Merged
merged 49 commits into from
Apr 8, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
5a0ae01
chore(kuma-dp) sed debug for envoy
Mar 24, 2021
950a625
chore(*) support IPv6
Mar 25, 2021
8c0450f
chore(*) ipv6 for K8s
Mar 29, 2021
8f5a74e
fix(*) check and test
Mar 29, 2021
7ed8502
test(circleci) enable IPv6 in e2e
Mar 29, 2021
5620f8f
Merge remote-tracking branch 'origin/master' into chore/ipv6
Mar 29, 2021
e21e329
fix(circleci) improve naming
Mar 29, 2021
501f36a
chore(*) bump go
Mar 29, 2021
2b25f0b
chore(*) bump minikube
Mar 29, 2021
3886f3a
chore(*) bump CircleCI image
Mar 29, 2021
147a259
fix(*) small fixes
Mar 30, 2021
ac73849
chore(*) transparent proxy for IPv4 and IPv6
Mar 30, 2021
0dae915
fix(*) make check
Mar 30, 2021
058ddec
fix(*) minukube
Mar 30, 2021
f24af19
test(e2e) universal image will have ip tool
Mar 30, 2021
ac28a4b
chore(*) iptables for IPv4/IPv6 mixed hosts
Mar 30, 2021
803a026
chore(*) introduce redirectPortInboundV6
Mar 30, 2021
1fdc9fa
chore(*) strict dns for IPv6
Mar 30, 2021
195fc7e
test(e2e) K8s privileged containers for iptables debugging
Mar 30, 2021
48ca5bf
chore(cni) bump version to 0.0.4
Mar 31, 2021
6313a5e
test(*) unit tests
Apr 2, 2021
25eda3e
Merge remote-tracking branch 'origin/master' into chore/ipv6
Apr 2, 2021
3233c0a
test(*) ipv6 unit tests
Apr 5, 2021
9e32c9c
fix(*) cleanups
Apr 5, 2021
9430eb4
Merge remote-tracking branch 'origin/master' into chore/ipv6
Apr 5, 2021
37196c0
chore(envoy) pass log level
Apr 6, 2021
58cd99b
fix(*) review
Apr 6, 2021
f2f04fa
test(*) failing envoy test
Apr 6, 2021
c8b37d0
chore(*) remove ipv4compat
Apr 6, 2021
ed959ef
Merge remote-tracking branch 'origin/master' into chore/ipv6
Apr 6, 2021
c2626b0
fix(*) circleci for IPv6 on master
Apr 6, 2021
305583a
test(e2e) increase timeout for IPv6
Apr 6, 2021
9c3cdab
test(*) load images fix
Apr 6, 2021
fbcbea5
test(e2e) split deploy and hybrid
nickolaev Apr 6, 2021
c719882
test(e2e) optimize GetEnv usage
nickolaev Apr 6, 2021
ed05725
chore(*) more CNI and probes for testing
Apr 7, 2021
0b97b6b
fix(*) review
Apr 7, 2021
64b9541
chore(*) IP generate for Envoy GRPC
Apr 7, 2021
968fcf8
test(e2e) fix nodeport logic
Apr 7, 2021
ccc89d3
fix(*) failing golden file test
Apr 8, 2021
6a3b39b
test(e2e) print pod name in healthcheck test
Apr 8, 2021
596b9e9
Merge remote-tracking branch 'origin/master' into chore/ipv6
Apr 8, 2021
73cc0f3
fix(*) go.sum
Apr 8, 2021
ea8eb67
test(ci) run IPv6 first
Apr 8, 2021
dd13143
test(e2e) be more precise when getting the IPv6 address
Apr 8, 2021
51caadc
test(ci) bump image to ubuntu-2004:202101-01
Apr 8, 2021
32b7a08
test(circleci) fix minikube and disable ipv6 in commit workflow
Apr 8, 2021
3e736bb
Merge remote-tracking branch 'origin/master' into chore/ipv6
Apr 8, 2021
31de8ec
fix(ci) minikube
Apr 8, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 49 additions & 15 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,14 @@ commands:
reusable:

constants:
- &go_version "1.15.6"
- &go_version "1.15.11"

docker_images:
- &golang_image "golang:1.15.6"
- &circleci_golang_image "circleci/golang:1.15.6"
- &golang_image "golang:1.15.11"
- &circleci_golang_image "circleci/golang:1.15.11"

vm_images:
- &ubuntu_vm_image "ubuntu-1604:202007-01"
- &ubuntu_vm_image "ubuntu-2004:202101-01"

snippets:

Expand Down Expand Up @@ -568,6 +568,10 @@ jobs:
description: XDS API version
type: string
default: v3
ipv6:
description: use IPv6
type: boolean
default: false
parallelism: 2
environment:
GOPATH: /home/circleci/.go-kuma-go
Expand Down Expand Up @@ -603,13 +607,29 @@ jobs:
command: |
export PATH=$HOME/go/bin:$PATH
helm repo add kuma https://kumahq.github.io/charts
- run:
name: "Run E2E tests"
command: |
export PATH=$HOME/go/bin:$PATH
export E2E_PKG_LIST=$(circleci tests glob ./test/e2e/* | circleci tests split | xargs printf "./%s/... ")
export API_VERSION=<< parameters.api >>
make test/e2e
- when:
condition: << parameters.ipv6 >>
steps:
- run:
name: "Run IPv6 E2E tests"
command: |
export PATH=$HOME/go/bin:$PATH
export E2E_PKG_LIST=$(circleci tests glob ./test/e2e/* | circleci tests split | xargs printf "./%s/... ")
export API_VERSION=<< parameters.api >>
export IPV6=true
export KUMA_DEFAULT_RETRIES=60
export KUMA_DEFAULT_TIMEOUT="6s"
make test/e2e
- unless:
condition: << parameters.ipv6 >>
steps:
- run:
name: "Run IPv4 E2E tests"
command: |
export PATH=$HOME/go/bin:$PATH
export E2E_PKG_LIST=$(circleci tests glob ./test/e2e/* | circleci tests split | xargs printf "./%s/... ")
export API_VERSION=<< parameters.api >>
make test/e2e
build:
executor: golang
Expand Down Expand Up @@ -695,7 +715,7 @@ jobs:
kubernetes_version:
description: The version of Kubernetes to test on.
type: string
default: "v1.15.0"
default: "v1.20.0"
use_local_kuma_images:
description: Use local Docker images from CircleCI workspace instead of downloading ones from bintray.io.
type: boolean
Expand All @@ -720,7 +740,7 @@ jobs:
sudo apt-get install -y conntrack
- run:
name: Start Minikube
command: sudo CHANGE_MINIKUBE_NONE_USER=true $HOME/bin/minikube start --vm-driver=none --kubernetes-version=<< parameters.kubernetes_version >>
command: $HOME/bin/minikube start --vm-driver=docker --kubernetes-version=<< parameters.kubernetes_version >>
- when:
condition: << parameters.use_local_kuma_images >>
steps:
Expand Down Expand Up @@ -887,10 +907,16 @@ workflows:
use_local_kuma_images: true
- e2e:
<<: *commit_workflow_filters
name: test/e2e
name: test/e2e-ipv4
requires:
- images
- check
# keep this one disabled and enable only for development
# - e2e:
# <<: *commit_workflow_filters
# name: test/e2e-ipv6
# # custom parameters
# ipv6: true

clean-eks:
triggers:
Expand Down Expand Up @@ -1018,13 +1044,21 @@ workflows:
- images
- check
- e2e:
<<: *commit_workflow_filters
<<: *master_workflow_filters
name: test/e2e V2
requires:
- images
- check
# custom parameters
api: v2
- e2e:
<<: *master_workflow_filters
name: test/e2e-ipv6
requires:
- images
- check
# custom parameters
ipv6: true

kuma-release:
jobs:
Expand Down
54 changes: 34 additions & 20 deletions api/mesh/v1alpha1/dataplane.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions api/mesh/v1alpha1/dataplane.proto
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,11 @@ message Dataplane {

// List of services that will be access directly via IP:PORT
repeated string direct_access_services = 3;

// Port on which all IPv6 inbound traffic is being transparently
// redirected.
uint32 redirect_port_inbound_v6 = 4
[ (validate.rules).uint32 = {lte : 65535} ];
}

// Gateway describes configuration of gateway of the dataplane.
Expand Down
5 changes: 4 additions & 1 deletion api/mesh/v1alpha1/dataplane_helpers.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,10 @@ package v1alpha1

import (
"fmt"
"net"
"reflect"
"sort"
"strconv"
"strings"

"github.com/pkg/errors"
Expand Down Expand Up @@ -69,7 +71,8 @@ type OutboundInterface struct {
}

func (i OutboundInterface) String() string {
return fmt.Sprintf("%s:%d", i.DataplaneIP, i.DataplanePort)
return net.JoinHostPort(i.DataplaneIP,
strconv.FormatUint(uint64(i.DataplanePort), 10))
}

func (n *Dataplane_Networking) GetOutboundInterfaces() ([]OutboundInterface, error) {
Expand Down
17 changes: 17 additions & 0 deletions api/mesh/v1alpha1/dataplane_helpers_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,23 @@ var _ = Describe("Dataplane_Networking", func() {
{DataplaneIP: "192.168.0.1", DataplanePort: 443},
},
}),
Entry("2 outbound interfaces IPv6", testCase{
input: &Dataplane_Networking{
Outbound: []*Dataplane_Networking_Outbound{
{
Port: 8080,
},
{
Address: "fd00::1",
Port: 443,
},
},
},
expected: []OutboundInterface{
{DataplaneIP: "127.0.0.1", DataplanePort: 8080},
{DataplaneIP: "fd00::1", DataplanePort: 443},
},
}),
)
})
})
Expand Down
3 changes: 3 additions & 0 deletions app/kuma-dp/cmd/context.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ import (
"net/http"
"time"

"github.com/kumahq/kuma/pkg/log"

"github.com/kumahq/kuma/app/kuma-dp/pkg/dataplane/envoy"
kumadp "github.com/kumahq/kuma/pkg/config/app/kuma-dp"
"github.com/kumahq/kuma/pkg/core/runtime/component"
Expand All @@ -17,6 +19,7 @@ type RootContext struct {
BootstrapGenerator envoy.BootstrapConfigFactoryFunc
BootstrapDynamicMetadata map[string]string
Config *kumadp.Config
LogLevel log.LogLevel
}

func DefaultRootContext() *RootContext {
Expand Down
1 change: 1 addition & 0 deletions app/kuma-dp/cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ func NewRootCmd(rootCtx *RootContext) *cobra.Command {
return err
}
core.SetLogger(core.NewLogger(level))
rootCtx.LogLevel = level

// once command line flags have been parsed,
// avoid printing usage instructions
Expand Down
1 change: 1 addition & 0 deletions app/kuma-dp/cmd/run.go
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ func newRunCmd(rootCtx *RootContext) *cobra.Command {
Stdout: cmd.OutOrStdout(),
Stderr: cmd.OutOrStderr(),
Quit: shouldQuit,
LogLevel: rootCtx.LogLevel,
})
if err != nil {
return err
Expand Down
3 changes: 3 additions & 0 deletions app/kuma-dp/pkg/dataplane/envoy/envoy.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"time"

"github.com/kumahq/kuma/pkg/core/resources/model/rest"
pkg_log "github.com/kumahq/kuma/pkg/log"
"github.com/kumahq/kuma/pkg/xds/bootstrap/types"

"github.com/pkg/errors"
Expand Down Expand Up @@ -42,6 +43,7 @@ type Opts struct {
Stdout io.Writer
Stderr io.Writer
Quit chan struct{}
LogLevel pkg_log.LogLevel
}

func New(opts Opts) (*Envoy, error) {
Expand Down Expand Up @@ -154,6 +156,7 @@ func (e *Envoy) Start(stop <-chan struct{}) error {
// and we don't expect users to do "hot restart" manually.
// so, let's turn it off to simplify getting started experience.
"--disable-hot-restart",
"-l ", e.opts.LogLevel.String(),
}
if version != "" { // version is always send by Kuma CP, but we check empty for backwards compatibility reasons (new Kuma DP connects to old Kuma CP)
args = append(args, "--bootstrap-version", string(version))
Expand Down
2 changes: 1 addition & 1 deletion app/kuma-dp/pkg/dataplane/envoy/envoy_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ var _ = Describe("Envoy", func() {
// then
Expect(err).ToNot(HaveOccurred())
// and
Expect(strings.TrimSpace(buf.String())).To(Equal(fmt.Sprintf("-c %s --drain-time-s 15 --disable-hot-restart --bootstrap-version 2", expectedConfigFile)))
Expect(strings.TrimSpace(buf.String())).To(Equal(fmt.Sprintf("-c %s --drain-time-s 15 --disable-hot-restart -l off --bootstrap-version 2", expectedConfigFile)))

By("verifying the contents Envoy config file")
// when
Expand Down
3 changes: 3 additions & 0 deletions app/kumactl/cmd/completion/testdata/bash.golden
Original file line number Diff line number Diff line change
Expand Up @@ -2399,6 +2399,9 @@ _kumactl_install_transparent-proxy()
flags+=("--redirect-inbound-port=")
two_word_flags+=("--redirect-inbound-port")
local_nonpersistent_flags+=("--redirect-inbound-port=")
flags+=("--redirect-inbound-port-v6=")
two_word_flags+=("--redirect-inbound-port-v6")
local_nonpersistent_flags+=("--redirect-inbound-port-v6=")
flags+=("--redirect-outbound-port=")
two_word_flags+=("--redirect-outbound-port")
local_nonpersistent_flags+=("--redirect-outbound-port=")
Expand Down
1 change: 1 addition & 0 deletions app/kumactl/cmd/completion/testdata/zsh.golden
Original file line number Diff line number Diff line change
Expand Up @@ -976,6 +976,7 @@ function _kumactl_install_transparent-proxy {
'--modify-iptables[modify the host iptables to redirect the traffic to Envoy]' \
'--redirect-inbound[redirect the inbound traffic to the Envoy. Should be disabled for Gateway data plane proxies.]' \
'--redirect-inbound-port[inbound port redirected to Envoy, as specified in dataplane'\''s `networking.transparentProxying.redirectPortInbound`]:' \
'--redirect-inbound-port-v6[IPv6 inbound port redirected to Envoy, as specified in dataplane'\''s `networking.transparentProxying.redirectPortInboundV6`]:' \
'--redirect-outbound-port[outbound port redirected to Envoy, as specified in dataplane'\''s `networking.transparentProxying.redirectPortOutbound`]:' \
'--skip-resolv-conf[skip modifying the host `/etc/resolv.conf`]' \
'--store-firewalld[store the iptables changes with firewalld]' \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ func DefaultInstallCpContext() InstallCpContext {
Cni_conf_name: "kuma-cni.conf",
Cni_image_registry: "docker.io",
Cni_image_repository: "lobkovilya/install-cni",
Cni_image_tag: "0.0.2",
Cni_image_tag: "0.0.4",
ControlPlane_mode: core.Standalone,
ControlPlane_zone: "",
ControlPlane_globalRemoteSyncService_type: "LoadBalancer",
Expand Down
3 changes: 2 additions & 1 deletion app/kumactl/cmd/install/install_dns.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"context"
"encoding/json"
"fmt"
"net"
"regexp"
"strings"

Expand Down Expand Up @@ -66,7 +67,7 @@ This command requires that the KUBECONFIG environment is set`,
return err
}

kumaDNSAddress := fmt.Sprintf("%s:%s", kumaCPSVC.Spec.ClusterIP, args.Port)
kumaDNSAddress := net.JoinHostPort(kumaCPSVC.Spec.ClusterIP, args.Port)

var errs error
generated := false
Expand Down
Loading