Add canary annotation and alternative backends for traffic shaping

[clandry94]

What this PR does / why we need it:

Adds support for multi-service routing per ingress by means of alternative backends. It also implements an interface to create these alternative backends through the form of canary annotations.

Some terms

AlternativeBackend - An alternative backend is a normal ingress-nginx backend with the condition that it is not backed through the typical 1:1 backend-server relationship. Instead, alternative backends share a server with another real backend.

Primary Backend - This is the same as a backend. I prefix with the word primary to have a clearer distinction between it and a alternative backend.

How it works

Canary

As described in the linked issue above, the following annotation

nginx.ingress.kubernetes.io/canary: true

Combined with a combination of the below annotations, we can define the behaviour of the canary.

The canary will define a canary-weight. If only weight is specified, then n% of traffic will be shifted to the canary backend. Additionally, if canary-by-cookie or canary-by-header are set, they will shift all traffic which match the cookie/header set with a value of always to the canary backend. If no value is set for the header, it will follow the rules of canary-weight.

nginx.ingress.kubernetes.io/canary-by-cookie: <cookie name>
nginx.ingress.kubernetes.io/canary-by-header: <header name>
nginx.ingress.kubernetes.io/canary-weight: <0 to 100>

After the spec is applied, the ingress in question has it's backends marked as alternative so that the merging code will know what to do with them.

AlternativeBackends

When ingresses are synced, the list of backends and servers is built up. After these lists are finished being built in getBackendServers(), but before they are sorted and returned to the sync function, we check if the current ingress has a canary enabled. If so, we pass a reference of the ingress, annotations, upstreams, and servers to the mergeVirtualBackends function. It works as such:

1. For each path that matches a alternative backend, find the server that matches its path and is not alternative.
2. If found, embed a alternative backend into the primary backend. If no matching path is found, then delete the alternative upstream as it is invalid and has no server to back it.

When alternative backends are matched to a primary backend, they will have access to metadata that describes the primary backend as well as rules to shape traffic. These rules come from the canary ingress configuration.

Which issue this PR fixes

This PR is based on this proposal https://docs.google.com/document/d/1qKTyLBLuKIYE6d6BsFXRM7zYB-2MUk6qJjtBL1KCz78/edit#heading=h.gv8zkanxal65 by @ElvinEfendi and from #2560

Closes #2419

Special notes for your reviewer:

[ElvinEfendi]

@clandry94 can you include some documentation showing an example usage and also noting the existing limitations (i.e single alternative backend support so far)

Also standard annotation documentation at https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/

[ElvinEfendi]

Why don't you retrieve the server directly using

server := servers[rule.Host]

?

[ElvinEfendi]

If balancer is nil we should short-circuit and return here. This is the case when all endpoints are unavailable or there's no endpoint configured.

Currently as you can see in the e2e tests, you will get an error like

018/11/02 07:56:39 [error] 228#228: *112 lua entry thread aborted: runtime error: /etc/nginx/lua/balancer.lua:126: attempt to index local 'balancer' (a nil value)
stack traceback:
coroutine 0:
	/etc/nginx/lua/balancer.lua: in function 'route_to_alternative_balancer'
	/etc/nginx/lua/balancer.lua:167: in function 'get_balancer'
	/etc/nginx/lua/balancer.lua:184: in function 'rewrite'
	rewrite_by_lua(nginx.conf:422):2: in function <rewrite_by_lua(nginx.conf:422):1>, client: 10.244.2.1, server: unavailable.svc.com, request: "GET / HTTP/1.1", host: "unavailable.svc.com"
2018/11/02 07:56:39 [error] 228#228: *112 failed to run log_by_lua*: /etc/nginx/lua/balancer.lua:126: attempt to index local 'balancer' (a nil value)
stack traceback:
	/etc/nginx/lua/balancer.lua:126: in function 'route_to_alternative_balancer'
	/etc/nginx/lua/balancer.lua:167: in function 'get_balancer'
	/etc/nginx/lua/balancer.lua:212: in function 'log'

[clandry94]

Fixed 👍

[clandry94]

/assign @aledbf

[ElvinEfendi]

I assume this change is from your testing.

[aledbf]

@clandry94 you can use FOCUS="Annotations\s-\scanary" make e2e-test to avoid commenting the code.

[ElvinEfendi]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: clandry94, ElvinEfendi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ElvinEfendi]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment