set maximum number of connections to 60000

[breuerfelix]

Signed-off-by: Felix Breuer fbreuer@pm.me

[nightlyone]

Looks like a very good idea to manage it more explicitly.

[dergeberl]

We do also limits in the createEnvoyCluster() function in internal/helper/yawollet.go:

	Thresholds: []*envoycluster.CircuitBreakers_Thresholds{
		{
			Priority:           envoycore.RoutingPriority_DEFAULT,
			MaxConnections:     &wrappers.UInt32Value{Value: 10000 * uint32(hostmetrics.GetCPUNum())},
			MaxRequests:        &wrappers.UInt32Value{Value: 8000 * uint32(hostmetrics.GetCPUNum())},
			MaxPendingRequests: &wrappers.UInt32Value{Value: 2000 * uint32(hostmetrics.GetCPUNum())},
		},
	},

This is set per upstream Port which even do not make sense. If I have 10 Ports the sum of all MaxConnections are 100.000 (with 1 CPU).
Should we setMaxConnections here always to a high value (default is 1024) and set it by CPUCount in the new Filter?

But the new setting dedicated for each listener port or is it global. Because it do also not make sense to limit each listener port to 60.000 connections.
If you run a LoadBalancer with 2 ports you still can get problems with no ports left on the LoadBalancer host.

[breuerfelix]

We do also limits in the createEnvoyCluster() function in internal/helper/yawollet.go:

	Thresholds: []*envoycluster.CircuitBreakers_Thresholds{
		{
			Priority:           envoycore.RoutingPriority_DEFAULT,
			MaxConnections:     &wrappers.UInt32Value{Value: 10000 * uint32(hostmetrics.GetCPUNum())},
			MaxRequests:        &wrappers.UInt32Value{Value: 8000 * uint32(hostmetrics.GetCPUNum())},
			MaxPendingRequests: &wrappers.UInt32Value{Value: 2000 * uint32(hostmetrics.GetCPUNum())},
		},
	},

This is set per upstream Port which even do not make sense. If I have 10 Ports the sum of all MaxConnections are 100.000 (with 1 CPU). Should we setMaxConnections here always to a high value (default is 1024) and set it by CPUCount in the new Filter?

But the new setting dedicated for each listener port or is it global. Because it do also not make sense to limit each listener port to 60.000 connections. If you run a LoadBalancer with 2 ports you still can get problems with no ports left on the LoadBalancer host.

Good point, i haven't found any option which limits the overall connections for envoy across all listeners / clusters yet.
I will put this PR into draft mode since i requires another architecture.

[breuerfelix]

I thought about this and i guess a solid way is to handle this on a system level. We can set the maximum number of open files for the yawol user to 60k and create another user which runs yawollet.
That way the yawollet is still able to open connections to the k8s api, even though we are close to our limits since envoy is not able to open more files.
Also, envoy does not not panic if there are no open files left.

What do you think about this @dergeberl @nightlyone ?

[dergeberl]

LGTM