Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow modifying trafficPolicy for L4 ILB services #1191

Merged
merged 2 commits into from
Jul 31, 2020
Merged

Allow modifying trafficPolicy for L4 ILB services #1191

merged 2 commits into from
Jul 31, 2020

Conversation

prameshj
Copy link
Contributor

Switching from Local to Cluster trafficPolicy (or vice versa) should change the endpoints calculator mode.

Also added logic to pick a random subset of upto 3 nodes in case of Local mode, with no endpoints. With this change, accessing the VIP will return "Connection refused" immediately, instead of timing out after a few seconds.
The behavior is unchanged for cases where there are endpoints.

/assign @freehan

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 20, 2020
@k8s-ci-robot k8s-ci-robot requested review from bowei and MrHohn July 20, 2020 16:57
freehan
freehan reviewed Jul 23, 2020
View reviewed changes
pkg/neg/manager.go Outdated
Subset: servicePortKey.Subset,
SubsetLabels: portInfo.SubsetLabels,
NegType: networkEndpointType,
RandomizeEndpoints: portInfo.RandomizeEndpoints,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about using the some representation of EndpointCalculator here?

FOr example types.L4LocalMode?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done. Also fixed the other comments.

pkg/neg/types/types.go Outdated
// RandomizeEndpoints indicates if the endpoints for the NEG associated with this port need to
// be selected at random, rather than selecting the endpoints of this service. This is applicable
// in GCE_VM_IP NEGs where the endpoints are the nodes instead of pods.
RandomizeEndpoints bool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to generalize this into a enum

pkg/neg/syncers/endpoints_calculator.go Outdated
@@ -85,8 +85,10 @@ func (l *LocalL4ILBEndpointsCalculator) CalculateEndpoints(ep *v1.Endpoints, cur
}
}
if numEndpoints == 0 {
// TODO verify the behavior seen by a client when accessing an ILB whose NEGs have no endpoints.
return nil, nil, nil
// pick upto 3 nodes at random to send "ConnectionRefused" response to clients.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering what happened when the ILB has no backends?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The requests to VIP will timeout instead of getting Connection Refused immediately. This is a regression from the k/k implementation where "ICMP Connection Refused" is returned.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it is worth introducing this complication. Recommend leaving this out from this fix.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

pkg/neg/syncers/endpoints_calculator.go Outdated

nodeZoneMap := make(map[string][]*v1.Node)
func getZoneNodesMap(nodes []*v1.Node, zonegetter types.ZoneGetter) map[string][]*v1.Node {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add comment

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 24, 2020
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 24, 2020
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 29, 2020
@prameshj prameshj force-pushed the vmipneg-fix3 branch 2 times, most recently from 387db86 to cd9eb11 Compare July 29, 2020 14:55
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 29, 2020
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 30, 2020
prameshj added 2 commits July 30, 2020 15:57
@prameshj
Include EndpointsCalculatorMode in negSyncerKey.
f5bad86 
This is neeeded to identify a service TrafficPolicy change as an
event to stop the old syncer and start a new one with the new mode.
This is relevant only for VM_IP NEGs. The field will have the value
"L7" for the VM_IP_PORT NEGs.

Changed portInfoMap as well.
@prameshj
unit test
17f7a04
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 30, 2020
freehan
freehan reviewed Jul 31, 2020
View reviewed changes
Copy link
Contributor

@freehan freehan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall

just one question left

pkg/neg/controller.go
@@ -527,7 +527,7 @@ func (c *Controller) mergeVmIpNEGsPortInfo(service *apiv1.Service, name types.Na
// Update usage metrics.
negUsage.VmIpNeg = usage.NewVmIpNegType(onlyLocal)

return portInfoMap.Merge(negtypes.NewPortInfoMapForVMIPNEG(name.Namespace, name.Name, c.namer, !onlyLocal))
return portInfoMap.Merge(negtypes.NewPortInfoMapForVMIPNEG(name.Namespace, name.Name, c.namer, onlyLocal))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this changed? Do you want to enable readiness feedback?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change is because i was previously using this parameter as "randomizeEndpoints" boolean. RandomizeEndpoints is true for Cluster traffic Policy and false for Local traffic policy. This is not the readiness gate boolean.

ingress-gce/pkg/neg/types/types.go

Line 161 in 332d3e0

func NewPortInfoMapForVMIPNEG(namespace, name string, namer NetworkEndpointGroupNamer, randomize bool) PortInfoMap {

@freehan
Copy link
Contributor

freehan commented Jul 31, 2020

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 31, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: freehan, prameshj

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 31, 2020
@k8s-ci-robot k8s-ci-robot merged commit b1e1c03 into kubernetes:master Jul 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@freehan freehan freehan left review comments

@bowei bowei Awaiting requested review from bowei

@MrHohn MrHohn Awaiting requested review from MrHohn

Assignees

@freehan freehan

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@prameshj @freehan @k8s-ci-robot