Clashes with fixed ports in NodePort based services

[krishnadurai]

Ran kfctl apply on a fresh GKE cluster and encountered this issue:

Error:  (kubeflow.error): Code 500 with message: kfApp Apply failed for kustomize:  (kubeflow.error): Code 500 with message: Apply.Run  Error error when creating "/tmp/kout114645107": Service "istio-ingressgateway" is invalid: spec.ports[1].nodePort: Invalid value: 31380: provided port is already allocated
Usage:
  kfctl apply -f ${CONFIG} [flags]

Flags:
  -f, --file string   Static config file to use. Can be either a local path or a URL.
                      For example:
                      export CONFIG=https://raw.githubusercontent.com/kubeflow/manifests/v0.7-branch/kfdef/kfctl_existing_arrikto.0.7.0.yaml
                      export CONFIG=https://raw.githubusercontent.com/kubeflow/manifests/v0.7-branch/kfdef/kfctl_aws.0.7.0.yaml
                      export CONFIG=https://raw.githubusercontent.com/kubeflow/manifests/v0.7-branch/kfdef/kfctl_gcp_iap.0.7.0.yaml
                      export CONFIG=https://raw.githubusercontent.com/kubeflow/manifests/v0.7-branch/kfdef/kfctl_k8s_istio.0.7.0.yaml
                      kfctl apply -V --file=${CONFIG}
  -h, --help          help for apply
  -V, --verbose       verbose output default is false

 (kubeflow.error): Code 500 with message: kfApp Apply failed for kustomize:  (kubeflow.error): Code 500 with message: Apply.Run  Error error when creating "/tmp/kout114645107": Service "istio-ingressgateway" is invalid: spec.ports[1].nodePort: Invalid value: 31380: provided port is already allocated

Apparently, default-http-backend in kube-system had already taken port number 31380:

kubectl get svc -n kube-system default-http-backend
NAME                   TYPE       CLUSTER-IP   EXTERNAL-IP   PORT(S)        AGE
default-http-backend   NodePort   10.0.7.235   <none>        80:31380/TCP   57m

We have a rare risk of running into this problem in case if a service deployed earlier occupies the NodePort of a fixed NodePort numbered service to be deployed later.
In our case, istio-ingressgateway service uses the port 31380.

/cc @yanniszark @jlewi

[jtfogarty]

/area engprod
/kind bug
/priority p1

[jlewi]

With 0.7 are we still using a fixed node port for the ingress gateway?

[yanniszark]

@jlewi we still are, looking at the Istio manifests:
https://github.com/kubeflow/manifests/blob/master/istio/istio-install/base/istio-noauth.yaml
https://github.com/kubeflow/manifests/blob/master/istio-1-3-1/istio-install-1-3-1/base/service.yaml

In addition, I don't see any code anywhere depending on these values.
Do you think we could get rid of them?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in one week if no further activity occurs. Thank you for your contributions.

[issue-label-bot]

Issue-Label Bot is automatically applying the labels:

Label	Probability
area/kfctl	0.51

Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

[stale]

This issue has been closed due to inactivity.