Extraneous Istio Gateway for Knative / KFServing

[yanniszark]

Problem Statement

KFServing and Knative use Istio Gateways to expose InferenceServices. They also use a cluster-local gateway (aka not exposed to the internet - ClusterIP service backed) for internal traffic for transformer/explainer use-cases.

Currently, KFServing and Knative use gateway cluster-local-gateway in namespace knative-serving.
However, we also create gateway cluster-local-gateway in namespace istio-system.

This was introduced in very early Kubeflow releases.
However, I see that Knative and KFServing NEVER used that Gateway!!!
https://github.com/kubeflow/manifests/blob/v1.0-branch/knative/knative-serving-install/base/config-map.yaml#L677
https://github.com/kubeflow/manifests/blob/v1.1-branch/knative/knative-serving-install/base/config-map.yaml#L677

https://github.com/kubeflow/manifests/blob/v1.0-branch/kfserving/kfserving-install/base/config-map.yaml#L90-L94
https://github.com/kubeflow/manifests/blob/v1.1-branch/kfserving/kfserving-install/base/config-map.yaml#L23-L27

So that cluster-local-gateway in Istio was extraneous all along. Why did we create it in the first place though? Maybe Istio auto-generated it before but not now?

Next Steps

We can either:

• Remove cluster-local-gateway Gateway from the Istio kustomization.
• Remove cluster-local-gateway Gateway from Knative and change settings in Knative and KFServing to match the gateway in istio.

We should also try to find out why the manifests did that in the first place.

cc'ing @yuzisun @pvaneck from KFServing

[davidspek]

This gateway was used in older versions of KNative and KFServing. Currently we are hovering right around the versions where these changes occurred so the setup required for the specific versions of KFServing and KNative we are targeting will need to be closely evaluated.

[yanniszark]

@davidspek I don't think that's the case. Did you see the links I provided to older versions of Knative, from Kubeflow v1.0 and v1.1?
It was always like this. Pasting the links here again:
https://github.com/kubeflow/manifests/blob/v1.0-branch/knative/knative-serving-install/base/config-map.yaml#L677
https://github.com/kubeflow/manifests/blob/v1.1-branch/knative/knative-serving-install/base/config-map.yaml#L677

[davidspek]

@yanniszark I think the difference here is that there used to be a dedicated Istio ingress pod that was being used (I guess you could see it as a physical gateway). Whereas now an Istio Gateway resource is used that uses the same Istio ("physical") ingress-gateway pod that Kubeflow and other things use, and a separate Istio Gateway resource used for in-cluster communication (not tied to the "physical" ingress-gateway pod).

[pvaneck]

From what I can tell, the cluster-local-gateway Gateway resource in the istio-system namespace can be removed, as I don't believe that Gateway was created or used in previous releases of Kubeflow.

KFServing v0.5.1 still leverages the cluster-local-gateway Gateway resource in the knative-serving namespace. This just wasn't customizable until the v0.5.0 release from this PR. However, Knative still relies on the cluster-local-gateway Deployment and Service resources in the istio-system namespace.

Starting at Knative serving v19.0, the use of cluster-local-gateway deployments/services/etc was removed and instead they made the knative local gateway use the istio ingressgateway deployment. More info. So eventually, we won't need the cluster-local-gateway kustomization folder and the knative install will handle the local gateway stuff.

Just for reference, to get the same effect as what later Knative serving versions are doing, you could delete the initial cluster-local-gateway deployment resources:

kustomize build common/istio-1-9-0/cluster-local-gateway/base/ | kubectl delete -f -

Then do the following:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: cluster-local-gateway
  namespace: istio-system
  labels:
    networking.knative.dev/ingress-provider: istio
spec:
  type: ClusterIP
  selector:
    istio: ingressgateway
  ports:
    - name: http2
      port: 80
      targetPort: 8081
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: cluster-local-gateway
  namespace: knative-serving
  labels:
    networking.knative.dev/ingress-provider: istio
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 8081
      name: http
      protocol: HTTP
    hosts:
    - "*"
EOF

Then InferenceServices should continue to work.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in one week if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been closed due to inactivity.