Passing custom service port name for spark UI

[marcozov]

Whenever a SparkApplication is created, a corresponding service is created as well in order to expose the spark UI.
The resulting spec looks like this:

spec:
  clusterIP: ...
  ports:
  - name: spark-driver-ui-port
    port: 4040
    protocol: TCP
    targetPort: 4040
  selector:
    spark-role: driver
    sparkoperator.k8s.io/app-name: spark-app-name
  sessionAffinity: None
  type: ClusterIP

Most of these fields can be customized, including the port. However, the resulting port name is always going to be spark-driver-ui-port, as it is hardcoded: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/master/pkg/controller/sparkapplication/sparkui.go#L153 .

While in general this is not an issue, there are some cases where it would be nice to have control over that field.
For instance, if the spark application driver is part of a service mesh, the traffic forwarded to the UI is redirected through an Istio ingress gateway, and the access has to be restricted according to some authorization policies (so, the spark driver would run with an Istio sidecar container), the easiest way to treat traffic as proper HTTP is via explicit protocol selection. And if protocol is not selected properly, any RBAC attempt at the network level will cause all the incoming traffic to be rejected (and Istio does not seem to be able to select protocol properly and it will just discard incoming traffic because it will be treated as TCP).

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.