Skip to content

Commit

Permalink
Expand ingress docs a bit (#1806)
Browse files Browse the repository at this point in the history
This notes the controller intended to be used with the operator-managed ingress resources.

When setting up my own cluster I also tried https://docs.nginx.com/nginx-ingress-controller/ and https://kubernetes-sigs.github.io/aws-load-balancer-controller/ but the path format generated by the operator won't work with those.
  • Loading branch information
matschaffer-roblox authored Oct 26, 2023
1 parent d596205 commit 0ad755c
Showing 1 changed file with 10 additions and 0 deletions.
10 changes: 10 additions & 0 deletions docs/quick-start-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -238,10 +238,20 @@ and deleting the pods outside the operator might lead to incorrect metric values
## Driver UI Access and Ingress

The operator, by default, makes the Spark UI accessible by creating a service of type `ClusterIP` which exposes the UI. This is only accessible from within the cluster.

The operator also supports creating an optional Ingress for the UI. This can be turned on by setting the `ingress-url-format` command-line flag. The `ingress-url-format` should be a template like `{{$appName}}.{ingress_suffix}/{{$appNamespace}}/{{$appName}}`. The `{ingress_suffix}` should be replaced by the user to indicate the cluster's ingress url and the operator will replace the `{{$appName}}` & `{{$appNamespace}}` with the appropriate value. Please note that Ingress support requires that cluster's ingress url routing is correctly set-up. For e.g. if the `ingress-url-format` is `{{$appName}}.ingress.cluster.com`, it requires that anything `*ingress.cluster.com` should be routed to the ingress-controller on the K8s cluster.

The operator also sets both `WebUIAddress` which is accessible from within the cluster as well as `WebUIIngressAddress` as part of the `DriverInfo` field of the `SparkApplication`.

The operator generates ingress resources intended for use with the [Ingress NGINX Controller](https://kubernetes.github.io/ingress-nginx/). Include this in your application spec for the controller to ensure it recognizes the ingress and provides appropriate routes to your Spark UI.

```yaml
spec:
sparkUIOptions:
ingressAnnotations:
kubernetes.io/ingress.class: nginx
```

## About the Mutating Admission Webhook

The Kubernetes Operator for Apache Spark comes with an optional mutating admission webhook for customizing Spark driver and executor pods based on the specification in `SparkApplication` objects, e.g., mounting user-specified ConfigMaps and volumes, and setting pod affinity/anti-affinity, and adding tolerations.
Expand Down

0 comments on commit 0ad755c

Please sign in to comment.