Add spark-operator chart

[hagaibarel]

Related to #1021

[scravy]

This fails on EKS Kubernetes v1.18.8 due to:

Error: chart requires kubeVersion: >= 1.13.x which is incompatible with Kubernetes v1.18.8-eks-7c9bda

[scravy]

I removed the kubeVersion and getting this now:

Release "spark-operator" does not exist. Installing it now.

Error: failed to install CRD crds/sparkoperator.k8s.io_scheduledsparkapplications.yaml: CustomResourceDefinition.apiextensions.k8s.io "scheduledsparkapplications.sparkoperator.k8s.io" is invalid: [spec.validation.openAPIV3Schema.properties[spec].properties[template].properties[executor].properties[sidecars].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property, spec.validation.openAPIV3Schema.properties[spec].properties[template].properties[executor].properties[initContainers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property, spec.validation.openAPIV3Schema.properties[spec].properties[template].properties[driver].properties[sidecars].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property, spec.validation.openAPIV3Schema.properties[spec].properties[template].properties[driver].properties[initContainers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property]

[hagaibarel]

I removed the kubeVersion and getting this now:

Release "spark-operator" does not exist. Installing it now.

Error: failed to install CRD crds/sparkoperator.k8s.io_scheduledsparkapplications.yaml: CustomResourceDefinition.apiextensions.k8s.io "scheduledsparkapplications.sparkoperator.k8s.io" is invalid: [spec.validation.openAPIV3Schema.properties[spec].properties[template].properties[executor].properties[sidecars].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property, spec.validation.openAPIV3Schema.properties[spec].properties[template].properties[executor].properties[initContainers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property, spec.validation.openAPIV3Schema.properties[spec].properties[template].properties[driver].properties[sidecars].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property, spec.validation.openAPIV3Schema.properties[spec].properties[template].properties[driver].properties[initContainers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property]

Seems like an issue with CRD changes intoduced in 1.18, I think this is the issue kubernetes/kubernetes#91395, didn't see it before as I was testing on versions 1.16 and 1.17

Pinging @yuchaoran2011 @liyinan926

[liyinan926]

LGTM. Thanks for adding the chart!

[scravy]

From which repo will we be able to install this chart?

[jkleckner]

The current most recent commit [1] still advises to use the deprecated [2] repo source.

If I fetch the chart it is the old one. What am I missing?

[1] 98414e7

[2] https://charts.helm.sh/incubator/ and follow the status or deprecation links.

[jgardner04]

@jkleckner the action is in place but the gh-pages branch needs to be created by a contributor. This will enable the creation of an index.yaml file in that branch that will allow you to add it as a source in Helm 3. I do have this working on my fork if you are in a bind and need to use it to deploy. You would use http://beyondthecorneroffice.com/spark-on-k8s-operator as your repo.

[jkleckner]

@jgardner04 Thank you.

It turns out you can use a local file path for the chart as well, which I got working. We are also going to bundle this chart into our own chart museum.

As a side-effect, I noticed that current kubernetes v1.19.x requires some patches as seen in [1].

So I created a PR #1084 for this repo [2] that brings those changes back here.

[1] kudobuilder/operators#297

[2] #1084

[jkleckner]

I removed the kubeVersion and getting this now:

Release "spark-operator" does not exist. Installing it now.

Error: failed to install CRD crds/sparkoperator.k8s.io_scheduledsparkapplications.yaml: CustomResourceDefinition.apiextensions.k8s.io "scheduledsparkapplications.sparkoperator.k8s.io" is invalid: [spec.validation.openAPIV3Schema.properties[spec].properties[template].properties[executor].properties[sidecars].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property, spec.validation.openAPIV3Schema.properties[spec].properties[template].properties[executor].properties[initContainers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property, spec.validation.openAPIV3Schema.properties[spec].properties[template].properties[driver].properties[sidecars].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property, spec.validation.openAPIV3Schema.properties[spec].properties[template].properties[driver].properties[initContainers].items.properties[ports].items.properties[protocol].default: Required value: this property is in x-kubernetes-list-map-keys, so it must have a default or be a required property]

Seems like an issue with CRD changes intoduced in 1.18, I think this is the issue kubernetes/kubernetes#91395, didn't see it before as I was testing on versions 1.16 and 1.17

Pinging @yuchaoran2011 @liyinan926

@hagaibarel This should be fixed with #1084

[jkleckner]

@hagaibarel have you tried tolerations at all?

I'm finding that nodeSelector and tolerations don't work declared in the executor spec.

The nodeSelector works if put directly in the spec and if the label is fully qualified with kubernetes.io/ as for example kubernetes.io/hostname: "docker-desktop" but tolerations don't work even with trying out various paths.

I'll file an issue.