[Core feature] Spark, Ray, Dask plugins should support pod_template and pod_template_name arguments

[cosmicBboy]

Motivation: Why do you think this is important?

Similar to flyteorg/flytekit#1515, we want to add support for the new pod template configuration for the spark, ray, and dask plugins:

• spark plugin code changes: https://github.com/flyteorg/flytekit/tree/master/plugins/flytekit-spark
• spark plugin flytesnacks docs: https://github.com/flyteorg/flytesnacks/tree/master/cookbook/integrations/kubernetes/k8s_spark
• ray plugin code changes: https://github.com/flyteorg/flytekit/tree/master/plugins/flytekit-ray
• ray plugin flytesnacks docs: https://github.com/flyteorg/flytesnacks/tree/master/cookbook/integrations/kubernetes/ray_example
• dask plugin code changes: https://github.com/flyteorg/flytekit/tree/master/plugins/flytekit-dask
• dask plugin flytesnacks docs: https://github.com/flyteorg/flytesnacks/tree/master/cookbook/integrations/kubernetes/k8s_dask

Goal: What should the final outcome look like, ideally?

Each of the plugins mentioned above needs to accept pod_template and pod_template_name arguments, like the ContainerTask PR, with documentation updates in the Spark, Ray, and Dask integration examples.

Describe alternatives you've considered

NA

Propose: Link/Inline OR Additional context

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[hamersaw]

@cosmicBboy we should close this issue, or at least reword it significantly because it is very misleading. There does not need to be any changes to flytekit for any of these plugins to support the new PodTemplate work. In addition, it is not clear that this is useful with Spark.

[cosmicBboy]

cool, closing this now. @hamersaw would you mind creating a new issue with the correct scope?