Caching of workflows and workflowtemplates

[ddseapy]

Summary

One of our java applications needs a cache of workflows and workflowtemplates to make various decisions, as requesting them regularly from the argo server causes bandwidth issues due to the number/size of our workflows. For some things we could use the field selection to limit the amount of data coming back, but it's not ideal when wanting the full objects. We would like to still use the argo server for other less-frequent/low-bandwidth operations (submit, resubmits, retries, deletes, suspend, resume, etc...).

We saw that the argo java client transitively pulls in the kubernetes java sdk, so we thought to simply use a kubernetes informer to maintain the cache the wf/wftmpl for us. Unfortunately there are several differences between the way wf/wftmpl objects are serialized when talking to the kubernetes API server and the argo server:

• https://github.com/argoproj-labs/argo-client-java/blob/7a8cb4a4ecf3dafbb0a2170665c2dd2c28317017/src/main/java/io/argoproj/workflow/models/WorkflowTemplateSpec.java#L34
• https://github.com/argoproj-labs/argo-client-java/blob/01d54d6064a02e14942fd30fd90267a4edde5264/src/main/java/io/argoproj/workflow/models/K8sIoApiPolicyV1beta1PodDisruptionBudgetSpec.java#L23 vs https://github.com/kubernetes-client/java/blob/31fbb3a52c6ca7a6ad6d71cd154b387191705adc/kubernetes/src/main/java/io/kubernetes/client/custom/IntOrString.java

To test, after a couple quick hacks on these and adding "implements KubernetesObject" to Workflow and WorkflowTemplate objects, the kubernetes api informer worked.

Questions

• What's the best way to retain a full cache of workflows and workflowtemplates in our applications, while still leveraging the argo server for other operations on the cluster?
• If leveraging a kubernetes shared informer (https://github.com/kubernetes-client/java/blob/master/examples/src/main/java/io/kubernetes/client/examples/InformerExample.java) is a valid approach, is there any way the kubernetes and openapi serialized representations can align? It would be great to be able to re-use the argo clients for both direct Kubernetes API Server access along with Argo Server access.

[ddseapy]

Looks like this can be fixed in the client gen project, so will put up a PR there.

[gpomykala]

@ddseapy looking for exactly same for Go api client, was aforementioned change applied? If so, would to mind to reference a PR?