Update kubernetes watcher to use official client-go libraries #13051
Conversation
|
Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually? |
1 similar comment
|
Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually? |
vjsamuel
force-pushed
the
client_go
branch
4 times, most recently
from
3b2c86e to
878b3a7
Compare
exekias
added
Team:Integrations
|
ok to test |
vjsamuel
force-pushed
the
client_go
branch
2 times, most recently
from
6689e91 to
9b74f99
Compare
| @@ -36,27 +35,15 @@ const defaultNode = "localhost" | |||
| // GetKubernetesClient returns a kubernetes client. If inCluster is true, it returns an | |||
| // in cluster configuration based on the secrets mounted in the Pod. If kubeConfig is passed, | |||
| // it parses the config file to get the config required to build a client. | |||
| func GetKubernetesClient(inCluster bool, kubeConfig string) (client *k8s.Client, err error) { | |||
| if inCluster == true { | |||
There was a problem hiding this comment.
it sems you don't use inCluster anymore, how would this be used now? We would need to update the comment and some docs
There was a problem hiding this comment.
the official kube client deduces if it runs in cluster or not and hence to get a kube client we dont really need it.
if kubeconfigPath == "" && masterUrl == "" {
klog.Warningf("Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.")
kubeconfig, err := restclient.InClusterConfig()
if err == nil {
return kubeconfig, nil
}
klog.Warning("error creating inClusterConfig, falling back to default config: ", err)
}
if kubeconfig path is provided then it just tries to create an out of cluster configuration.
That said. let me fix the discover node to just follow the same thing.
| w.handler.OnAdd(obj) | ||
| } | ||
| func (w *watcher) enqueue(obj interface{}, state string) { | ||
| key, err := cache.DeletionHandlingMetaNamespaceKeyFunc(obj) |
There was a problem hiding this comment.
what is this line for? a comment could help here
There was a problem hiding this comment.
this just makes sure that it gets the namespace/name combination only for resources where state is not Unknown
vjsamuel
changed the title
…e close to Stop()
|
not sure if the failures are related. I have done manual testing on all flows and things look file. Lets have action items to now have fake client based test cases on the autodiscover/processor paths so that it is easier to find failures. there were a few issues that i fixed in the last commit one of which is to increase the default resync period. with the new client, it is no longer necessary to sync every minute. the resync period forces all items on the cache to be replayed. this is undesirable at a 1 minute interval. i have also gone ahead and removed the |
vjsamuel
changed the title
|
Thank you so much for this contribution @vjsamuel! it's a real enabler for future features |
This configuration was removed by elastic/beats#13051 and elastic/beats#13651
This configuration was removed by elastic/beats#13051 and elastic/beats#13651
This PR moves away from using the Eric Chang client-go for Kubernetes interaction in favor of the official kube client. As a result, we now use
SharedInformers. This makes sure that at any given time there is only 1 connection to the API server per resource that is requested. Moving to the client-go also makes sure that we no longer need to check ResourceVersion while watching. It is implicitly done under the hood.This change now allows the watcher to be testable and as a follow up we could add test cases.