[Kubernetes Provider] Allow wildcards or regex when specifying "include_labels" and "include_annotations"

[gizas]

Describe the enhancement:

Currently the Kubernetes Provider includes the following config options: include_annotations, include_labels(and also exclude_labels) and allows users to optionally add to the provider config the list of annotations and labels present accordingly.

Those config options does not support wild cards or regex expressions to dynamically configure the wanted list of annotations or labels. This enhancement request focuses to this functionality.

Describe a specific use case for the enhancement or feature:

Lets assume the following pod specification:

apiVersion: v1
kind: Pod
metadata:
  name: demo
  labels:
    demo-1: production
    demo-2: nginx

The user would require to configure "include_labels": ["demo-1", "demo-2"] to match all labels
So the enhancement will allow "include_labels": "demo-*" to match all demo-1 and demo-2 labels of the example pod.

What is the definition of done?

• Implement the support of wildcards (or regex) for the include_annotations, include_labelsand also exclude_labels
• Test and provide evidence for correct inclusion and exclusion
• Document the new updates in the https://www.elastic.co/guide/en/fleet/current/kubernetes-provider.html#_provider_configuration_2 section

Relevant Links:

• Config options of Kubernetes Provider
• Relevant PR to add functionality in Kubernetes Processor

[gizas]

Added initial effort elastic/elastic-agent-autodiscover@main...regexannotations

We will need to try testing this first effort

[gizas]

The PR for elastic-agent-autodiscovery library introduces the functionality.

• We will need to merge the above PR and then run some manual tests with and updated elastic agent image.
• Main question for @elastic/obs-cloudnative-monitoring team is whether we want to support regex expression for excluding labels (For now I have added the regex support also to exclude_labels)?
• Documentation updates to be provided as DoD of this story

[ChrsMark]

• Main question for @elastic/obs-cloudnative-monitoring team is whether we want to support regex expression for excluding libraries?

What kind of libraries? 🤔

[gizas]

What kind of libraries? 🤔

:) Meant to write labels !!! Rephrased it

[gizas]

I have run the manual tests end-to-end and I can verify that by adding in state_pod

add_resource_metadata:
  namespace:
    use_regex_inlcude: true
    include_labels: ["kubernetes"]

And As you can see below we match only kubernetes* labels

But we hit the bug: #3636

So we need to fix this bug before retesting this feature further

[gizas]

The above #3636 has been fixed in elastic/beats#37458, all agent versions with elastic-agent-autodiscovery library >= v0.6.6

I have resumed the tests for the regex support. The functionality is there

Draft Documentation RP: elastic/ingest-docs#814

Still I see an issue with mix regex in both namespace and node:

Config:

      add_resource_metadata:
        namespace:
          use_regex_include: true
          include_labels:
            - pas
        node:
          use_regex_include: true
          include_labels:
            - kubernetes

Output:

Only namespace is honoured. I will keep searching for this

Everything tested correctly in version>= 8.12.0