Some fields have invalid dynamic mapping definitions

[jsoriano]

While investigating elastic/kibana#129344 (comment), I found some field definitions for dynamic mappings that seem invalid.

The fields are these ones:

• kubernetes.scheduler.pod.preemption.victims.bucket.*
• azure.auditlogs.target_resources.*
• azure.auditlogs.target_resources.*.modified_properties.*

They have wildcards, but they don't have type: object, so no dynamic mapping is being installed for them. They are being installed as plain fields with the wildcard as literal part of their names.

For example, the definition for azure.auditlogs.target_resources.*.modified_properties.* is not generating any dynamic mapping, and is generating this:

{
  "template": {
    "mappings": {
      ...
      "properties": {
        ...
        "azure": {
          "properties": {
            ...
            "auditlogs": {
              "properties": {
                ...
                "properties": {
                    ...
                    "target_resources": {
                      "properties": {
                        "*": {
                          "properties": {
                            "display_name": {
                              "ignore_above": 1024,
                              "type": "keyword"
                            },
                            "id": {
                              "ignore_above": 1024,
                              "type": "keyword"
                            },
                            "ip_address": {
                              "ignore_above": 1024,
                              "type": "keyword"
                            },
                            "modified_properties": {
                              "properties": {
                                "*": {
                                  "properties": {
                                     ...
                                  }
                                }
                              }
                            },
...

Some possible action points to solve this would be:

• Fix these definitions, so they include type: object and object_type, so dynamic mappings are generated for them.
• Disallow the use of wildcards (*) when type: object is not set.
• Do nothing as this hasn't caused user-facing issues so far.

We are discussing about what to do on these cases in Fleet in elastic/kibana#129344 (comment), some ideas may also apply here.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[jsoriano]

Pinging @elastic/obs-cloud-monitoring and @elastic/obs-cloudnative-monitoring as the fields are in their integrations.

[kaiyan-sheng]

@jsoriano Thanks for catching this. Will the change on field type here consider a breaking change?

[jsoriano]

@jsoriano Thanks for catching this. Will the change on field type here consider a breaking change?

Well, currently there is no valid mapping for these fields, so this could be seen as adding fields, more than changing their types.

For example, with the current kubernetes mapping, there is only a mapping for literal kubernetes.scheduler.pod.preemption.victims.bucket.*, but there is no mapping for each individual bucket.

The change might be breaking for the literal * field, but I'd bet this is not being used 🙂. For any other field under kubernetes.scheduler.pod.preemption.victims.bucket we would be adding a mapping, not changing it.

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[jsoriano]

These fields should be handled by elastic/kibana#137978, closing this.