Setting publisher_pipeline.disable_host: true has no effect

[chrisberkhout]

The publisher_pipeline.disable_host option is documented as:

publisher_pipeline.disable_host

By default, all events contain host.name. This option can be set to true to disable the addition of this field to all events. The default value is false.

The setting is applied in libbeat/publisher/processing/default.go.

However, with publisher_pipeline.disable_host: true set, host.* fields will still be added, unless the forwarded tag is also set.

I suspect this is because there is a separate default processor to add host metadata when tags don't contain forwarded.

The documentation above, and some logic, refers only to host.name, but without the forwarded tag a range of fields will be set. For example:

    "host": {
      "hostname": "docker-fleet-agent",
      "os": {
        "kernel": "6.10.9-arch1-2",
        "codename": "focal",
        "name": "Ubuntu",
        "type": "linux",
        "family": "debian",
        "version": "20.04.6 LTS (Focal Fossa)",
        "platform": "ubuntu"
      },
      "containerized": false,
      "ip": [
        "172.18.0.7"
      ],
      "name": "docker-fleet-agent",
      "id": "d7b94aeb4f9141eaa5f345ec31e65c86",
      "mac": [
        "02-42-AC-12-00-07"
      ],
      "architecture": "x86_64"
    },

It would be good to remove the host.name and publisher_pipeline.disable_host: true logic and documentation, and document the host.* default processor behavior.

An integration that demonstrates the effect of these settings can be generated with this script.

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[taylor-swanson]

To add to this, I've observed another strange behavior.

While true that adding forwarded to the list of tags will prevent host.* fields from being added, if you add an append processor that adds fields to tags, host.* fields will be added regardless of if forwarded is part of the tag list.

Scenarios tested (this was done using a Custom TCP integration):

• Default TCP integration settings: (host.* fields added, this is expected)
• Added forwarded tag using TCP integration settings: (NO host.* fields added, this is expected)
• Added append processor that adds an unrelated tag, the forwarded tag is still present from before. (host.* fields added, this is NOT expected)

With that final test case, here is the list of tags in the indexed document:

[cisco_ise, forwarded]

Here is the rendered configuration for the stream from the agent policy:

- id: tcp-tcp-2d21df58-7339-4419-bb1a-5d2fbe75efad
  name: tcp-1
  revision: 3
  type: tcp
  use_output: 35856fb2-d4f6-4fe5-ac91-4745ac5a40f7
  meta:
    package:
      name: tcp
      version: 1.19.1
  data_stream:
    namespace: default
  package_policy_id: 2d21df58-7339-4419-bb1a-5d2fbe75efad
  streams:
    - id: tcp-tcp.generic-2d21df58-7339-4419-bb1a-5d2fbe75efad
      data_stream:
        dataset: tcp.generic
      host: 'localhost:9002'
      ssl: null
      tags:
        - forwarded
      publisher_pipeline.disable_host: true
      processors:
        - append:
            target_field: tags
            values:
              - cisco_ise