[ECS] Cleaning up incorrect invalid ECS field usages at root-level

packages/azure_frontdoor/changelog.yml

@@ -1,9 +1,14 @@
-- version: 1.3.1
+- version: "1.3.2"
+  changes:
+    - description: Removing unused ECS field declarations.
+      type: bugfix
+      link: https://github.com/elastic/integrations/pull/7965
+- version: "1.3.1"
   changes:
     - description: Add null checks and ignore_missing checks to the rename processor
       type: bugfix
       link: https://github.com/elastic/integrations/pull/7953
-- version: 1.3.0
+- version: "1.3.0"
   changes:
     - description: ECS version updated to 8.10.0.
       type: enhancement

packages/azure_frontdoor/data_stream/access/fields/ecs.yml

@@ -10,22 +10,6 @@
   external: ecs
 - name: destination.as.organization.name
   external: ecs
-- name: destination.geo.city_name
-  external: ecs
-- name: destination.geo.continent_name
-  external: ecs
-- name: destination.geo.country_iso_code
-  external: ecs
-- name: destination.geo.country_name
-  external: ecs
-- name: destination.geo.location
-  external: ecs
-- name: destination.geo.name
-  external: ecs
-- name: destination.geo.region_iso_code
-  external: ecs
-- name: destination.geo.region_name
-  external: ecs
 - name: destination.ip
   external: ecs
 - name: destination.port
@@ -66,16 +50,6 @@
   external: ecs
 - name: source.as.organization.name
   external: ecs
-- name: geo.continent_name
-  external: ecs
-- name: geo.country_iso_code
-  external: ecs
-- name: geo.country_name
-  external: ecs
-- name: geo.location
-  external: ecs
-- name: geo.city_name
-  external: ecs
 - name: log.level
   external: ecs
 - name: source.geo.city_name

packages/azure_frontdoor/data_stream/waf/fields/ecs.yml

@@ -10,22 +10,6 @@
   external: ecs
 - name: destination.as.organization.name
   external: ecs
-- name: destination.geo.city_name
-  external: ecs
-- name: destination.geo.continent_name
-  external: ecs
-- name: destination.geo.country_iso_code
-  external: ecs
-- name: destination.geo.country_name
-  external: ecs
-- name: destination.geo.location
-  external: ecs
-- name: destination.geo.name
-  external: ecs
-- name: destination.geo.region_iso_code
-  external: ecs
-- name: destination.geo.region_name
-  external: ecs
 - name: destination.ip
   external: ecs
 - name: destination.port
@@ -66,16 +50,6 @@
   external: ecs
 - name: source.as.organization.name
   external: ecs
-- name: geo.continent_name
-  external: ecs
-- name: geo.country_iso_code
-  external: ecs
-- name: geo.country_name
-  external: ecs
-- name: geo.location
-  external: ecs
-- name: geo.city_name
-  external: ecs
 - name: log.level
   external: ecs
 - name: source.geo.city_name

packages/azure_frontdoor/docs/README.md

@@ -96,14 +96,6 @@ Users can also use this in case of a Hybrid Cloud model, where one may define th
 | destination.as.number | Unique number allocated to the autonomous system. The autonomous system number (ASN) uniquely identifies each network on the Internet. | long |
 | destination.as.organization.name | Organization name. | keyword |
 | destination.as.organization.name.text | Multi-field of `destination.as.organization.name`. | match_only_text |
-| destination.geo.city_name | City name. | keyword |
-| destination.geo.continent_name | Name of the continent. | keyword |
-| destination.geo.country_iso_code | Country ISO code. | keyword |
-| destination.geo.country_name | Country name. | keyword |
-| destination.geo.location | Longitude and latitude. | geo_point |
-| destination.geo.name | User-defined description of a location, at the level of granularity they care about. Could be the name of their data centers, the floor number, if this describes a local physical entity, city names. Not typically used in automated geolocation. | keyword |
-| destination.geo.region_iso_code | Region ISO code. | keyword |
-| destination.geo.region_name | Region name. | keyword |
 | destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
 | destination.port | Port of the destination. | long |
 | ecs.version | ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
@@ -116,11 +108,6 @@ Users can also use this in case of a Hybrid Cloud model, where one may define th
 | event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. `event.type` represents a categorization "sub-bucket" that, when used along with the `event.category` field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
 | file.mime_type | MIME type should identify the format of the file or stream of bytes using https://www.iana.org/assignments/media-types/media-types.xhtml[IANA official types], where possible. When more than one type is applicable, the most specific type should be used. | keyword |
 | file.size | File size in bytes. Only relevant when `file.type` is "file". | long |
-| geo.city_name | City name. | keyword |
-| geo.continent_name | Name of the continent. | keyword |
-| geo.country_iso_code | Country ISO code. | keyword |
-| geo.country_name | Country name. | keyword |
-| geo.location | Longitude and latitude. | geo_point |
 | host.architecture | Operating system architecture. | keyword |
 | host.containerized | If the host is a container. | boolean |
 | host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |
@@ -228,14 +215,6 @@ Users can also use this in case of a Hybrid Cloud model, where one may define th
 | destination.as.number | Unique number allocated to the autonomous system. The autonomous system number (ASN) uniquely identifies each network on the Internet. | long |
 | destination.as.organization.name | Organization name. | keyword |
 | destination.as.organization.name.text | Multi-field of `destination.as.organization.name`. | match_only_text |
-| destination.geo.city_name | City name. | keyword |
-| destination.geo.continent_name | Name of the continent. | keyword |
-| destination.geo.country_iso_code | Country ISO code. | keyword |
-| destination.geo.country_name | Country name. | keyword |
-| destination.geo.location | Longitude and latitude. | geo_point |
-| destination.geo.name | User-defined description of a location, at the level of granularity they care about. Could be the name of their data centers, the floor number, if this describes a local physical entity, city names. Not typically used in automated geolocation. | keyword |
-| destination.geo.region_iso_code | Region ISO code. | keyword |
-| destination.geo.region_name | Region name. | keyword |
 | destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
 | destination.port | Port of the destination. | long |
 | ecs.version | ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword |
@@ -248,11 +227,6 @@ Users can also use this in case of a Hybrid Cloud model, where one may define th
 | event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. `event.type` represents a categorization "sub-bucket" that, when used along with the `event.category` field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword |
 | file.mime_type | MIME type should identify the format of the file or stream of bytes using https://www.iana.org/assignments/media-types/media-types.xhtml[IANA official types], where possible. When more than one type is applicable, the most specific type should be used. | keyword |
 | file.size | File size in bytes. Only relevant when `file.type` is "file". | long |
-| geo.city_name | City name. | keyword |
-| geo.continent_name | Name of the continent. | keyword |
-| geo.country_iso_code | Country ISO code. | keyword |
-| geo.country_name | Country name. | keyword |
-| geo.location | Longitude and latitude. | geo_point |
 | host.architecture | Operating system architecture. | keyword |
 | host.containerized | If the host is a container. | boolean |
 | host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword |

packages/azure_frontdoor/manifest.yml

@@ -1,7 +1,7 @@
 format_version: 2.11.0
 name: azure_frontdoor
 title: "Azure Frontdoor"
-version: "1.3.1"
+version: "1.3.2"
 description: "This Elastic integration collects logs from Azure Frontdoor."
 type: integration
 categories:

packages/carbonblack_edr/changelog.yml

@@ -1,5 +1,10 @@
 # newer versions go on top
-- version: 1.14.0
+- version: "1.14.1"
+  changes:
+    - description: Removing unused ECS field declarations.
+      type: bugfix
+      link: https://github.com/elastic/integrations/pull/7965
+- version: "1.14.0"
   changes:
     - description: ECS version updated to 8.10.0.
       type: enhancement

packages/carbonblack_edr/data_stream/log/fields/ecs.yml

@@ -64,8 +64,6 @@
   external: ecs
 - name: observer.version
   external: ecs
-- name: os.type
-  external: ecs
 - name: process.command_line
   external: ecs
 - name: process.entity_id

packages/carbonblack_edr/docs/README.md

@@ -319,7 +319,6 @@ An example event for `log` looks as following:
 | observer.type | The type of the observer the data is coming from. There is no predefined list of observer types. Some examples are `forwarder`, `firewall`, `ids`, `ips`, `proxy`, `poller`, `sensor`, `APM server`. | keyword |
 | observer.vendor | Vendor name of the observer. | keyword |
 | observer.version | Observer version. | keyword |
-| os.type | Use the `os.type` field to categorize the operating system into one of the broad commercial families. If the OS you're dealing with is not listed as an expected value, the field should not be populated. Please let us know by opening an issue with ECS, to propose its addition. | keyword |
 | process.command_line | Full command line that started the process, including the absolute path to the executable, and all arguments. Some arguments may be filtered to protect sensitive information. | wildcard |
 | process.command_line.text | Multi-field of `process.command_line`. | match_only_text |
 | process.entity_id | Unique identifier for the process. The implementation of this is specified by the data source, but some examples of what could be used here are a process-generated UUID, Sysmon Process GUIDs, or a hash of some uniquely identifying components of a process. Constructing a globally unique identifier is a common practice to mitigate PID reuse as well as to identify a specific process over time, across multiple monitored hosts. | keyword |

packages/carbonblack_edr/manifest.yml

@@ -1,6 +1,6 @@
 name: carbonblack_edr
 title: VMware Carbon Black EDR
-version: "1.14.0"
+version: "1.14.1"
 description: Collect logs from VMware Carbon Black EDR with Elastic Agent.
 type: integration
 format_version: 2.11.0

packages/cisco_meraki/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "1.15.1"
+  changes:
+    - description: Removing unused ECS field declarations.
+      type: bugfix
+      link: https://github.com/elastic/integrations/pull/7965
 - version: "1.15.0"
   changes:
     - description: Add event.action and message to specific events.

packages/cisco_meraki/data_stream/events/fields/ecs.yml

@@ -22,12 +22,6 @@
   name: destination.bytes
 - external: ecs
   name: destination.domain
-- external: ecs
-  name: destination.geo.city_name
-- external: ecs
-  name: destination.geo.country_name
-- external: ecs
-  name: destination.geo.location
 - external: ecs
   name: destination.ip
 - external: ecs
@@ -90,14 +84,6 @@
   name: file.size
 - external: ecs
   name: file.type
-- external: ecs
-  name: geo.city_name
-- external: ecs
-  name: geo.country_name
-- external: ecs
-  name: geo.name
-- external: ecs
-  name: geo.region_name
 - external: ecs
   name: group.id
 - external: ecs
@@ -200,12 +186,6 @@
   name: source.bytes
 - external: ecs
   name: source.domain
-- external: ecs
-  name: source.geo.city_name
-- external: ecs
-  name: source.geo.country_name
-- external: ecs
-  name: source.geo.location
 - external: ecs
   name: source.ip
 - external: ecs
@@ -246,22 +226,6 @@
   name: user_agent.original
 - external: ecs
   name: observer.hostname
-- external: ecs
-  name: destination.geo.continent_name
-- external: ecs
-  name: destination.geo.country_iso_code
-- external: ecs
-  name: destination.geo.region_iso_code
-- external: ecs
-  name: destination.geo.region_name
-- external: ecs
-  name: source.geo.continent_name
-- external: ecs
-  name: source.geo.country_iso_code
-- external: ecs
-  name: source.geo.region_iso_code
-- external: ecs
-  name: source.geo.region_name
 - external: ecs
   name: network.vlan.id
 - external: ecs
@@ -276,22 +240,6 @@
   name: threat.indicator.file.name
 - external: ecs
   name: threat.indicator.file.hash.sha256
-- external: ecs
-  name: client.geo.city_name
-- external: ecs
-  name: client.geo.continent_name
-- external: ecs
-  name: client.geo.country_iso_code
-- external: ecs
-  name: client.geo.country_name
-- external: ecs
-  name: client.geo.location.lat
-- external: ecs
-  name: client.geo.location.lon
-- external: ecs
-  name: client.geo.region_iso_code
-- external: ecs
-  name: client.geo.region_name
 - external: ecs
   name: organization.id
 - external: ecs

packages/cisco_meraki/data_stream/log/fields/ecs.yml

@@ -90,14 +90,6 @@
   name: file.size
 - external: ecs
   name: file.type
-- external: ecs
-  name: geo.city_name
-- external: ecs
-  name: geo.country_name
-- external: ecs
-  name: geo.name
-- external: ecs
-  name: geo.region_name
 - external: ecs
   name: group.id
 - external: ecs

packages/cisco_meraki/docs/README.md

@@ -168,10 +168,6 @@ The `cisco_meraki.log` dataset provides events from the configured syslog server
 | file.path.text | Multi-field of `file.path`. | match_only_text |
 | file.size | File size in bytes. Only relevant when `file.type` is "file". | long |
 | file.type | File type (file, dir, or symlink). | keyword |
-| geo.city_name | City name. | keyword |
-| geo.country_name | Country name. | keyword |
-| geo.name | User-defined description of a location, at the level of granularity they care about. Could be the name of their data centers, the floor number, if this describes a local physical entity, city names. Not typically used in automated geolocation. | keyword |
-| geo.region_name | Region name. | keyword |
 | group.id | Unique identifier for the group on the system/platform. | keyword |
 | group.name | Name of the group. | keyword |
 | host.architecture | Operating system architecture. | keyword |
@@ -421,14 +417,6 @@ An example event for `log` looks as following:
 | cisco_meraki.event.sharedSecret | User defined secret to be validated by the webhook receiver (optional) | keyword |
 | cisco_meraki.event.version | Current version of webhook format | keyword |
 | client.domain | The domain name of the client system. This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment. | keyword |
-| client.geo.city_name | City name. | keyword |
-| client.geo.continent_name | Name of the continent. | keyword |
-| client.geo.country_iso_code | Country ISO code. | keyword |
-| client.geo.country_name | Country name. | keyword |
-| client.geo.location.lat | Longitude and latitude. | geo_point |
-| client.geo.location.lon | Longitude and latitude. | geo_point |
-| client.geo.region_iso_code | Region ISO code. | keyword |
-| client.geo.region_name | Region name. | keyword |
 | client.ip | IP address of the client (IPv4 or IPv6). | ip |
 | client.mac | MAC address of the client. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
 | client.registered_domain | The highest registered client domain, stripped of the subdomain. For example, the registered domain for "foo.example.com" is "example.com". This value can be determined precisely with a list like the public suffix list (http://publicsuffix.org). Trying to approximate this by simply taking the last two labels will not work well for TLDs such as "co.uk". | keyword |
@@ -456,13 +444,6 @@ An example event for `log` looks as following:
 | destination.as.organization.name.text | Multi-field of `destination.as.organization.name`. | match_only_text |
 | destination.bytes | Bytes sent from the destination to the source. | long |
 | destination.domain | The domain name of the destination system. This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment. | keyword |
-| destination.geo.city_name | City name. | keyword |
-| destination.geo.continent_name | Name of the continent. | keyword |
-| destination.geo.country_iso_code | Country ISO code. | keyword |
-| destination.geo.country_name | Country name. | keyword |
-| destination.geo.location | Longitude and latitude. | geo_point |
-| destination.geo.region_iso_code | Region ISO code. | keyword |
-| destination.geo.region_name | Region name. | keyword |
 | destination.ip | IP address of the destination (IPv4 or IPv6). | ip |
 | destination.mac | MAC address of the destination. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
 | destination.nat.ip | Translated ip of destination based NAT sessions (e.g. internet to private DMZ) Typically used with load balancers, firewalls, or routers. | ip |
@@ -497,10 +478,6 @@ An example event for `log` looks as following:
 | file.path.text | Multi-field of `file.path`. | match_only_text |
 | file.size | File size in bytes. Only relevant when `file.type` is "file". | long |
 | file.type | File type (file, dir, or symlink). | keyword |
-| geo.city_name | City name. | keyword |
-| geo.country_name | Country name. | keyword |
-| geo.name | User-defined description of a location, at the level of granularity they care about. Could be the name of their data centers, the floor number, if this describes a local physical entity, city names. Not typically used in automated geolocation. | keyword |
-| geo.region_name | Region name. | keyword |
 | group.id | Unique identifier for the group on the system/platform. | keyword |
 | group.name | Name of the group. | keyword |
 | host.architecture | Operating system architecture. | keyword |
@@ -578,13 +555,6 @@ An example event for `log` looks as following:
 | source.as.organization.name.text | Multi-field of `source.as.organization.name`. | match_only_text |
 | source.bytes | Bytes sent from the source to the destination. | long |
 | source.domain | The domain name of the source system. This value may be a host name, a fully qualified domain name, or another host naming format. The value may derive from the original event or be added from enrichment. | keyword |
-| source.geo.city_name | City name. | keyword |
-| source.geo.continent_name | Name of the continent. | keyword |
-| source.geo.country_iso_code | Country ISO code. | keyword |
-| source.geo.country_name | Country name. | keyword |
-| source.geo.location | Longitude and latitude. | geo_point |
-| source.geo.region_iso_code | Region ISO code. | keyword |
-| source.geo.region_name | Region name. | keyword |
 | source.ip | IP address of the source (IPv4 or IPv6). | ip |
 | source.mac | MAC address of the source. The notation format from RFC 7042 is suggested: Each octet (that is, 8-bit byte) is represented by two [uppercase] hexadecimal digits giving the value of the octet as an unsigned integer. Successive octets are separated by a hyphen. | keyword |
 | source.nat.ip | Translated ip of source based NAT sessions (e.g. internal client to internet) Typically connections traversing load balancers, firewalls, or routers. | ip |