Skip to content

Latest commit

 

History

History
419 lines (316 loc) · 27.1 KB

CHANGELOG.next.asciidoc

File metadata and controls

419 lines (316 loc) · 27.1 KB
Raw

Beats version HEAD

Breaking changes

Affecting all Beats

  • Update Go version to 1.22.5. 40082

  • Fix FQDN being lowercased when used as host.hostname 39993

  • Beats won’t log start up information when running under the Elastic Agent {40390}40390[40390]

Auditbeat

Filebeat

Heartbeat

Metricbeat

  • Setting period for counter cache for Prometheus remote_write at least to 60sec 38553

  • Add support of Graphite series 1.1.0+ tagging extension for statsd module. 39619

  • Allow metricsets to report their status via control v2 protocol. 40025

  • Remove fallback to the node limit for the kubernetes.pod.cpu.usage.limit.pct and kubernetes.pod.memory.usage.limit.pct metrics calculation

  • Add support for Kibana status metricset in v8 format 40275

Osquerybeat

  • Add action responses data stream, allowing osquerybeat to post action results directly to elasticsearch. 39143

  • Disable allow_unsafe osquery configuration. 40130

  • Upgrade to osquery 5.12.1. 40368

Packetbeat

Winlogbeat

  • Add "event.category" and "event.type" to Sysmon module for EventIDs 8, 9, 19, 20, 27, 28, 255 35193

Functionbeat

Elastic Logging Plugin

Bugfixes

Affecting all Beats

  • Fix namespace filter option on add_kubernetes_metadata processor. 39934

  • Support for multiline zookeeper logs 2496

  • Add checks to ensure reloading of units if the configuration actually changed. 34346

  • Fix namespacing on self-monitoring 32336

  • Fix namespacing on self-monitoring 32336

  • Fix Beats started by agent do not respect the allow_older_versions: true configuration flag 34227 34964

  • Fix performance issues when we have a lot of inputs starting and stopping by allowing to disable global processors under fleet. 35000 35031

  • 'add_cloud_metadata' processor - add cloud.region field for GCE cloud provider

  • 'add_cloud_metadata' processor - update azure metadata api version to get missing cloud.account.id field

  • Upgraded apache arrow library used in x-pack/libbeat/reader/parquet from v11 to v12.0.1 in order to fix cross-compilation issues 35640

  • Fix panic when MaxRetryInterval is specified, but RetryInterval is not 35820

  • Support build of projects outside of beats directory 36126

  • Support Elastic Agent control protocol chunking support 37343

  • Lower logging level to debug when attempting to configure beats with unknown fields from autodiscovered events/environments 37816[37816]

  • Set timeout of 1 minute for FQDN requests 37756

  • Fix the paths in the .cmd script added to the path by the Windows MSI to point to the new C:\Program Files installation location. elastic/elastic-stack-installers#238

  • Change cache processor documentation from write_period to write_interval. 38561

  • Fix cache processor expiries heap cleanup on partial file writes. 38561

  • Fix cache processor expiries infinite growth when large a large TTL is used and recurring keys are cached. 38561

  • Fix parsing of RFC 3164 process IDs in syslog processor. 38947 38982

  • Rename the field "apache2.module.error" to "apache.module.error" in Apache error visualization. 39480 39481

  • Validate config of the replace processor 40047

  • Fix handling of escaped brackets in syslog structured data. 40445 40446

Auditbeat

Filebeat

  • [Gcs Input] - Added missing locks for safe concurrency 34914

  • Fix the ignore_inactive option being ignored in Filebeat’s filestream input 34770

  • Fix TestMultiEventForEOFRetryHandlerInput unit test of CometD input 34903

  • Add input instance id to request trace filename for httpjson and cel inputs 35024

  • Fixes "Can only start an input when all related states are finished" error when running under Elastic-Agent 35250 33653

  • [system] sync system/auth dataset with system integration 1.29.0. 35581

  • [GCS Input] - Fixed an issue where bucket_timeout was being applied to the entire bucket poll interval and not individual bucket object read operations. Fixed a map write concurrency issue arising from data races when using a high number of workers. Fixed the flaky tests that were present in the GCS test suit. 35605

  • Fixed concurrency and flakey tests issue in azure blob storage input. 35983 36124

  • Fix panic when sqs input metrics getter is invoked 36101 36077

  • Fix handling of Juniper SRX structured data when there is no leading junos element. 36270 36308

  • Fix Filebeat Cisco module with missing escape character 36325 36326

  • Added a fix for Crowdstrike pipeline handling process arrays 36496

  • [threatintel] MISP pagination fixes 37898

  • Fix file handle leak when handling errors in filestream 37973

  • Fix a race condition that could crash Filebeat with a "negative WaitGroup counter" error 38094

  • Prevent HTTPJSON holding response bodies between executions. 35219 38116

  • Fix "failed processing S3 event for object key" error on aws-s3 input when key contains the "+" character 38012 38125

  • Fix duplicated addition of regexp extension in CEL input. 38181

  • Fix the incorrect values generated by the uri_parts processor. 38216

  • Fix HTTPJSON handling of empty object bodies in POST requests. 33961 38290

  • Fix PEM key validation for CEL and HTTPJSON inputs. 38405

  • Fix filebeat gcs input panic 38407

  • Rename activity_guid to activity_id in ETW input events to suit other Windows inputs. 38530

  • Add missing provider registration and fix published entity for Active Directory entityanalytics provider. 38645

  • Fix handling of un-parsed JSON in O365 module. 37800 38709

  • Fix filestream’s registry GC: registry entries are now removed from the in-memory and disk store when they’re older than the set TTL 36761 38488

  • Fix indexing failures by re-enabling event normalisation in netflow input. 38703 38780

  • Fix handling of truncated files in Filestream 38070 38416

  • Fix panic when more than 32767 pipeline clients are active. 38197 38556

  • Fix filestream’s registry GC: registry entries are now removed from the in-memory and disk store when they’re older than the set TTL 36761 38488

  • [threatintel] MISP splitting fix for empty responses 38739 38917

  • Fix a bug in cloudwatch task allocation that could skip some logs 38918 38953

  • Prevent GCP Pub/Sub input blockage by increasing default value of max_outstanding_messages 35029 38985

  • entity-analytics input: Improve structured logging. 38990

  • Fix config validation for CEL and HTTPJSON inputs when using password grant authentication and client.id or client.secret are not present. 38962

  • Updated Websocket input title to align with existing inputs 39006

  • Restore netflow input on Windows 39024

  • Upgrade azure-event-hubs-go and azure-storage-blob-go dependencies. 38861

  • Fix concurrency/error handling bugs in the AWS S3 input that could drop data and prevent ingestion of large buckets. 39131

  • Fix EntraID query handling. 39419 39420

  • Fix request trace filename handling in http_endpoint input. 39410

  • Fix filestream not correctly tracking the offset of a file when using the include_message parser. 39873 39653

  • Upgrade github.com/hashicorp/go-retryablehttp to mitigate CVE-2024-6104 40036

  • Fix for Google Workspace duplicate events issue by adding canonical sorting over fingerprint keys array to maintain key order. 40055 39859

  • Fix handling of deeply nested numeric values in HTTP Endpoint CEL programs. 40115

  • Prevent panic in CEL and salesforce inputs when github.com/hashicorp/go-retryablehttp exceeds maximum retries. 40144

  • Fix bug in CEL input rate limit logic. 40106 40270

  • Relax requirements in Okta entity analytics provider user and device profile data shape. 40359

  • Fix bug in Okta entity analytics rate limit logic. 40106 40267

Heartbeat

Metricbeat

  • Fix namespace filter option on metricset state_namespace enricher. 39934

  • Fix namespace filter option at Kubernetes provider level. 39881

  • Fix Azure Monitor 429 error by causing metricbeat to retry the request again. 38294

  • Fix fields not being parsed correctly in postgresql/database 25301 37720

  • rabbitmq/queue - Change the mapping type of rabbitmq.queue.consumers.utilisation.pct to scaled_float from long because the values fall within the range of [0.0, 1.0]. Previously, conversion to integer resulted in reporting either 0 or 1.

  • Fix timeout caused by the retrival of which indices are hidden 39165

  • Fix Azure Monitor support for multiple aggregation types 39192 39204

  • Fix handling of access errors when reading process metrics 39627

  • Fix behavior of cgroups path discovery when monitoring the host system from within a container 39627

  • Fix issue where beats may report incorrect metrics for its own process when running inside a container 39627

  • Fix for MySQL/Performance - Query failure for MySQL versions below v8.0.1, for performance metric quantile_95. 38710

  • Fix Prometheus helper text parser to store each metric family type. 39743

  • Normalize AWS RDS CPU Utilization values before making the metadata API call. 39664

  • Fix behavior of pagetypeinfo metrics 39985

  • Fix query logic for temp and non-temp tablespaces in Oracle module. 38051 39787

  • Set GCP metrics config period to the default (60s) when the value is below the minimum allowed period. 30434 40020

  • Fix statistic methods for metrics collected for SQS. 40207

  • Add GCP 'instance_id' resource label in ECS cloud fields. 40033 40062

  • Fix missing metrics from CloudWatch when include_linked_accounts set to false. 40071 40135

  • Update beat module with apm-server monitoring metrics fields 40127

  • Fix Azure Monitor metric timespan to restore Storage Account PT1H metrics 40376 40367

Osquerybeat

Packetbeat

Winlogbeat

Elastic Logging Plugin

Added

Affecting all Beats

  • Added append Processor which will append concrete values or values from a field to target. 29934 33364

  • dns processor: Add support for forward lookups (A, AAAA, and TXT). 11416 36394

  • [Enhanncement for host.ip and host.mac] Disabling netinfo.enabled option of add-host-metadata processor 36506

  • allow queue configuration settings to be set under the output. 35615 36788

  • Beats will now connect to older Elasticsearch instances by default 36884

  • Raise up logging level to warning when attempting to configure beats with unknown fields from autodiscovered events/environments

  • elasticsearch output now supports idle_connection_timeout. 35615 36843

  • Update to Go 1.21.12. 40114

  • Enable early event encoding in the Elasticsearch output, improving cpu and memory use 38572

  • The environment variable BEATS_ADD_CLOUD_METADATA_PROVIDERS overrides configured/default add_cloud_metadata providers 38669

  • Introduce log message for not supported annotations for Hints based autodiscover 38213

  • Add persistent volume claim name to volume if available 38839

  • Raw events are now logged to a different file, this prevents potentially sensitive information from leaking into log files 38767

  • Websocket input: Added runtime URL modification support based on state and cursor values 39858 39997

Auditbeat

  • Added add_session_metadata processor, which enables session viewer on Auditbeat data. 37640

  • Add linux capabilities to processes in the system/process. 37453

  • Add opt-in eBPF backend for file_integrity module. 37223

  • Add linux capabilities to processes in the system/process. 37453

  • Add opt-in eBPF backend for file_integrity module. 37223

  • Add process data to file events (Linux only, eBPF backend). 38199

  • Add container id to file events (Linux only, eBPF backend). 38328

  • Add procfs backend to the add_session_metadata processor. 38799

  • Add process.entity_id, process.group.name and process.group.id in add_process_metadata processor. Make fim module with kprobes backend to always add an appropriately configured add_process_metadata processor to enrich file events 38776

  • Reduce data size for add_session_metadata processor by removing unneeded fields 39500

  • Enrich process events with user and group names, with add_session_metadata processor 39537

Auditbeat

Auditbeat

Filebeat

  • add documentation for decode_xml_wineventlog processor field mappings. 32456

  • httpjson input: Add request tracing logger. 32402 32412

  • Add cloudflare R2 to provider list in AWS S3 input. 32620

  • Add support for single string containing multiple relation-types in getRFC5988Link. 32811

  • Added separation of transform context object inside httpjson. Introduced new clause .parent_last_response.* 33499

  • Added metric sqs_messages_waiting_gauge for aws-s3 input. 34488

  • Add nginx.ingress_controller.upstream.ip to related.ip 34645 34672

  • Add unix socket log parsing for nginx ingress_controller 34732

  • Added metric sqs_worker_utilization for aws-s3 input. 34793

  • Add MySQL authentication message parsing and related.ip and related.user fields 34810

  • Add nginx ingress_controller parsing if one of upstreams fails to return response 34787

  • Add oracle authentication messages parsing 35127

  • Add clean_session configuration setting for MQTT input. 16204

  • Add support for a simplified input configuraton when running under Elastic-Agent 36390

  • Added support for Okta OAuth2 provider in the CEL input. 36336 36521

  • Added support for new features & removed partial save mechanism in the Azure Blob Storage input. 35126 36690

  • Added support for new features and removed partial save mechanism in the GCS input. 35847 36713

  • Use filestream input with file_identity.fingerprint as default for hints autodiscover. 35984 36950

  • Add setup option --force-enable-module-filesets, that will act as if all filesets have been enabled in a module during setup. 30915 99999

  • Made Azure Blob Storage input GA and updated docs accordingly. 37128

  • Made GCS input GA and updated docs accordingly. 37127

  • Suppress and log max HTTP request retry errors in CEL input. 37160

  • Prevent CEL input from re-entering the eval loop when an evaluation failed. 37161

  • Update CEL extensions library to v1.7.0. 37172

  • Add support for complete URL replacement in HTTPJSON chain steps. 37486

  • Add support for user-defined query selection in EntraID entity analytics provider. 37653

  • Update CEL extensions library to v1.8.0 to provide runtime error location reporting. 37304 37718

  • Add request trace logging for chained API requests. 36551 37682

  • Relax TCP/UDP metric polling expectations to improve metric collection. 37714

  • Add support for PEM-based Okta auth in HTTPJSON. 37772

  • Prevent complete loss of long request trace data. 37826 37836

  • Added experimental version of the Websocket Input. 37774

  • Add support for PEM-based Okta auth in CEL. 37813

  • Add Salesforce input. 37331

  • Add ETW input. 36915

  • Update CEL mito extensions to v1.9.0 to add keys/values helper. 37971

  • Add logging for cache processor file reads and writes. 38052

  • Add parseDateInTZ value template for the HTTPJSON input 37738

  • Support VPC endpoint for aws-s3 input SQS queue url. 38189

  • Improve rate limit handling by HTTPJSON 36207 38161 38237

  • Add parseDateInTZ value template for the HTTPJSON input. 37738

  • Add support for complex event objects in the HTTP Endpoint input. 37910 38193

  • Parse more fields from Elasticsearch slowlogs 38295

  • Update CEL mito extensions to v1.10.0 to add base64 decode functions. 38504

  • Add support for Active Directory an entity analytics provider. 37919

  • Add AWS AWSHealth metricset. 38370

  • Add debugging breadcrumb to logs when writing request trace log. 38636

  • added benchmark input 37437

  • added benchmark input and discard output 37437

  • Ensure all responses sent by HTTP Endpoint are HTML-escaped. 39329

  • Update CEL mito extensions to v1.11.0 to improve type checking. 39460

  • Improve logging of request and response with request trace logging in error conditions. 39455

  • Implement Elastic Agent status and health reporting for CEL Filebeat input. 39209

  • Add HTTP metrics to CEL input. 39501 39503

  • Add default user-agent to CEL HTTP requests. 39502 39587

  • Improve reindexing support in security module pipelines. 38224 39588

  • Make HTTP Endpoint input GA. 38979 39410

  • Update CEL mito extensions to v1.12.2. 39755

  • Add support for base64-encoded HMAC headers to HTTP Endpoint. 39655

  • Add user group membership support to Okta entity analytics provider. 39814 39815

  • Add request trace support for Okta and EntraID entity analytics providers. 39821

  • Fix handling of infinite rate values in CEL rate limit handling logic. 39940

  • Allow elision of set and append failure logging. 34544 39929

  • Add ability to remove request trace logs from CEL input. 39969

  • Add ability to remove request trace logs from HTTPJSON input. 40003

  • Update CEL mito extensions to v1.13.0. 40035

  • Add Jamf entity analytics provider. 39996

  • Add ability to remove request trace logs from http_endpoint input. 40005

  • Add ability to remove request trace logs from entityanalytics input. 40004

  • Relax constraint on Base DN in entity analytics Active Directory provider. 40054

  • Implement Elastic Agent status and health reporting for Netflow Filebeat input. 40080

  • Enhance input state reporting for CEL evaluations that return a single error object in events. 40083

  • Allow absent credentials when using GCS with Application Default Credentials. 39977 40072

  • Add SSL and username support for Redis input, now the input includes support for Redis 6.0+. 40111

  • Add scaling up support for Netflow input. 37761 40122

  • Update CEL mito extensions to v1.15.0. 40294

  • Allow cross-region bucket configuration in s3 input. 22161 40309

  • Improve logging in Okta Entity Analytics provider. 40106 40347

Auditbeat

Libbeat

Heartbeat

  • Added status to monitor run log report.

  • Upgrade node to latest LTS v18.20.3. 40038

  • Add journey duration to synthetics browser events. 40230

Metricbeat

  • Add per-thread metrics to system_summary 33614

  • Add GCP CloudSQL metadata 33066

  • Add GCP Carbon Footprint metricbeat data 34820

  • Add event loop utilization metric to Kibana module 35020

  • Add metrics grouping by dimensions and time to Azure app insights 36634

  • Align on the algorithm used to transform Prometheus histograms into Elasticsearch histograms 36647

  • Add linux IO metrics to system/process 37213

  • Add new memory/cgroup metrics to Kibana module 37232

  • Support schema_name for MySQL performance metricset 38363

  • Add SSL support to mysql module 37997

  • Add SSL support for aerospike module 38126

  • Add last_terminated_timestamp metric in kubernetes module 39200 3802

  • Add pod.status.ready_time and pod.status.reason metrics in kubernetes module 39316

  • Add "Buffer cache hit ratio base" to calculate "Buffer cache hit ratio" for performance metrics 40022

Metricbeat

Osquerybeat

Packetbeat

Winlogbeat

  • Use fixed size buffer at first pass for event parsing, improving throughput 39530 39544

  • Add ERROR_INVALID_PARAMETER to the list of recoverable errors. 39781

Functionbeat

Elastic Log Driver Elastic Logging Plugin

Deprecated

Auditbeat

Filebeat

Heartbeat

Metricbeat

Osquerybeat

Packetbeat

Winlogbeat

Functionbeat

Elastic Logging Plugin

Known Issues