Merge branch 'Add_A11y_Tests_To_Upgrade_Assistant' of github.com:cuff…

…-links/kibana into Add_A11y_Tests_To_Upgrade_Assistant

docs/development/core/public/kibana-plugin-core-public.doclinksstart.links.md

@@ -21,6 +21,7 @@ readonly links: {
             readonly installation: string;
             readonly configuration: string;
             readonly elasticsearchOutput: string;
+            readonly elasticsearchModule: string;
             readonly startup: string;
             readonly exportedFields: string;
         };
@@ -29,6 +30,10 @@ readonly links: {
         };
         readonly metricbeat: {
             readonly base: string;
+            readonly configure: string;
+            readonly httpEndpoint: string;
+            readonly install: string;
+            readonly start: string;
         };
         readonly enterpriseSearch: {
             readonly base: string;

docs/maps/connect-to-ems.asciidoc

@@ -86,7 +86,7 @@ endif::[]
 
 [cols="2*<"]
 |===
-| [[ems-hostname]]`hostname`
+| [[ems-host]]`host`
  | Specifies the host of the backend server. To allow remote users to connect, set the value to the IP address or DNS name of the {hosted-ems} container. *Default: _your-hostname_*. <<server-host,Equivalent {kib} setting>>.
 
 | `port`
@@ -199,7 +199,7 @@ TIP: The available basemaps and boundaries can be explored from the `/maps` endp
 [[elastic-maps-server-kibana]]
 ==== Kibana configuration
 
-With {hosted-ems} running, add the `map.emsUrl` configuration key in your <<settings, kibana.yml>> file pointing to the root of the service. This setting will point {kib} to request EMS basemaps and boundaries from {hosted-ems}. Typically this will be the URL to the <<ems-hostname,hostname and port>> of {hosted-ems}. For example, `map.emsUrl: https://my-ems-server:8080`. 
+With {hosted-ems} running, add the `map.emsUrl` configuration key in your <<settings, kibana.yml>> file pointing to the root of the service. This setting will point {kib} to request EMS basemaps and boundaries from {hosted-ems}. Typically this will be the URL to the <<ems-host,host and port>> of {hosted-ems}. For example, `map.emsUrl: https://my-ems-server:8080`. 
 
 
 [float]

docs/maps/maps-aggregations.asciidoc

@@ -68,9 +68,9 @@ To enable a blended layer that dynamically shows clusters or documents:
 
 [role="xpack"]
 [[maps-top-hits-aggregation]]
-=== Top hits per entity
+=== Display the most relevant documents per entity
 
-You can display the most relevant documents per entity, for example, the most recent GPS tracks per flight.
+Use *Top hits per entity* to display the most relevant documents per entity, for example, the most recent GPS tracks per flight route.
 To get this data, {es} first groups your data using a {ref}/search-aggregations-bucket-terms-aggregation.html[terms aggregation],
 then accumulates the most relevant documents based on sort order for each entry using a {ref}/search-aggregations-metrics-top-hits-aggregation.html[top hits metric aggregation].
 

docs/migration/migrate_8_0.asciidoc

@@ -58,7 +58,16 @@ for example, `logstash-*`.
 ==== Responses are never logged by default
 *Details:* Previously responses would be logged if either `logging.json` was true, `logging.dest` was specified, or a `TTY` was detected.
 
-*Impact:* To restore the previous behavior, in kibana.yml set `logging.events.response=*`.
+*Impact:* To restore the previous behavior, in kibana.yml enable `debug` logs for the `http.server.response` context under `logging.loggers`:
+[source,yaml]
+-------------------
+logging:
+  loggers:
+    - context: http.server.response
+      appenders: [console]
+      level: debug
+-------------------
+See https://github.com/elastic/kibana/pull/87939 for more details.
 
 [float]
 ==== `xpack.security.authProviders` is no longer valid

docs/setup/install/deb.asciidoc

@@ -8,12 +8,8 @@ The Debian package for Kibana can be <<install-deb,downloaded from our website>>
 or from our <<deb-repo,APT repository>>. It can be used to install
 Kibana on any Debian-based system such as Debian and Ubuntu.
 
-This package is free to use under the Elastic license. It contains open source
-and free commercial features and access to paid commercial features.
-<<managing-licenses,Start a 30-day trial>> to try out all of the
-paid commercial features. See the
-https://www.elastic.co/subscriptions[Subscriptions] page for information about
-Elastic license levels.
+This package contains both free and subscription features.
+<<managing-licenses,Start a 30-day trial>> to try out all of the features.
 
 The latest stable version of Kibana can be found on the
 link:/downloads/kibana[Download Kibana] page. Other versions can

docs/setup/install/rpm.asciidoc

@@ -13,12 +13,8 @@ and Oracle Enterprise.
 NOTE: RPM install is not supported on distributions with old versions of RPM,
 such as SLES 11 and CentOS 5.  Please see <<targz>> instead.
 
-This package is free to use under the Elastic license. It contains open source
-and free commercial features and access to paid commercial features.
-<<managing-licenses,Start a 30-day trial>> to try out all of the
-paid commercial features. See the
-https://www.elastic.co/subscriptions[Subscriptions] page for information about
-Elastic license levels.
+This package contains both free and subscription features.
+<<managing-licenses,Start a 30-day trial>> to try out all of the features.
 
 The latest stable version of Kibana can be found on the
 link:/downloads/kibana[Download Kibana] page. Other versions can

docs/setup/install/targz.asciidoc

@@ -7,12 +7,8 @@
 Kibana is provided for Linux and Darwin as a `.tar.gz` package. These packages
 are the easiest formats to use when trying out Kibana.
 
-These packages are free to use under the Elastic license. They contain open
-source and free commercial features and access to paid commercial features.
-<<managing-licenses,Start a 30-day trial>> to try out all of the
-paid commercial features. See the
-https://www.elastic.co/subscriptions[Subscriptions] page for information about
-Elastic license levels.
+This package contains both free and subscription features.
+<<managing-licenses,Start a 30-day trial>> to try out all of the features.
 
 The latest stable version of Kibana can be found on the
 link:/downloads/kibana[Download Kibana] page.

docs/setup/install/windows.asciidoc

@@ -6,12 +6,8 @@
 
 Kibana can be installed on Windows using the `.zip` package.
 
-This package is free to use under the Elastic license. It contains open source
-and free commercial features and access to paid commercial features.
-<<managing-licenses,Start a 30-day trial>> to try out all of the
-paid commercial features. See the
-https://www.elastic.co/subscriptions[Subscriptions] page for information about
-Elastic license levels.
+This package contains both free and subscription features.
+<<managing-licenses,Start a 30-day trial>> to try out all of the features.
 
 The latest stable version of Kibana can be found on the
 link:/downloads/kibana[Download Kibana] page.

packages/kbn-legacy-logging/src/get_logging_config.ts

@@ -27,14 +27,14 @@ export function getLoggingConfiguration(config: LegacyLoggingConfig, opsInterval
     });
   } else if (config.verbose) {
     _.defaults(events, {
+      error: '*',
       log: '*',
-      // To avoid duplicate logs, we explicitly disable this in verbose
-      // mode as it is already provided by the new logging config under
-      // the `metrics.ops` context.
+      // To avoid duplicate logs, we explicitly disable these in verbose
+      // mode as they are already provided by the new logging config under
+      // the `http.server.response` and `metrics.ops` contexts.
       ops: '!',
-      request: '*',
-      response: '*',
-      error: '*',
+      request: '!',
+      response: '!',
     });
   } else {
     _.defaults(events, {
@@ -75,6 +75,7 @@ export function getLoggingConfiguration(config: LegacyLoggingConfig, opsInterval
     },
     includes: {
       request: ['headers', 'payload'],
+      response: ['headers', 'payload'],
     },
     reporters: {
       logReporter: [loggerStream],

packages/kbn-legacy-logging/src/log_events.ts

@@ -6,6 +6,7 @@
  * Side Public License, v 1.
  */
 
+import type { ResponseObject } from '@hapi/hapi';
 import { EventData, isEventData } from './metadata';
 
 export interface BaseEvent {
@@ -21,7 +22,8 @@ export interface ResponseEvent extends BaseEvent {
   statusCode: number;
   path: string;
   headers: Record<string, string | string[]>;
-  responsePayload: string;
+  responseHeaders: Record<string, string | string[]>;
+  responsePayload: ResponseObject['source'];
   responseTime: string;
   query: Record<string, any>;
 }

packages/kbn-legacy-logging/src/log_format.ts

@@ -12,15 +12,14 @@ import _ from 'lodash';
 import queryString from 'query-string';
 import numeral from '@elastic/numeral';
 import chalk from 'chalk';
-// @ts-expect-error missing type def
-import stringify from 'json-stringify-safe';
 import { inspect } from 'util';
 
-import { applyFiltersToKeys } from './utils';
+import { applyFiltersToKeys, getResponsePayloadBytes } from './utils';
 import { getLogEventData } from './metadata';
 import { LegacyLoggingConfig } from './schema';
 import {
   AnyEvent,
+  ResponseEvent,
   isResponseEvent,
   isOpsEvent,
   isErrorEvent,
@@ -70,6 +69,23 @@ export abstract class BaseLogFormat extends Stream.Transform {
     next();
   }
 
+  getContentLength({ responsePayload, responseHeaders }: ResponseEvent): number | undefined {
+    try {
+      return getResponsePayloadBytes(responsePayload, responseHeaders);
+    } catch (e) {
+      // We intentionally swallow any errors as this information is
+      // only a nicety for logging purposes, and should not cause the
+      // server to crash if it cannot be determined.
+      this.push(
+        this.format({
+          type: 'log',
+          tags: ['warning', 'logging'],
+          message: `Failed to calculate response payload bytes. [${e}]`,
+        }) + '\n'
+      );
+    }
+  }
+
   extractAndFormatTimestamp(data: Record<string, any>, format?: string) {
     const { timezone } = this.config;
     const date = moment(data['@timestamp']);
@@ -100,15 +116,10 @@ export abstract class BaseLogFormat extends Stream.Transform {
         referer: source.referer,
       };
 
-      const contentLength =
-        event.responsePayload === 'object'
-          ? stringify(event.responsePayload).length
-          : String(event.responsePayload).length;
-
       data.res = {
         statusCode: event.statusCode,
         responseTime: event.responseTime,
-        contentLength,
+        contentLength: this.getContentLength(event),
       };
 
       const query = queryString.stringify(event.query, { sort: false });
@@ -122,7 +133,9 @@ export abstract class BaseLogFormat extends Stream.Transform {
       data.message += levelColor(data.res.statusCode);
       data.message += ' ';
       data.message += chalk.gray(data.res.responseTime + 'ms');
-      data.message += chalk.gray(' - ' + numeral(contentLength).format('0.0b'));
+      if (data.res.contentLength) {
+        data.message += chalk.gray(' - ' + numeral(data.res.contentLength).format('0.0b'));
+      }
     } else if (isOpsEvent(event)) {
       _.defaults(data, _.pick(event, ['pid', 'os', 'proc', 'load']));
       data.message = chalk.gray('memory: ');

packages/kbn-legacy-logging/src/log_format_json.test.ts

@@ -39,30 +39,45 @@ describe('KbnLoggerJsonFormat', () => {
       expect(message).toBe('undefined');
     });
 
-    it('response', async () => {
-      const event = {
-        ...makeEvent('response'),
-        statusCode: 200,
-        contentLength: 800,
-        responseTime: 12000,
-        method: 'GET',
-        path: '/path/to/resource',
-        responsePayload: '1234567879890',
-        source: {
-          remoteAddress: '127.0.0.1',
-          userAgent: 'Test Thing',
-          referer: 'elastic.co',
-        },
-      };
-      const result = await createPromiseFromStreams<string>([createListStream([event]), format]);
-      const { type, method, statusCode, message, req } = JSON.parse(result);
-
-      expect(type).toBe('response');
-      expect(method).toBe('GET');
-      expect(statusCode).toBe(200);
-      expect(message).toBe('GET /path/to/resource 200 12000ms - 13.0B');
-      expect(req.remoteAddress).toBe('127.0.0.1');
-      expect(req.userAgent).toBe('Test Thing');
+    describe('response', () => {
+      it('handles a response object', async () => {
+        const event = {
+          ...makeEvent('response'),
+          statusCode: 200,
+          contentLength: 800,
+          responseTime: 12000,
+          method: 'GET',
+          path: '/path/to/resource',
+          responsePayload: '1234567879890',
+          source: {
+            remoteAddress: '127.0.0.1',
+            userAgent: 'Test Thing',
+            referer: 'elastic.co',
+          },
+        };
+        const result = await createPromiseFromStreams<string>([createListStream([event]), format]);
+        const { type, method, statusCode, message, req } = JSON.parse(result);
+
+        expect(type).toBe('response');
+        expect(method).toBe('GET');
+        expect(statusCode).toBe(200);
+        expect(message).toBe('GET /path/to/resource 200 12000ms - 13.0B');
+        expect(req.remoteAddress).toBe('127.0.0.1');
+        expect(req.userAgent).toBe('Test Thing');
+      });
+
+      it('leaves payload size empty if not available', async () => {
+        const event = {
+          ...makeEvent('response'),
+          statusCode: 200,
+          responseTime: 12000,
+          method: 'GET',
+          path: '/path/to/resource',
+          responsePayload: null,
+        };
+        const result = await createPromiseFromStreams<string>([createListStream([event]), format]);
+        expect(JSON.parse(result).message).toBe('GET /path/to/resource 200 12000ms');
+      });
     });
 
     it('ops', async () => {