diff --git a/docs/apm/images/apm-services-overview.png b/docs/apm/images/apm-services-overview.png index 85b441d47f0c2f..3a56d597abfb79 100644 Binary files a/docs/apm/images/apm-services-overview.png and b/docs/apm/images/apm-services-overview.png differ diff --git a/docs/apm/images/apm-traces.png b/docs/apm/images/apm-traces.png index 97e801606c613d..ed15423b42c511 100644 Binary files a/docs/apm/images/apm-traces.png and b/docs/apm/images/apm-traces.png differ diff --git a/docs/apm/images/apm-transactions-overview.png b/docs/apm/images/apm-transactions-overview.png index 80fca19ff96cc0..1b25668f0fd92a 100644 Binary files a/docs/apm/images/apm-transactions-overview.png and b/docs/apm/images/apm-transactions-overview.png differ diff --git a/docs/apm/images/traffic-transactions.png b/docs/apm/images/traffic-transactions.png index 134bc0e6bcb42c..ef429740ceee31 100644 Binary files a/docs/apm/images/traffic-transactions.png and b/docs/apm/images/traffic-transactions.png differ diff --git a/docs/apm/service-overview.asciidoc b/docs/apm/service-overview.asciidoc index 088791e6098e6d..5fd214e6ce613a 100644 --- a/docs/apm/service-overview.asciidoc +++ b/docs/apm/service-overview.asciidoc @@ -17,8 +17,8 @@ Response times for the service. You can filter the *Latency* chart to display th image::apm/images/latency.png[Service latency] [discrete] -[[service-traffic-transactions]] -=== Traffic and transactions +[[service-throughput-transactions]] +=== Throughput and transactions The *Throughput* chart visualizes the average number of transactions per minute for the selected service. @@ -62,6 +62,9 @@ each dependency. By default, dependencies are sorted by _Impact_ to show the mos If there is a particular dependency you are interested in, click *View service map* to view the related <>. +IMPORTANT: A known issue prevents Real User Monitoring (RUM) dependencies from being shown in the +*Dependencies* table. We are working on a fix for this issue. + [role="screenshot"] image::apm/images/spans-dependencies.png[Span type duration and dependencies] diff --git a/docs/apm/transactions.asciidoc b/docs/apm/transactions.asciidoc index 83ca9e5a10a9b0..8c8da81aa577e1 100644 --- a/docs/apm/transactions.asciidoc +++ b/docs/apm/transactions.asciidoc @@ -17,10 +17,10 @@ If there's a weird spike that you'd like to investigate, you can simply zoom in on the graph - this will adjust the specific time range, and all of the data on the page will update accordingly. -*Transactions per minute*:: -Visualize response codes: `2xx`, `3xx`, `4xx`, etc., -and is useful for determining if you're serving more of one code than you typically do. -Like in the Transaction duration graph, you can zoom in on anomalies to further investigate them. +*Throughput*:: +Visualize response codes: `2xx`, `3xx`, `4xx`, etc. +Useful for determining if more responses than usual are being served with a particular response code. +Like in the latency graph, you can zoom in on anomalies to further investigate them. *Error rate*:: Visualize the total number of transactions with errors divided by the total number of transactions. @@ -157,4 +157,4 @@ and solve problems. [role="screenshot"] image::apm/images/apm-logs-tab.png[APM logs tab] -// To do: link to log correlation \ No newline at end of file +// To do: link to log correlation diff --git a/docs/apm/troubleshooting.asciidoc b/docs/apm/troubleshooting.asciidoc index 7084777cbb6f95..465a3d652046db 100644 --- a/docs/apm/troubleshooting.asciidoc +++ b/docs/apm/troubleshooting.asciidoc @@ -1,14 +1,30 @@ [[troubleshooting]] -== Troubleshoot common problems +== Troubleshooting ++++ Troubleshooting ++++ -If you have something to add to this section, please consider creating a pull request with -your proposed changes at https://github.com/elastic/kibana. - -Also, check out the https://discuss.elastic.co/c/apm[APM discussion forum]. +This section describes common problems you might encounter with the APM app. +To add to this page, please consider opening a +https://github.com/elastic/kibana/pulls[pull request] with your proposed changes. + +If your issue is potentially related to other components of the APM ecosystem, +don't forget to check our other troubleshooting guides or discussion forum: + +* {apm-server-ref}/troubleshooting.html[APM Server troubleshooting] +* {apm-dotnet-ref}/troubleshooting.html[.NET agent troubleshooting] +* {apm-go-ref}/troubleshooting.html[Go agent troubleshooting] +* {apm-java-ref}/trouble-shooting.html[Java agent troubleshooting] +* {apm-node-ref}/troubleshooting.html[Node.js agent troubleshooting] +* {apm-py-ref}/troubleshooting.html[Python agent troubleshooting] +* {apm-ruby-ref}/debugging.html[Ruby agent troubleshooting] +* {apm-rum-ref/troubleshooting.html[RUM troubleshooting] +* https://discuss.elastic.co/c/apm[APM discussion forum]. + +[discrete] +[[troubleshooting-apm-app]] +== Troubleshoot common APM app problems * <> * <> diff --git a/docs/development/core/server/kibana-plugin-core-server.savedobjectserrorhelpers.createindexaliasnotfounderror.md b/docs/development/core/server/kibana-plugin-core-server.savedobjectserrorhelpers.createindexaliasnotfounderror.md deleted file mode 100644 index 2b897db7bba4c3..00000000000000 --- a/docs/development/core/server/kibana-plugin-core-server.savedobjectserrorhelpers.createindexaliasnotfounderror.md +++ /dev/null @@ -1,22 +0,0 @@ - - -[Home](./index.md) > [kibana-plugin-core-server](./kibana-plugin-core-server.md) > [SavedObjectsErrorHelpers](./kibana-plugin-core-server.savedobjectserrorhelpers.md) > [createIndexAliasNotFoundError](./kibana-plugin-core-server.savedobjectserrorhelpers.createindexaliasnotfounderror.md) - -## SavedObjectsErrorHelpers.createIndexAliasNotFoundError() method - -Signature: - -```typescript -static createIndexAliasNotFoundError(alias: string): DecoratedError; -``` - -## Parameters - -| Parameter | Type | Description | -| --- | --- | --- | -| alias | string | | - -Returns: - -`DecoratedError` - diff --git a/docs/development/core/server/kibana-plugin-core-server.savedobjectserrorhelpers.decorateindexaliasnotfounderror.md b/docs/development/core/server/kibana-plugin-core-server.savedobjectserrorhelpers.decorateindexaliasnotfounderror.md deleted file mode 100644 index c7e10fc42ead19..00000000000000 --- a/docs/development/core/server/kibana-plugin-core-server.savedobjectserrorhelpers.decorateindexaliasnotfounderror.md +++ /dev/null @@ -1,23 +0,0 @@ - - -[Home](./index.md) > [kibana-plugin-core-server](./kibana-plugin-core-server.md) > [SavedObjectsErrorHelpers](./kibana-plugin-core-server.savedobjectserrorhelpers.md) > [decorateIndexAliasNotFoundError](./kibana-plugin-core-server.savedobjectserrorhelpers.decorateindexaliasnotfounderror.md) - -## SavedObjectsErrorHelpers.decorateIndexAliasNotFoundError() method - -Signature: - -```typescript -static decorateIndexAliasNotFoundError(error: Error, alias: string): DecoratedError; -``` - -## Parameters - -| Parameter | Type | Description | -| --- | --- | --- | -| error | Error | | -| alias | string | | - -Returns: - -`DecoratedError` - diff --git a/docs/development/core/server/kibana-plugin-core-server.savedobjectserrorhelpers.isgeneralerror.md b/docs/development/core/server/kibana-plugin-core-server.savedobjectserrorhelpers.isgeneralerror.md deleted file mode 100644 index 4b4ede2f77a7e2..00000000000000 --- a/docs/development/core/server/kibana-plugin-core-server.savedobjectserrorhelpers.isgeneralerror.md +++ /dev/null @@ -1,22 +0,0 @@ - - -[Home](./index.md) > [kibana-plugin-core-server](./kibana-plugin-core-server.md) > [SavedObjectsErrorHelpers](./kibana-plugin-core-server.savedobjectserrorhelpers.md) > [isGeneralError](./kibana-plugin-core-server.savedobjectserrorhelpers.isgeneralerror.md) - -## SavedObjectsErrorHelpers.isGeneralError() method - -Signature: - -```typescript -static isGeneralError(error: Error | DecoratedError): boolean; -``` - -## Parameters - -| Parameter | Type | Description | -| --- | --- | --- | -| error | Error | DecoratedError | | - -Returns: - -`boolean` - diff --git a/docs/development/core/server/kibana-plugin-core-server.savedobjectserrorhelpers.md b/docs/development/core/server/kibana-plugin-core-server.savedobjectserrorhelpers.md index 2dc78f2df3a833..9b69012ed5f123 100644 --- a/docs/development/core/server/kibana-plugin-core-server.savedobjectserrorhelpers.md +++ b/docs/development/core/server/kibana-plugin-core-server.savedobjectserrorhelpers.md @@ -18,7 +18,6 @@ export declare class SavedObjectsErrorHelpers | [createBadRequestError(reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.createbadrequesterror.md) | static | | | [createConflictError(type, id, reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.createconflicterror.md) | static | | | [createGenericNotFoundError(type, id)](./kibana-plugin-core-server.savedobjectserrorhelpers.creategenericnotfounderror.md) | static | | -| [createIndexAliasNotFoundError(alias)](./kibana-plugin-core-server.savedobjectserrorhelpers.createindexaliasnotfounderror.md) | static | | | [createInvalidVersionError(versionInput)](./kibana-plugin-core-server.savedobjectserrorhelpers.createinvalidversionerror.md) | static | | | [createTooManyRequestsError(type, id)](./kibana-plugin-core-server.savedobjectserrorhelpers.createtoomanyrequestserror.md) | static | | | [createUnsupportedTypeError(type)](./kibana-plugin-core-server.savedobjectserrorhelpers.createunsupportedtypeerror.md) | static | | @@ -28,7 +27,6 @@ export declare class SavedObjectsErrorHelpers | [decorateEsUnavailableError(error, reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.decorateesunavailableerror.md) | static | | | [decorateForbiddenError(error, reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.decorateforbiddenerror.md) | static | | | [decorateGeneralError(error, reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.decorategeneralerror.md) | static | | -| [decorateIndexAliasNotFoundError(error, alias)](./kibana-plugin-core-server.savedobjectserrorhelpers.decorateindexaliasnotfounderror.md) | static | | | [decorateNotAuthorizedError(error, reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.decoratenotauthorizederror.md) | static | | | [decorateRequestEntityTooLargeError(error, reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.decoraterequestentitytoolargeerror.md) | static | | | [decorateTooManyRequestsError(error, reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.decoratetoomanyrequestserror.md) | static | | @@ -37,7 +35,6 @@ export declare class SavedObjectsErrorHelpers | [isEsCannotExecuteScriptError(error)](./kibana-plugin-core-server.savedobjectserrorhelpers.isescannotexecutescripterror.md) | static | | | [isEsUnavailableError(error)](./kibana-plugin-core-server.savedobjectserrorhelpers.isesunavailableerror.md) | static | | | [isForbiddenError(error)](./kibana-plugin-core-server.savedobjectserrorhelpers.isforbiddenerror.md) | static | | -| [isGeneralError(error)](./kibana-plugin-core-server.savedobjectserrorhelpers.isgeneralerror.md) | static | | | [isInvalidVersionError(error)](./kibana-plugin-core-server.savedobjectserrorhelpers.isinvalidversionerror.md) | static | | | [isNotAuthorizedError(error)](./kibana-plugin-core-server.savedobjectserrorhelpers.isnotauthorizederror.md) | static | | | [isNotFoundError(error)](./kibana-plugin-core-server.savedobjectserrorhelpers.isnotfounderror.md) | static | | diff --git a/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md b/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md new file mode 100644 index 00000000000000..937e20a7a95795 --- /dev/null +++ b/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md @@ -0,0 +1,13 @@ + + +[Home](./index.md) > [kibana-plugin-plugins-data-public](./kibana-plugin-plugins-data-public.md) > [ISearchOptions](./kibana-plugin-plugins-data-public.isearchoptions.md) > [legacyHitsTotal](./kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md) + +## ISearchOptions.legacyHitsTotal property + +Request the legacy format for the total number of hits. If sending `rest_total_hits_as_int` to something other than `true`, this should be set to `false`. + +Signature: + +```typescript +legacyHitsTotal?: boolean; +``` diff --git a/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.md b/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.md index 5acd837495dac4..fc2767cd0231f9 100644 --- a/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.md +++ b/docs/development/plugins/data/public/kibana-plugin-plugins-data-public.isearchoptions.md @@ -17,6 +17,7 @@ export interface ISearchOptions | [abortSignal](./kibana-plugin-plugins-data-public.isearchoptions.abortsignal.md) | AbortSignal | An AbortSignal that allows the caller of search to abort a search request. | | [isRestore](./kibana-plugin-plugins-data-public.isearchoptions.isrestore.md) | boolean | Whether the session is restored (i.e. search requests should re-use the stored search IDs, rather than starting from scratch) | | [isStored](./kibana-plugin-plugins-data-public.isearchoptions.isstored.md) | boolean | Whether the session is already saved (i.e. sent to background) | +| [legacyHitsTotal](./kibana-plugin-plugins-data-public.isearchoptions.legacyhitstotal.md) | boolean | Request the legacy format for the total number of hits. If sending rest_total_hits_as_int to something other than true, this should be set to false. | | [sessionId](./kibana-plugin-plugins-data-public.isearchoptions.sessionid.md) | string | A session ID, grouping multiple search requests into a single session. | | [strategy](./kibana-plugin-plugins-data-public.isearchoptions.strategy.md) | string | Use this option to force using a specific server side search strategy. Leave empty to use the default strategy. | diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.md new file mode 100644 index 00000000000000..eaac671b9a1826 --- /dev/null +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.md @@ -0,0 +1,18 @@ + + +[Home](./index.md) > [kibana-plugin-plugins-data-server](./kibana-plugin-plugins-data-server.md) > [IScopedSessionService](./kibana-plugin-plugins-data-server.iscopedsessionservice.md) + +## IScopedSessionService interface + +Signature: + +```typescript +export interface IScopedSessionService +``` + +## Properties + +| Property | Type | Description | +| --- | --- | --- | +| [search](./kibana-plugin-plugins-data-server.iscopedsessionservice.search.md) | <Request extends IKibanaSearchRequest, Response extends IKibanaSearchResponse>(strategy: ISearchStrategy<Request, Response>, ...args: Parameters<ISearchStrategy<Request, Response>['search']>) => Observable<Response> | | + diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.search.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.search.md new file mode 100644 index 00000000000000..d58a9fd9f37619 --- /dev/null +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.iscopedsessionservice.search.md @@ -0,0 +1,11 @@ + + +[Home](./index.md) > [kibana-plugin-plugins-data-server](./kibana-plugin-plugins-data-server.md) > [IScopedSessionService](./kibana-plugin-plugins-data-server.iscopedsessionservice.md) > [search](./kibana-plugin-plugins-data-server.iscopedsessionservice.search.md) + +## IScopedSessionService.search property + +Signature: + +```typescript +search: (strategy: ISearchStrategy, ...args: Parameters['search']>) => Observable; +``` diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md new file mode 100644 index 00000000000000..59b8b2c6b446f6 --- /dev/null +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md @@ -0,0 +1,13 @@ + + +[Home](./index.md) > [kibana-plugin-plugins-data-server](./kibana-plugin-plugins-data-server.md) > [ISearchOptions](./kibana-plugin-plugins-data-server.isearchoptions.md) > [legacyHitsTotal](./kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md) + +## ISearchOptions.legacyHitsTotal property + +Request the legacy format for the total number of hits. If sending `rest_total_hits_as_int` to something other than `true`, this should be set to `false`. + +Signature: + +```typescript +legacyHitsTotal?: boolean; +``` diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.md index 85847e1c61d25b..9de351b2b90194 100644 --- a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.md +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.isearchoptions.md @@ -17,6 +17,7 @@ export interface ISearchOptions | [abortSignal](./kibana-plugin-plugins-data-server.isearchoptions.abortsignal.md) | AbortSignal | An AbortSignal that allows the caller of search to abort a search request. | | [isRestore](./kibana-plugin-plugins-data-server.isearchoptions.isrestore.md) | boolean | Whether the session is restored (i.e. search requests should re-use the stored search IDs, rather than starting from scratch) | | [isStored](./kibana-plugin-plugins-data-server.isearchoptions.isstored.md) | boolean | Whether the session is already saved (i.e. sent to background) | +| [legacyHitsTotal](./kibana-plugin-plugins-data-server.isearchoptions.legacyhitstotal.md) | boolean | Request the legacy format for the total number of hits. If sending rest_total_hits_as_int to something other than true, this should be set to false. | | [sessionId](./kibana-plugin-plugins-data-server.isearchoptions.sessionid.md) | string | A session ID, grouping multiple search requests into a single session. | | [strategy](./kibana-plugin-plugins-data-server.isearchoptions.strategy.md) | string | Use this option to force using a specific server side search strategy. Leave empty to use the default strategy. | diff --git a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.md b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.md index 84c7875c26ce89..27a386a714fc1a 100644 --- a/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.md +++ b/docs/development/plugins/data/server/kibana-plugin-plugins-data-server.md @@ -53,6 +53,7 @@ | [IFieldSubType](./kibana-plugin-plugins-data-server.ifieldsubtype.md) | | | [IFieldType](./kibana-plugin-plugins-data-server.ifieldtype.md) | | | [IndexPatternAttributes](./kibana-plugin-plugins-data-server.indexpatternattributes.md) | Interface for an index pattern saved object | +| [IScopedSessionService](./kibana-plugin-plugins-data-server.iscopedsessionservice.md) | | | [ISearchOptions](./kibana-plugin-plugins-data-server.isearchoptions.md) | | | [ISearchSetup](./kibana-plugin-plugins-data-server.isearchsetup.md) | | | [ISearchStart](./kibana-plugin-plugins-data-server.isearchstart.md) | | diff --git a/docs/maps/maps-getting-started.asciidoc b/docs/maps/maps-getting-started.asciidoc index 32a81c8e65f565..609a133c92ad1c 100644 --- a/docs/maps/maps-getting-started.asciidoc +++ b/docs/maps/maps-getting-started.asciidoc @@ -1,15 +1,11 @@ [role="xpack"] [[maps-getting-started]] -== Create a map with multiple layers and data sources - -++++ -Create a multilayer map -++++ +== Build a map to compare metrics by country or region If you are new to **Maps**, this tutorial is a good place to start. It guides you through the common steps for working with your location data. -You'll learn to: +You will learn to: - Create a map with multiple layers and data sources - Use symbols, colors, and labels to style data values diff --git a/packages/kbn-es-archiver/src/actions/empty_kibana_index.ts b/packages/kbn-es-archiver/src/actions/empty_kibana_index.ts index 6272d6ba00ee85..56c75c5aca4193 100644 --- a/packages/kbn-es-archiver/src/actions/empty_kibana_index.ts +++ b/packages/kbn-es-archiver/src/actions/empty_kibana_index.ts @@ -25,6 +25,5 @@ export async function emptyKibanaIndexAction({ await cleanKibanaIndices({ client, stats, log, kibanaPluginIds }); await migrateKibanaIndex({ client, kbnClient }); - stats.createdIndex('.kibana'); - return stats.toJSON(); + return stats; } diff --git a/packages/kbn-es-archiver/src/actions/load.ts b/packages/kbn-es-archiver/src/actions/load.ts index 8afd7d79a98f73..c32496ad426945 100644 --- a/packages/kbn-es-archiver/src/actions/load.ts +++ b/packages/kbn-es-archiver/src/actions/load.ts @@ -98,9 +98,11 @@ export async function loadAction({ // If we affected the Kibana index, we need to ensure it's migrated... if (Object.keys(result).some((k) => k.startsWith('.kibana'))) { await migrateKibanaIndex({ client, kbnClient }); + log.debug('[%s] Migrated Kibana index after loading Kibana data', name); if (kibanaPluginIds.includes('spaces')) { await createDefaultSpace({ client, index: '.kibana' }); + log.debug('[%s] Ensured that default space exists in .kibana', name); } } diff --git a/packages/kbn-es-archiver/src/es_archiver.ts b/packages/kbn-es-archiver/src/es_archiver.ts index 8601dedad0e277..f101c5d6867f10 100644 --- a/packages/kbn-es-archiver/src/es_archiver.ts +++ b/packages/kbn-es-archiver/src/es_archiver.ts @@ -155,7 +155,7 @@ export class EsArchiver { * @return Promise */ async emptyKibanaIndex() { - return await emptyKibanaIndexAction({ + await emptyKibanaIndexAction({ client: this.client, log: this.log, kbnClient: this.kbnClient, diff --git a/packages/kbn-es-archiver/src/lib/indices/kibana_index.ts b/packages/kbn-es-archiver/src/lib/indices/kibana_index.ts index 91c0bd8343a360..0459a4301cf6bf 100644 --- a/packages/kbn-es-archiver/src/lib/indices/kibana_index.ts +++ b/packages/kbn-es-archiver/src/lib/indices/kibana_index.ts @@ -76,9 +76,7 @@ export async function migrateKibanaIndex({ */ async function fetchKibanaIndices(client: Client) { const kibanaIndices = await client.cat.indices({ index: '.kibana*', format: 'json' }); - const isKibanaIndex = (index: string) => - /^\.kibana(:?_\d*)?$/.test(index) || - /^\.kibana(_task_manager)?_(pre)?\d+\.\d+\.\d+/.test(index); + const isKibanaIndex = (index: string) => /^\.kibana(:?_\d*)?$/.test(index); return kibanaIndices.map((x: { index: string }) => x.index).filter(isKibanaIndex); } @@ -105,7 +103,7 @@ export async function cleanKibanaIndices({ while (true) { const resp = await client.deleteByQuery({ - index: `.kibana,.kibana_task_manager`, + index: `.kibana`, body: { query: { bool: { @@ -117,7 +115,7 @@ export async function cleanKibanaIndices({ }, }, }, - ignore: [404, 409], + ignore: [409], }); if (resp.total !== resp.deleted) { diff --git a/packages/kbn-optimizer/limits.yml b/packages/kbn-optimizer/limits.yml index 794503656ba048..74ee3f7c46e1b1 100644 --- a/packages/kbn-optimizer/limits.yml +++ b/packages/kbn-optimizer/limits.yml @@ -78,7 +78,7 @@ pageLoadAssetSize: tileMap: 65337 timelion: 29920 transform: 41007 - triggersActionsUi: 170001 + triggersActionsUi: 186732 uiActions: 97717 uiActionsEnhanced: 313011 upgradeAssistant: 81241 diff --git a/packages/kbn-pm/dist/index.js b/packages/kbn-pm/dist/index.js index df04965cd8c32b..e8c6fa4d5a0131 100644 --- a/packages/kbn-pm/dist/index.js +++ b/packages/kbn-pm/dist/index.js @@ -47961,11 +47961,13 @@ __webpack_require__.r(__webpack_exports__); "use strict"; __webpack_require__.r(__webpack_exports__); /* harmony export (binding) */ __webpack_require__.d(__webpack_exports__, "installBazelTools", function() { return installBazelTools; }); -/* harmony import */ var path__WEBPACK_IMPORTED_MODULE_0__ = __webpack_require__(4); -/* harmony import */ var path__WEBPACK_IMPORTED_MODULE_0___default = /*#__PURE__*/__webpack_require__.n(path__WEBPACK_IMPORTED_MODULE_0__); -/* harmony import */ var _child_process__WEBPACK_IMPORTED_MODULE_1__ = __webpack_require__(319); -/* harmony import */ var _fs__WEBPACK_IMPORTED_MODULE_2__ = __webpack_require__(131); -/* harmony import */ var _log__WEBPACK_IMPORTED_MODULE_3__ = __webpack_require__(246); +/* harmony import */ var dedent__WEBPACK_IMPORTED_MODULE_0__ = __webpack_require__(2); +/* harmony import */ var dedent__WEBPACK_IMPORTED_MODULE_0___default = /*#__PURE__*/__webpack_require__.n(dedent__WEBPACK_IMPORTED_MODULE_0__); +/* harmony import */ var path__WEBPACK_IMPORTED_MODULE_1__ = __webpack_require__(4); +/* harmony import */ var path__WEBPACK_IMPORTED_MODULE_1___default = /*#__PURE__*/__webpack_require__.n(path__WEBPACK_IMPORTED_MODULE_1__); +/* harmony import */ var _child_process__WEBPACK_IMPORTED_MODULE_2__ = __webpack_require__(319); +/* harmony import */ var _fs__WEBPACK_IMPORTED_MODULE_3__ = __webpack_require__(131); +/* harmony import */ var _log__WEBPACK_IMPORTED_MODULE_4__ = __webpack_require__(246); /* * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one * or more contributor license agreements. Licensed under the Elastic License @@ -47978,8 +47980,9 @@ __webpack_require__.r(__webpack_exports__); + async function readBazelToolsVersionFile(repoRootPath, versionFilename) { - const version = (await Object(_fs__WEBPACK_IMPORTED_MODULE_2__["readFile"])(Object(path__WEBPACK_IMPORTED_MODULE_0__["resolve"])(repoRootPath, versionFilename))).toString().split('\n')[0]; + const version = (await Object(_fs__WEBPACK_IMPORTED_MODULE_3__["readFile"])(Object(path__WEBPACK_IMPORTED_MODULE_1__["resolve"])(repoRootPath, versionFilename))).toString().split('\n')[0]; if (!version) { throw new Error(`[bazel_tools] Failed on reading bazel tools versions\n ${versionFilename} file do not contain any version set`); @@ -47988,30 +47991,49 @@ async function readBazelToolsVersionFile(repoRootPath, versionFilename) { return version; } +async function isBazelBinAvailable() { + try { + await Object(_child_process__WEBPACK_IMPORTED_MODULE_2__["spawn"])('bazel', ['--version'], { + stdio: 'pipe' + }); + return true; + } catch { + return false; + } +} + async function installBazelTools(repoRootPath) { - _log__WEBPACK_IMPORTED_MODULE_3__["log"].debug(`[bazel_tools] reading bazel tools versions from version files`); + _log__WEBPACK_IMPORTED_MODULE_4__["log"].debug(`[bazel_tools] reading bazel tools versions from version files`); const bazeliskVersion = await readBazelToolsVersionFile(repoRootPath, '.bazeliskversion'); const bazelVersion = await readBazelToolsVersionFile(repoRootPath, '.bazelversion'); // Check what globals are installed - _log__WEBPACK_IMPORTED_MODULE_3__["log"].debug(`[bazel_tools] verify if bazelisk is installed`); + _log__WEBPACK_IMPORTED_MODULE_4__["log"].debug(`[bazel_tools] verify if bazelisk is installed`); const { - stdout - } = await Object(_child_process__WEBPACK_IMPORTED_MODULE_1__["spawn"])('yarn', ['global', 'list'], { + stdout: bazeliskPkgInstallStdout + } = await Object(_child_process__WEBPACK_IMPORTED_MODULE_2__["spawn"])('yarn', ['global', 'list'], { stdio: 'pipe' - }); // Install bazelisk if not installed + }); + const isBazelBinAlreadyAvailable = await isBazelBinAvailable(); // Install bazelisk if not installed - if (!stdout.includes(`@bazel/bazelisk@${bazeliskVersion}`)) { - _log__WEBPACK_IMPORTED_MODULE_3__["log"].info(`[bazel_tools] installing Bazel tools`); - _log__WEBPACK_IMPORTED_MODULE_3__["log"].debug(`[bazel_tools] bazelisk is not installed. Installing @bazel/bazelisk@${bazeliskVersion} and bazel@${bazelVersion}`); - await Object(_child_process__WEBPACK_IMPORTED_MODULE_1__["spawn"])('yarn', ['global', 'add', `@bazel/bazelisk@${bazeliskVersion}`], { + if (!bazeliskPkgInstallStdout.includes(`@bazel/bazelisk@${bazeliskVersion}`) || !isBazelBinAlreadyAvailable) { + _log__WEBPACK_IMPORTED_MODULE_4__["log"].info(`[bazel_tools] installing Bazel tools`); + _log__WEBPACK_IMPORTED_MODULE_4__["log"].debug(`[bazel_tools] bazelisk is not installed. Installing @bazel/bazelisk@${bazeliskVersion} and bazel@${bazelVersion}`); + await Object(_child_process__WEBPACK_IMPORTED_MODULE_2__["spawn"])('yarn', ['global', 'add', `@bazel/bazelisk@${bazeliskVersion}`], { env: { USE_BAZEL_VERSION: bazelVersion }, stdio: 'pipe' }); + const isBazelBinAvailableAfterInstall = await isBazelBinAvailable(); + + if (!isBazelBinAvailableAfterInstall) { + throw new Error(dedent__WEBPACK_IMPORTED_MODULE_0___default.a` + [bazel_tools] an error occurred when installing the Bazel tools. Please make sure 'yarn global bin' is on your $PATH, otherwise just add it there + `); + } } - _log__WEBPACK_IMPORTED_MODULE_3__["log"].success(`[bazel_tools] all bazel tools are correctly installed`); + _log__WEBPACK_IMPORTED_MODULE_4__["log"].success(`[bazel_tools] all bazel tools are correctly installed`); } /***/ }), diff --git a/packages/kbn-pm/src/utils/bazel/install_tools.ts b/packages/kbn-pm/src/utils/bazel/install_tools.ts index 4e19974590e83b..3440d32ee4b51d 100644 --- a/packages/kbn-pm/src/utils/bazel/install_tools.ts +++ b/packages/kbn-pm/src/utils/bazel/install_tools.ts @@ -6,6 +6,7 @@ * Public License, v 1. */ +import dedent from 'dedent'; import { resolve } from 'path'; import { spawn } from '../child_process'; import { readFile } from '../fs'; @@ -25,6 +26,16 @@ async function readBazelToolsVersionFile(repoRootPath: string, versionFilename: return version; } +async function isBazelBinAvailable() { + try { + await spawn('bazel', ['--version'], { stdio: 'pipe' }); + + return true; + } catch { + return false; + } +} + export async function installBazelTools(repoRootPath: string) { log.debug(`[bazel_tools] reading bazel tools versions from version files`); const bazeliskVersion = await readBazelToolsVersionFile(repoRootPath, '.bazeliskversion'); @@ -32,10 +43,17 @@ export async function installBazelTools(repoRootPath: string) { // Check what globals are installed log.debug(`[bazel_tools] verify if bazelisk is installed`); - const { stdout } = await spawn('yarn', ['global', 'list'], { stdio: 'pipe' }); + const { stdout: bazeliskPkgInstallStdout } = await spawn('yarn', ['global', 'list'], { + stdio: 'pipe', + }); + + const isBazelBinAlreadyAvailable = await isBazelBinAvailable(); // Install bazelisk if not installed - if (!stdout.includes(`@bazel/bazelisk@${bazeliskVersion}`)) { + if ( + !bazeliskPkgInstallStdout.includes(`@bazel/bazelisk@${bazeliskVersion}`) || + !isBazelBinAlreadyAvailable + ) { log.info(`[bazel_tools] installing Bazel tools`); log.debug( @@ -47,6 +65,13 @@ export async function installBazelTools(repoRootPath: string) { }, stdio: 'pipe', }); + + const isBazelBinAvailableAfterInstall = await isBazelBinAvailable(); + if (!isBazelBinAvailableAfterInstall) { + throw new Error(dedent` + [bazel_tools] an error occurred when installing the Bazel tools. Please make sure 'yarn global bin' is on your $PATH, otherwise just add it there + `); + } } log.success(`[bazel_tools] all bazel tools are correctly installed`); diff --git a/preinstall_check.js b/preinstall_check.js index e761afa91022cc..e508f9ecb10c68 100644 --- a/preinstall_check.js +++ b/preinstall_check.js @@ -6,34 +6,37 @@ * Public License, v 1. */ -const isUsingNpm = process.env.npm_config_git !== undefined; +(() => { + const isUsingNpm = process.env.npm_config_git !== undefined; -if (isUsingNpm) { - throw `Use Yarn instead of npm, see Kibana's contributing guidelines`; -} - -// The value of the `npm_config_argv` env for each command: -// -// - `npm install`: '{"remain":[],"cooked":["install"],"original":[]}' -// - `yarn`: '{"remain":[],"cooked":["install"],"original":[]}' -// - `yarn kbn bootstrap`: '{"remain":[],"cooked":["run","kbn"],"original":["kbn","bootstrap"]}' -const rawArgv = process.env.npm_config_argv; - -if (rawArgv === undefined) { - return; -} + if (isUsingNpm) { + throw `Use Yarn instead of npm, see Kibana's contributing guidelines`; + } -try { - const argv = JSON.parse(rawArgv); + // The value of the `npm_config_argv` env for each command: + // + // - `npm install`: '{"remain":[],"cooked":["install"],"original":[]}' + // - `yarn`: '{"remain":[],"cooked":["install"],"original":[]}' + // - `yarn kbn bootstrap`: '{"remain":[],"cooked":["run","kbn"],"original":["kbn","bootstrap"]}' + const rawArgv = process.env.npm_config_argv; - if (argv.cooked.includes('kbn')) { - // all good, trying to install deps using `kbn` + if (rawArgv === undefined) { return; } - if (argv.cooked.includes('install')) { - console.log('\nWARNING: When installing dependencies, prefer `yarn kbn bootstrap`\n'); + try { + const argv = JSON.parse(rawArgv); + + // allow dependencies to be installed with `yarn kbn bootstrap` or `bazel run @nodejs//:yarn` (called under the hood by bazel) + if (argv.cooked.includes('kbn') || !!process.env.BAZEL_YARN_INSTALL) { + // all good, trying to install deps using `kbn` or bazel directly + return; + } + + if (argv.cooked.includes('install')) { + console.log('\nWARNING: When installing dependencies, prefer `yarn kbn bootstrap`\n'); + } + } catch (e) { + // if it fails we do nothing, as this is just intended to be a helpful message } -} catch (e) { - // if it fails we do nothing, as this is just intended to be a helpful message -} +})(); diff --git a/src/core/server/saved_objects/migrations/kibana/kibana_migrator.test.ts b/src/core/server/saved_objects/migrations/kibana/kibana_migrator.test.ts index dac93ff29b68f6..e6fd5004dd7d5a 100644 --- a/src/core/server/saved_objects/migrations/kibana/kibana_migrator.test.ts +++ b/src/core/server/saved_objects/migrations/kibana/kibana_migrator.test.ts @@ -14,6 +14,16 @@ import { loggingSystemMock } from '../../../logging/logging_system.mock'; import { SavedObjectTypeRegistry } from '../../saved_objects_type_registry'; import { SavedObjectsType } from '../../types'; import { errors as esErrors } from '@elastic/elasticsearch'; +import { DocumentMigrator } from '../core/document_migrator'; +jest.mock('../core/document_migrator', () => { + return { + // Create a mock for spying on the constructor + DocumentMigrator: jest.fn().mockImplementation((...args) => { + const { DocumentMigrator: RealDocMigrator } = jest.requireActual('../core/document_migrator'); + return new RealDocMigrator(args[0]); + }), + }; +}); const createRegistry = (types: Array>) => { const registry = new SavedObjectTypeRegistry(); @@ -31,12 +41,16 @@ const createRegistry = (types: Array>) => { }; describe('KibanaMigrator', () => { + beforeEach(() => { + (DocumentMigrator as jest.Mock).mockClear(); + }); describe('constructor', () => { it('coerces the current Kibana version if it has a hyphen', () => { const options = mockOptions(); options.kibanaVersion = '3.2.1-SNAPSHOT'; const migrator = new KibanaMigrator(options); expect(migrator.kibanaVersion).toEqual('3.2.1'); + expect((DocumentMigrator as jest.Mock).mock.calls[0][0].kibanaVersion).toEqual('3.2.1'); }); }); describe('getActiveMappings', () => { @@ -105,8 +119,8 @@ describe('KibanaMigrator', () => { const migrator = new KibanaMigrator(options); - expect(() => migrator.runMigrations()).rejects.toThrow( - /Migrations are not ready. Make sure prepareMigrations is called first./i + await expect(() => migrator.runMigrations()).toThrowErrorMatchingInlineSnapshot( + `"Migrations are not ready. Make sure prepareMigrations is called first."` ); }); diff --git a/src/core/server/saved_objects/migrations/kibana/kibana_migrator.ts b/src/core/server/saved_objects/migrations/kibana/kibana_migrator.ts index ecef84a6e297ce..1a4611b4914197 100644 --- a/src/core/server/saved_objects/migrations/kibana/kibana_migrator.ts +++ b/src/core/server/saved_objects/migrations/kibana/kibana_migrator.ts @@ -12,6 +12,7 @@ */ import { BehaviorSubject } from 'rxjs'; +import Semver from 'semver'; import { KibanaConfigType } from '../../../kibana_config'; import { ElasticsearchClient } from '../../../elasticsearch'; import { Logger } from '../../../logging'; @@ -97,7 +98,7 @@ export class KibanaMigrator { this.log = logger; this.kibanaVersion = kibanaVersion.split('-')[0]; // coerce a semver-like string (x.y.z-SNAPSHOT) or prerelease version (x.y.z-alpha) to a regular semver (x.y.z); this.documentMigrator = new DocumentMigrator({ - kibanaVersion, + kibanaVersion: this.kibanaVersion, typeRegistry, log: this.log, }); @@ -163,6 +164,15 @@ export class KibanaMigrator { registry: this.typeRegistry, }); + this.log.debug('Applying registered migrations for the following saved object types:'); + Object.entries(this.documentMigrator.migrationVersion) + .sort(([t1, v1], [t2, v2]) => { + return Semver.compare(v1, v2); + }) + .forEach(([type, migrationVersion]) => { + this.log.debug(`migrationVersion: ${migrationVersion} saved object type: ${type}`); + }); + const migrators = Object.keys(indexMap).map((index) => { // TODO migrationsV2: remove old migrations algorithm if (this.savedObjectsConfig.enableV2) { diff --git a/src/core/server/saved_objects/migrationsv2/model.test.ts b/src/core/server/saved_objects/migrationsv2/model.test.ts index a9aa69960b1c2f..d5ab85c54a7287 100644 --- a/src/core/server/saved_objects/migrationsv2/model.test.ts +++ b/src/core/server/saved_objects/migrationsv2/model.test.ts @@ -182,21 +182,6 @@ describe('migrations v2 model', () => { versionAlias: '.kibana_7.11.0', versionIndex: '.kibana_7.11.0_001', }; - const mappingsWithUnknownType = { - properties: { - disabled_saved_object_type: { - properties: { - value: { type: 'keyword' }, - }, - }, - }, - _meta: { - migrationMappingPropertyHashes: { - disabled_saved_object_type: '7997cf5a56cc02bdc9c93361bde732b0', - }, - }, - }; - test('INIT -> OUTDATED_DOCUMENTS_SEARCH if .kibana is already pointing to the target index', () => { const res: ResponseType<'INIT'> = Either.right({ '.kibana_7.11.0_001': { @@ -204,27 +189,38 @@ describe('migrations v2 model', () => { '.kibana': {}, '.kibana_7.11.0': {}, }, - mappings: mappingsWithUnknownType, + mappings: { + properties: { + disabled_saved_object_type: { + properties: { + value: { type: 'keyword' }, + }, + }, + }, + _meta: { + migrationMappingPropertyHashes: { + disabled_saved_object_type: '7997cf5a56cc02bdc9c93361bde732b0', + }, + }, + }, settings: {}, }, }); const newState = model(initState, res); expect(newState.controlState).toEqual('OUTDATED_DOCUMENTS_SEARCH'); - // This snapshot asserts that we merge the - // migrationMappingPropertyHashes of the existing index, but we leave - // the mappings for the disabled_saved_object_type untouched. There - // might be another Kibana instance that knows about this type and - // needs these mappings in place. expect(newState.targetIndexMappings).toMatchInlineSnapshot(` Object { "_meta": Object { "migrationMappingPropertyHashes": Object { - "disabled_saved_object_type": "7997cf5a56cc02bdc9c93361bde732b0", "new_saved_object_type": "4a11183eee21e6fbad864f7a30b39ad0", }, }, "properties": Object { + "disabled_saved_object_type": Object { + "dynamic": false, + "properties": Object {}, + }, "new_saved_object_type": Object { "properties": Object { "value": Object { @@ -275,7 +271,7 @@ describe('migrations v2 model', () => { '.kibana': {}, '.kibana_7.12.0': {}, }, - mappings: mappingsWithUnknownType, + mappings: { properties: {}, _meta: { migrationMappingPropertyHashes: {} } }, settings: {}, }, '.kibana_7.11.0_001': { @@ -292,37 +288,12 @@ describe('migrations v2 model', () => { sourceIndex: Option.some('.kibana_7.invalid.0_001'), targetIndex: '.kibana_7.11.0_001', }); - // This snapshot asserts that we disable the unknown saved object - // type. Because it's mappings are disabled, we also don't copy the - // `_meta.migrationMappingPropertyHashes` for the disabled type. - expect(newState.targetIndexMappings).toMatchInlineSnapshot(` - Object { - "_meta": Object { - "migrationMappingPropertyHashes": Object { - "new_saved_object_type": "4a11183eee21e6fbad864f7a30b39ad0", - }, - }, - "properties": Object { - "disabled_saved_object_type": Object { - "dynamic": false, - "properties": Object {}, - }, - "new_saved_object_type": Object { - "properties": Object { - "value": Object { - "type": "text", - }, - }, - }, - }, - } - `); }); test('INIT -> SET_SOURCE_WRITE_BLOCK when migrating from a v2 migrations index (>= 7.11.0)', () => { const res: ResponseType<'INIT'> = Either.right({ '.kibana_7.11.0_001': { aliases: { '.kibana': {}, '.kibana_7.11.0': {} }, - mappings: mappingsWithUnknownType, + mappings: { properties: {}, _meta: { migrationMappingPropertyHashes: {} } }, settings: {}, }, '.kibana_3': { @@ -348,31 +319,6 @@ describe('migrations v2 model', () => { sourceIndex: Option.some('.kibana_7.11.0_001'), targetIndex: '.kibana_7.12.0_001', }); - // This snapshot asserts that we disable the unknown saved object - // type. Because it's mappings are disabled, we also don't copy the - // `_meta.migrationMappingPropertyHashes` for the disabled type. - expect(newState.targetIndexMappings).toMatchInlineSnapshot(` - Object { - "_meta": Object { - "migrationMappingPropertyHashes": Object { - "new_saved_object_type": "4a11183eee21e6fbad864f7a30b39ad0", - }, - }, - "properties": Object { - "disabled_saved_object_type": Object { - "dynamic": false, - "properties": Object {}, - }, - "new_saved_object_type": Object { - "properties": Object { - "value": Object { - "type": "text", - }, - }, - }, - }, - } - `); expect(newState.retryCount).toEqual(0); expect(newState.retryDelay).toEqual(0); }); @@ -382,7 +328,7 @@ describe('migrations v2 model', () => { aliases: { '.kibana': {}, }, - mappings: mappingsWithUnknownType, + mappings: { properties: {}, _meta: { migrationMappingPropertyHashes: {} } }, settings: {}, }, }); @@ -393,31 +339,6 @@ describe('migrations v2 model', () => { sourceIndex: Option.some('.kibana_3'), targetIndex: '.kibana_7.11.0_001', }); - // This snapshot asserts that we disable the unknown saved object - // type. Because it's mappings are disabled, we also don't copy the - // `_meta.migrationMappingPropertyHashes` for the disabled type. - expect(newState.targetIndexMappings).toMatchInlineSnapshot(` - Object { - "_meta": Object { - "migrationMappingPropertyHashes": Object { - "new_saved_object_type": "4a11183eee21e6fbad864f7a30b39ad0", - }, - }, - "properties": Object { - "disabled_saved_object_type": Object { - "dynamic": false, - "properties": Object {}, - }, - "new_saved_object_type": Object { - "properties": Object { - "value": Object { - "type": "text", - }, - }, - }, - }, - } - `); expect(newState.retryCount).toEqual(0); expect(newState.retryDelay).toEqual(0); }); @@ -425,7 +346,7 @@ describe('migrations v2 model', () => { const res: ResponseType<'INIT'> = Either.right({ '.kibana': { aliases: {}, - mappings: mappingsWithUnknownType, + mappings: { properties: {}, _meta: {} }, settings: {}, }, }); @@ -436,31 +357,6 @@ describe('migrations v2 model', () => { sourceIndex: Option.some('.kibana_pre6.5.0_001'), targetIndex: '.kibana_7.11.0_001', }); - // This snapshot asserts that we disable the unknown saved object - // type. Because it's mappings are disabled, we also don't copy the - // `_meta.migrationMappingPropertyHashes` for the disabled type. - expect(newState.targetIndexMappings).toMatchInlineSnapshot(` - Object { - "_meta": Object { - "migrationMappingPropertyHashes": Object { - "new_saved_object_type": "4a11183eee21e6fbad864f7a30b39ad0", - }, - }, - "properties": Object { - "disabled_saved_object_type": Object { - "dynamic": false, - "properties": Object {}, - }, - "new_saved_object_type": Object { - "properties": Object { - "value": Object { - "type": "text", - }, - }, - }, - }, - } - `); expect(newState.retryCount).toEqual(0); expect(newState.retryDelay).toEqual(0); }); @@ -470,7 +366,7 @@ describe('migrations v2 model', () => { aliases: { 'my-saved-objects': {}, }, - mappings: mappingsWithUnknownType, + mappings: { properties: {}, _meta: { migrationMappingPropertyHashes: {} } }, settings: {}, }, }); @@ -490,31 +386,6 @@ describe('migrations v2 model', () => { sourceIndex: Option.some('my-saved-objects_3'), targetIndex: 'my-saved-objects_7.11.0_001', }); - // This snapshot asserts that we disable the unknown saved object - // type. Because it's mappings are disabled, we also don't copy the - // `_meta.migrationMappingPropertyHashes` for the disabled type. - expect(newState.targetIndexMappings).toMatchInlineSnapshot(` - Object { - "_meta": Object { - "migrationMappingPropertyHashes": Object { - "new_saved_object_type": "4a11183eee21e6fbad864f7a30b39ad0", - }, - }, - "properties": Object { - "disabled_saved_object_type": Object { - "dynamic": false, - "properties": Object {}, - }, - "new_saved_object_type": Object { - "properties": Object { - "value": Object { - "type": "text", - }, - }, - }, - }, - } - `); expect(newState.retryCount).toEqual(0); expect(newState.retryDelay).toEqual(0); }); @@ -524,7 +395,7 @@ describe('migrations v2 model', () => { aliases: { 'my-saved-objects': {}, }, - mappings: mappingsWithUnknownType, + mappings: { properties: {}, _meta: { migrationMappingPropertyHashes: {} } }, settings: {}, }, }); @@ -545,31 +416,6 @@ describe('migrations v2 model', () => { sourceIndex: Option.some('my-saved-objects_7.11.0'), targetIndex: 'my-saved-objects_7.12.0_001', }); - // This snapshot asserts that we disable the unknown saved object - // type. Because it's mappings are disabled, we also don't copy the - // `_meta.migrationMappingPropertyHashes` for the disabled type. - expect(newState.targetIndexMappings).toMatchInlineSnapshot(` - Object { - "_meta": Object { - "migrationMappingPropertyHashes": Object { - "new_saved_object_type": "4a11183eee21e6fbad864f7a30b39ad0", - }, - }, - "properties": Object { - "disabled_saved_object_type": Object { - "dynamic": false, - "properties": Object {}, - }, - "new_saved_object_type": Object { - "properties": Object { - "value": Object { - "type": "text", - }, - }, - }, - }, - } - `); expect(newState.retryCount).toEqual(0); expect(newState.retryDelay).toEqual(0); }); diff --git a/src/core/server/saved_objects/migrationsv2/model.ts b/src/core/server/saved_objects/migrationsv2/model.ts index 3556bb611bb671..1119edde8e2681 100644 --- a/src/core/server/saved_objects/migrationsv2/model.ts +++ b/src/core/server/saved_objects/migrationsv2/model.ts @@ -60,13 +60,13 @@ function throwBadResponse(state: State, res: any): never { * Merge the _meta.migrationMappingPropertyHashes mappings of an index with * the given target mappings. * - * @remarks When another instance already completed a migration, the existing - * target index might contain documents and mappings created by a plugin that - * is disabled in the current Kibana instance performing this migration. - * Mapping updates are commutative (deeply merged) by Elasticsearch, except - * for the `_meta` key. By merging the `_meta.migrationMappingPropertyHashes` - * mappings from the existing target index index into the targetMappings we - * ensure that any `migrationPropertyHashes` for disabled plugins aren't lost. + * @remarks Mapping updates are commutative (deeply merged) by Elasticsearch, + * except for the _meta key. The source index we're migrating from might + * contain documents created by a plugin that is disabled in the Kibana + * instance performing this migration. We merge the + * _meta.migrationMappingPropertyHashes mappings from the source index into + * the targetMappings to ensure that any `migrationPropertyHashes` for + * disabled plugins aren't lost. * * Right now we don't use these `migrationPropertyHashes` but it could be used * in the future to detect if mappings were changed. If mappings weren't @@ -209,7 +209,7 @@ export const model = (currentState: State, resW: ResponseType): // index sourceIndex: Option.none, targetIndex: `${stateP.indexPrefix}_${stateP.kibanaVersion}_001`, - targetIndexMappings: mergeMigrationMappingPropertyHashes( + targetIndexMappings: disableUnknownTypeMappingFields( stateP.targetIndexMappings, indices[aliases[stateP.currentAlias]].mappings ), @@ -242,7 +242,7 @@ export const model = (currentState: State, resW: ResponseType): controlState: 'SET_SOURCE_WRITE_BLOCK', sourceIndex: Option.some(source) as Option.Some, targetIndex: target, - targetIndexMappings: disableUnknownTypeMappingFields( + targetIndexMappings: mergeMigrationMappingPropertyHashes( stateP.targetIndexMappings, indices[source].mappings ), @@ -279,7 +279,7 @@ export const model = (currentState: State, resW: ResponseType): controlState: 'LEGACY_SET_WRITE_BLOCK', sourceIndex: Option.some(legacyReindexTarget) as Option.Some, targetIndex: target, - targetIndexMappings: disableUnknownTypeMappingFields( + targetIndexMappings: mergeMigrationMappingPropertyHashes( stateP.targetIndexMappings, indices[stateP.legacyIndex].mappings ), diff --git a/src/core/server/saved_objects/routes/bulk_create.ts b/src/core/server/saved_objects/routes/bulk_create.ts index b85747985e523f..6d57eaa3777e6d 100644 --- a/src/core/server/saved_objects/routes/bulk_create.ts +++ b/src/core/server/saved_objects/routes/bulk_create.ts @@ -9,7 +9,6 @@ import { schema } from '@kbn/config-schema'; import { IRouter } from '../../http'; import { CoreUsageDataSetup } from '../../core_usage_data'; -import { catchAndReturnBoomErrors } from './utils'; interface RouteDependencies { coreUsageData: CoreUsageDataSetup; @@ -45,7 +44,7 @@ export const registerBulkCreateRoute = (router: IRouter, { coreUsageData }: Rout ), }, }, - catchAndReturnBoomErrors(async (context, req, res) => { + router.handleLegacyErrors(async (context, req, res) => { const { overwrite } = req.query; const usageStatsClient = coreUsageData.getClient(); diff --git a/src/core/server/saved_objects/routes/bulk_get.ts b/src/core/server/saved_objects/routes/bulk_get.ts index 580bf26a4e5294..a2603016336684 100644 --- a/src/core/server/saved_objects/routes/bulk_get.ts +++ b/src/core/server/saved_objects/routes/bulk_get.ts @@ -9,7 +9,6 @@ import { schema } from '@kbn/config-schema'; import { IRouter } from '../../http'; import { CoreUsageDataSetup } from '../../core_usage_data'; -import { catchAndReturnBoomErrors } from './utils'; interface RouteDependencies { coreUsageData: CoreUsageDataSetup; @@ -29,7 +28,7 @@ export const registerBulkGetRoute = (router: IRouter, { coreUsageData }: RouteDe ), }, }, - catchAndReturnBoomErrors(async (context, req, res) => { + router.handleLegacyErrors(async (context, req, res) => { const usageStatsClient = coreUsageData.getClient(); usageStatsClient.incrementSavedObjectsBulkGet({ request: req }).catch(() => {}); diff --git a/src/core/server/saved_objects/routes/bulk_update.ts b/src/core/server/saved_objects/routes/bulk_update.ts index e592adc72a2446..f9b8d4a2f567f5 100644 --- a/src/core/server/saved_objects/routes/bulk_update.ts +++ b/src/core/server/saved_objects/routes/bulk_update.ts @@ -9,7 +9,6 @@ import { schema } from '@kbn/config-schema'; import { IRouter } from '../../http'; import { CoreUsageDataSetup } from '../../core_usage_data'; -import { catchAndReturnBoomErrors } from './utils'; interface RouteDependencies { coreUsageData: CoreUsageDataSetup; @@ -40,7 +39,7 @@ export const registerBulkUpdateRoute = (router: IRouter, { coreUsageData }: Rout ), }, }, - catchAndReturnBoomErrors(async (context, req, res) => { + router.handleLegacyErrors(async (context, req, res) => { const usageStatsClient = coreUsageData.getClient(); usageStatsClient.incrementSavedObjectsBulkUpdate({ request: req }).catch(() => {}); diff --git a/src/core/server/saved_objects/routes/create.ts b/src/core/server/saved_objects/routes/create.ts index f6043ca96398d9..fd256abac35266 100644 --- a/src/core/server/saved_objects/routes/create.ts +++ b/src/core/server/saved_objects/routes/create.ts @@ -9,7 +9,6 @@ import { schema } from '@kbn/config-schema'; import { IRouter } from '../../http'; import { CoreUsageDataSetup } from '../../core_usage_data'; -import { catchAndReturnBoomErrors } from './utils'; interface RouteDependencies { coreUsageData: CoreUsageDataSetup; @@ -44,7 +43,7 @@ export const registerCreateRoute = (router: IRouter, { coreUsageData }: RouteDep }), }, }, - catchAndReturnBoomErrors(async (context, req, res) => { + router.handleLegacyErrors(async (context, req, res) => { const { type, id } = req.params; const { overwrite } = req.query; const { diff --git a/src/core/server/saved_objects/routes/delete.ts b/src/core/server/saved_objects/routes/delete.ts index b127f64b74a0c9..a7846c3dc845b5 100644 --- a/src/core/server/saved_objects/routes/delete.ts +++ b/src/core/server/saved_objects/routes/delete.ts @@ -9,7 +9,6 @@ import { schema } from '@kbn/config-schema'; import { IRouter } from '../../http'; import { CoreUsageDataSetup } from '../../core_usage_data'; -import { catchAndReturnBoomErrors } from './utils'; interface RouteDependencies { coreUsageData: CoreUsageDataSetup; @@ -29,7 +28,7 @@ export const registerDeleteRoute = (router: IRouter, { coreUsageData }: RouteDep }), }, }, - catchAndReturnBoomErrors(async (context, req, res) => { + router.handleLegacyErrors(async (context, req, res) => { const { type, id } = req.params; const { force } = req.query; diff --git a/src/core/server/saved_objects/routes/export.ts b/src/core/server/saved_objects/routes/export.ts index f064cf1ca0ec1e..9b40855afec2e4 100644 --- a/src/core/server/saved_objects/routes/export.ts +++ b/src/core/server/saved_objects/routes/export.ts @@ -18,7 +18,7 @@ import { SavedObjectsExportByObjectOptions, SavedObjectsExportError, } from '../export'; -import { validateTypes, validateObjects, catchAndReturnBoomErrors } from './utils'; +import { validateTypes, validateObjects } from './utils'; interface RouteDependencies { config: SavedObjectConfig; @@ -163,7 +163,7 @@ export const registerExportRoute = ( }), }, }, - catchAndReturnBoomErrors(async (context, req, res) => { + router.handleLegacyErrors(async (context, req, res) => { const cleaned = cleanOptions(req.body); const supportedTypes = context.core.savedObjects.typeRegistry .getImportableAndExportableTypes() diff --git a/src/core/server/saved_objects/routes/find.ts b/src/core/server/saved_objects/routes/find.ts index c814fd310dc523..747070e54e5ad8 100644 --- a/src/core/server/saved_objects/routes/find.ts +++ b/src/core/server/saved_objects/routes/find.ts @@ -9,7 +9,6 @@ import { schema } from '@kbn/config-schema'; import { IRouter } from '../../http'; import { CoreUsageDataSetup } from '../../core_usage_data'; -import { catchAndReturnBoomErrors } from './utils'; interface RouteDependencies { coreUsageData: CoreUsageDataSetup; @@ -50,7 +49,7 @@ export const registerFindRoute = (router: IRouter, { coreUsageData }: RouteDepen }), }, }, - catchAndReturnBoomErrors(async (context, req, res) => { + router.handleLegacyErrors(async (context, req, res) => { const query = req.query; const namespaces = diff --git a/src/core/server/saved_objects/routes/get.ts b/src/core/server/saved_objects/routes/get.ts index 2dd812f35cefd5..c66a11dcf0cdd0 100644 --- a/src/core/server/saved_objects/routes/get.ts +++ b/src/core/server/saved_objects/routes/get.ts @@ -9,7 +9,6 @@ import { schema } from '@kbn/config-schema'; import { IRouter } from '../../http'; import { CoreUsageDataSetup } from '../../core_usage_data'; -import { catchAndReturnBoomErrors } from './utils'; interface RouteDependencies { coreUsageData: CoreUsageDataSetup; @@ -26,7 +25,7 @@ export const registerGetRoute = (router: IRouter, { coreUsageData }: RouteDepend }), }, }, - catchAndReturnBoomErrors(async (context, req, res) => { + router.handleLegacyErrors(async (context, req, res) => { const { type, id } = req.params; const usageStatsClient = coreUsageData.getClient(); diff --git a/src/core/server/saved_objects/routes/import.ts b/src/core/server/saved_objects/routes/import.ts index 5fd132acafbed6..6c4c759460ce3a 100644 --- a/src/core/server/saved_objects/routes/import.ts +++ b/src/core/server/saved_objects/routes/import.ts @@ -13,7 +13,7 @@ import { IRouter } from '../../http'; import { CoreUsageDataSetup } from '../../core_usage_data'; import { SavedObjectConfig } from '../saved_objects_config'; import { SavedObjectsImportError } from '../import'; -import { catchAndReturnBoomErrors, createSavedObjectsStreamFromNdJson } from './utils'; +import { createSavedObjectsStreamFromNdJson } from './utils'; interface RouteDependencies { config: SavedObjectConfig; @@ -61,7 +61,7 @@ export const registerImportRoute = ( }), }, }, - catchAndReturnBoomErrors(async (context, req, res) => { + router.handleLegacyErrors(async (context, req, res) => { const { overwrite, createNewCopies } = req.query; const usageStatsClient = coreUsageData.getClient(); diff --git a/src/core/server/saved_objects/routes/migrate.ts b/src/core/server/saved_objects/routes/migrate.ts index 7c2f4bfb067106..8b347d4725b08e 100644 --- a/src/core/server/saved_objects/routes/migrate.ts +++ b/src/core/server/saved_objects/routes/migrate.ts @@ -8,7 +8,6 @@ import { IRouter } from '../../http'; import { IKibanaMigrator } from '../migrations'; -import { catchAndReturnBoomErrors } from './utils'; export const registerMigrateRoute = ( router: IRouter, @@ -22,7 +21,7 @@ export const registerMigrateRoute = ( tags: ['access:migrateSavedObjects'], }, }, - catchAndReturnBoomErrors(async (context, req, res) => { + router.handleLegacyErrors(async (context, req, res) => { const migrator = await migratorPromise; await migrator.runMigrations({ rerun: true }); return res.ok({ diff --git a/src/core/server/saved_objects/routes/resolve_import_errors.ts b/src/core/server/saved_objects/routes/resolve_import_errors.ts index 6f0a3d028baf95..0cf976c30b311e 100644 --- a/src/core/server/saved_objects/routes/resolve_import_errors.ts +++ b/src/core/server/saved_objects/routes/resolve_import_errors.ts @@ -13,7 +13,8 @@ import { IRouter } from '../../http'; import { CoreUsageDataSetup } from '../../core_usage_data'; import { SavedObjectConfig } from '../saved_objects_config'; import { SavedObjectsImportError } from '../import'; -import { catchAndReturnBoomErrors, createSavedObjectsStreamFromNdJson } from './utils'; +import { createSavedObjectsStreamFromNdJson } from './utils'; + interface RouteDependencies { config: SavedObjectConfig; coreUsageData: CoreUsageDataSetup; @@ -68,7 +69,7 @@ export const registerResolveImportErrorsRoute = ( }), }, }, - catchAndReturnBoomErrors(async (context, req, res) => { + router.handleLegacyErrors(async (context, req, res) => { const { createNewCopies } = req.query; const usageStatsClient = coreUsageData.getClient(); diff --git a/src/core/server/saved_objects/routes/update.ts b/src/core/server/saved_objects/routes/update.ts index dbc69f743df76b..17cfd438d47bfe 100644 --- a/src/core/server/saved_objects/routes/update.ts +++ b/src/core/server/saved_objects/routes/update.ts @@ -9,7 +9,6 @@ import { schema } from '@kbn/config-schema'; import { IRouter } from '../../http'; import { CoreUsageDataSetup } from '../../core_usage_data'; -import { catchAndReturnBoomErrors } from './utils'; interface RouteDependencies { coreUsageData: CoreUsageDataSetup; @@ -39,7 +38,7 @@ export const registerUpdateRoute = (router: IRouter, { coreUsageData }: RouteDep }), }, }, - catchAndReturnBoomErrors(async (context, req, res) => { + router.handleLegacyErrors(async (context, req, res) => { const { type, id } = req.params; const { attributes, version, references } = req.body; const options = { version, references }; diff --git a/src/core/server/saved_objects/routes/utils.test.ts b/src/core/server/saved_objects/routes/utils.test.ts index 1d7e86e288b182..ade7b03f6a8c22 100644 --- a/src/core/server/saved_objects/routes/utils.test.ts +++ b/src/core/server/saved_objects/routes/utils.test.ts @@ -9,15 +9,6 @@ import { createSavedObjectsStreamFromNdJson, validateTypes, validateObjects } from './utils'; import { Readable } from 'stream'; import { createPromiseFromStreams, createConcatStream } from '@kbn/utils'; -import { catchAndReturnBoomErrors } from './utils'; -import Boom from '@hapi/boom'; -import { - KibanaRequest, - RequestHandler, - RequestHandlerContext, - KibanaResponseFactory, - kibanaResponseFactory, -} from '../../'; async function readStreamToCompletion(stream: Readable) { return createPromiseFromStreams([stream, createConcatStream([])]); @@ -152,69 +143,3 @@ describe('validateObjects', () => { ).toBeUndefined(); }); }); - -describe('catchAndReturnBoomErrors', () => { - let context: RequestHandlerContext; - let request: KibanaRequest; - let response: KibanaResponseFactory; - - const createHandler = (handler: () => any): RequestHandler => () => { - return handler(); - }; - - beforeEach(() => { - context = {} as any; - request = {} as any; - response = kibanaResponseFactory; - }); - - it('should pass-though call parameters to the handler', async () => { - const handler = jest.fn(); - const wrapped = catchAndReturnBoomErrors(handler); - await wrapped(context, request, response); - expect(handler).toHaveBeenCalledWith(context, request, response); - }); - - it('should pass-though result from the handler', async () => { - const handler = createHandler(() => { - return 'handler-response'; - }); - const wrapped = catchAndReturnBoomErrors(handler); - const result = await wrapped(context, request, response); - expect(result).toBe('handler-response'); - }); - - it('should intercept and convert thrown Boom errors', async () => { - const handler = createHandler(() => { - throw Boom.notFound('not there'); - }); - const wrapped = catchAndReturnBoomErrors(handler); - const result = await wrapped(context, request, response); - expect(result.status).toBe(404); - expect(result.payload).toEqual({ - error: 'Not Found', - message: 'not there', - statusCode: 404, - }); - }); - - it('should re-throw non-Boom errors', async () => { - const handler = createHandler(() => { - throw new Error('something went bad'); - }); - const wrapped = catchAndReturnBoomErrors(handler); - await expect(wrapped(context, request, response)).rejects.toMatchInlineSnapshot( - `[Error: something went bad]` - ); - }); - - it('should re-throw Boom internal/500 errors', async () => { - const handler = createHandler(() => { - throw Boom.internal(); - }); - const wrapped = catchAndReturnBoomErrors(handler); - await expect(wrapped(context, request, response)).rejects.toMatchInlineSnapshot( - `[Error: Internal Server Error]` - ); - }); -}); diff --git a/src/core/server/saved_objects/routes/utils.ts b/src/core/server/saved_objects/routes/utils.ts index 269f3f0698561d..b9e7df48a4b4c7 100644 --- a/src/core/server/saved_objects/routes/utils.ts +++ b/src/core/server/saved_objects/routes/utils.ts @@ -7,11 +7,7 @@ */ import { Readable } from 'stream'; -import { - RequestHandlerWrapper, - SavedObject, - SavedObjectsExportResultDetails, -} from 'src/core/server'; +import { SavedObject, SavedObjectsExportResultDetails } from 'src/core/server'; import { createSplitStream, createMapStream, @@ -20,7 +16,6 @@ import { createListStream, createConcatStream, } from '@kbn/utils'; -import Boom from '@hapi/boom'; export async function createSavedObjectsStreamFromNdJson(ndJsonStream: Readable) { const savedObjects = await createPromiseFromStreams([ @@ -57,30 +52,3 @@ export function validateObjects( .join(', ')}`; } } - -/** - * Catches errors thrown by saved object route handlers and returns an error - * with the payload and statusCode of the boom error. - * - * This is very close to the core `router.handleLegacyErrors` except that it - * throws internal errors (statusCode: 500) so that the internal error's - * message get logged by Core. - * - * TODO: Remove once https://github.com/elastic/kibana/issues/65291 is fixed. - */ -export const catchAndReturnBoomErrors: RequestHandlerWrapper = (handler) => { - return async (context, request, response) => { - try { - return await handler(context, request, response); - } catch (e) { - if (Boom.isBoom(e) && e.output.statusCode !== 500) { - return response.customError({ - body: e.output.payload, - statusCode: e.output.statusCode, - headers: e.output.headers as { [key: string]: string }, - }); - } - throw e; - } - }; -}; diff --git a/src/core/server/saved_objects/service/lib/decorate_es_error.test.ts b/src/core/server/saved_objects/service/lib/decorate_es_error.test.ts index da1ebec2c0f7df..cc497ca6348b87 100644 --- a/src/core/server/saved_objects/service/lib/decorate_es_error.test.ts +++ b/src/core/server/saved_objects/service/lib/decorate_es_error.test.ts @@ -109,27 +109,6 @@ describe('savedObjectsClient/decorateEsError', () => { expect(SavedObjectsErrorHelpers.isNotFoundError(genericError)).toBe(true); }); - it('if saved objects index does not exist makes NotFound a SavedObjectsClient/generalError', () => { - const error = new esErrors.ResponseError( - elasticsearchClientMock.createApiResponse({ - statusCode: 404, - body: { - error: { - reason: - 'no such index [.kibana_8.0.0] and [require_alias] request flag is [true] and [.kibana_8.0.0] is not an alias', - }, - }, - }) - ); - expect(SavedObjectsErrorHelpers.isGeneralError(error)).toBe(false); - const genericError = decorateEsError(error); - expect(genericError.message).toEqual( - `Saved object index alias [.kibana_8.0.0] not found: Response Error` - ); - expect(genericError.output.statusCode).toBe(500); - expect(SavedObjectsErrorHelpers.isGeneralError(error)).toBe(true); - }); - it('makes BadRequest a SavedObjectsClient/BadRequest error', () => { const error = new esErrors.ResponseError( elasticsearchClientMock.createApiResponse({ statusCode: 400 }) diff --git a/src/core/server/saved_objects/service/lib/decorate_es_error.ts b/src/core/server/saved_objects/service/lib/decorate_es_error.ts index aabca2d602cb39..40f18c9c94c25b 100644 --- a/src/core/server/saved_objects/service/lib/decorate_es_error.ts +++ b/src/core/server/saved_objects/service/lib/decorate_es_error.ts @@ -63,12 +63,6 @@ export function decorateEsError(error: EsErrors) { } if (responseErrors.isNotFound(error.statusCode)) { - const match = error?.meta?.body?.error?.reason?.match( - /no such index \[(.+)\] and \[require_alias\] request flag is \[true\] and \[.+\] is not an alias/ - ); - if (match?.length > 0) { - return SavedObjectsErrorHelpers.decorateIndexAliasNotFoundError(error, match[1]); - } return SavedObjectsErrorHelpers.createGenericNotFoundError(); } diff --git a/src/core/server/saved_objects/service/lib/errors.ts b/src/core/server/saved_objects/service/lib/errors.ts index c348196aaba21f..f216e72efbcf82 100644 --- a/src/core/server/saved_objects/service/lib/errors.ts +++ b/src/core/server/saved_objects/service/lib/errors.ts @@ -135,19 +135,6 @@ export class SavedObjectsErrorHelpers { return decorate(Boom.notFound(), CODE_NOT_FOUND, 404); } - public static createIndexAliasNotFoundError(alias: string) { - return SavedObjectsErrorHelpers.decorateIndexAliasNotFoundError(Boom.internal(), alias); - } - - public static decorateIndexAliasNotFoundError(error: Error, alias: string) { - return decorate( - error, - CODE_GENERAL_ERROR, - 500, - `Saved object index alias [${alias}] not found` - ); - } - public static isNotFoundError(error: Error | DecoratedError) { return isSavedObjectsClientError(error) && error[code] === CODE_NOT_FOUND; } @@ -198,8 +185,4 @@ export class SavedObjectsErrorHelpers { public static decorateGeneralError(error: Error, reason?: string) { return decorate(error, CODE_GENERAL_ERROR, 500, reason); } - - public static isGeneralError(error: Error | DecoratedError) { - return isSavedObjectsClientError(error) && error[code] === CODE_GENERAL_ERROR; - } } diff --git a/src/core/server/saved_objects/service/lib/repository.test.js b/src/core/server/saved_objects/service/lib/repository.test.js index 68fdea0f9eb25e..216e1c4bd2d3c9 100644 --- a/src/core/server/saved_objects/service/lib/repository.test.js +++ b/src/core/server/saved_objects/service/lib/repository.test.js @@ -18,7 +18,6 @@ import { DocumentMigrator } from '../../migrations/core/document_migrator'; import { mockKibanaMigrator } from '../../migrations/kibana/kibana_migrator.mock'; import { elasticsearchClientMock } from '../../../elasticsearch/client/mocks'; import { esKuery } from '../../es_query'; -import { errors as EsErrors } from '@elastic/elasticsearch'; const { nodeTypes } = esKuery; jest.mock('./search_dsl/search_dsl', () => ({ getSearchDsl: jest.fn() })); @@ -4342,14 +4341,8 @@ describe('SavedObjectsRepository', () => { }); it(`throws when ES is unable to find the document during update`, async () => { - const notFoundError = new EsErrors.ResponseError( - elasticsearchClientMock.createApiResponse({ - statusCode: 404, - body: { error: { type: 'es_type', reason: 'es_reason' } }, - }) - ); client.update.mockResolvedValueOnce( - elasticsearchClientMock.createErrorTransportRequestPromise(notFoundError) + elasticsearchClientMock.createSuccessTransportRequestPromise({}, { statusCode: 404 }) ); await expectNotFoundError(type, id); expect(client.update).toHaveBeenCalledTimes(1); diff --git a/src/core/server/saved_objects/service/lib/repository.ts b/src/core/server/saved_objects/service/lib/repository.ts index da80971635a93b..2993d4234bd2e8 100644 --- a/src/core/server/saved_objects/service/lib/repository.ts +++ b/src/core/server/saved_objects/service/lib/repository.ts @@ -299,7 +299,6 @@ export class SavedObjectsRepository { refresh, body: raw._source, ...(overwrite && version ? decodeRequestVersion(version) : {}), - require_alias: true, }; const { body } = @@ -470,7 +469,6 @@ export class SavedObjectsRepository { const bulkResponse = bulkCreateParams.length ? await this.client.bulk({ refresh, - require_alias: true, body: bulkCreateParams, }) : undefined; @@ -1119,8 +1117,8 @@ export class SavedObjectsRepository { ...(Array.isArray(references) && { references }), }; - const { body } = await this.client - .update({ + const { body, statusCode } = await this.client.update( + { id: this._serializer.generateRawId(namespace, type, id), index: this.getIndexForType(type), ...getExpectedVersionProperties(version, preflightResult), @@ -1130,15 +1128,14 @@ export class SavedObjectsRepository { doc, }, _source_includes: ['namespace', 'namespaces', 'originId'], - require_alias: true, - }) - .catch((err) => { - if (SavedObjectsErrorHelpers.isNotFoundError(err)) { - // see "404s from missing index" above - throw SavedObjectsErrorHelpers.createGenericNotFoundError(type, id); - } - throw err; - }); + }, + { ignore: [404] } + ); + + if (statusCode === 404) { + // see "404s from missing index" above + throw SavedObjectsErrorHelpers.createGenericNotFoundError(type, id); + } const { originId } = body.get._source; let namespaces = []; @@ -1499,7 +1496,6 @@ export class SavedObjectsRepository { refresh, body: bulkUpdateParams, _source_includes: ['originId'], - require_alias: true, }) : undefined; @@ -1716,7 +1712,6 @@ export class SavedObjectsRepository { id: raw._id, index: this.getIndexForType(type), refresh, - require_alias: true, _source: 'true', body: { script: { @@ -1938,18 +1933,12 @@ export class SavedObjectsRepository { } } -function getBulkOperationError( - error: { type: string; reason?: string; index?: string }, - type: string, - id: string -) { +function getBulkOperationError(error: { type: string; reason?: string }, type: string, id: string) { switch (error.type) { case 'version_conflict_engine_exception': return errorContent(SavedObjectsErrorHelpers.createConflictError(type, id)); case 'document_missing_exception': return errorContent(SavedObjectsErrorHelpers.createGenericNotFoundError(type, id)); - case 'index_not_found_exception': - return errorContent(SavedObjectsErrorHelpers.createIndexAliasNotFoundError(error.index!)); default: return { message: error.reason || JSON.stringify(error), diff --git a/src/core/server/server.api.md b/src/core/server/server.api.md index f3191c5625f8d9..40a12290be31b8 100644 --- a/src/core/server/server.api.md +++ b/src/core/server/server.api.md @@ -2335,8 +2335,6 @@ export class SavedObjectsErrorHelpers { // (undocumented) static createGenericNotFoundError(type?: string | null, id?: string | null): DecoratedError; // (undocumented) - static createIndexAliasNotFoundError(alias: string): DecoratedError; - // (undocumented) static createInvalidVersionError(versionInput?: string): DecoratedError; // (undocumented) static createTooManyRequestsError(type: string, id: string): DecoratedError; @@ -2355,8 +2353,6 @@ export class SavedObjectsErrorHelpers { // (undocumented) static decorateGeneralError(error: Error, reason?: string): DecoratedError; // (undocumented) - static decorateIndexAliasNotFoundError(error: Error, alias: string): DecoratedError; - // (undocumented) static decorateNotAuthorizedError(error: Error, reason?: string): DecoratedError; // (undocumented) static decorateRequestEntityTooLargeError(error: Error, reason?: string): DecoratedError; @@ -2373,8 +2369,6 @@ export class SavedObjectsErrorHelpers { // (undocumented) static isForbiddenError(error: Error | DecoratedError): boolean; // (undocumented) - static isGeneralError(error: Error | DecoratedError): boolean; - // (undocumented) static isInvalidVersionError(error: Error | DecoratedError): boolean; // (undocumented) static isNotAuthorizedError(error: Error | DecoratedError): boolean; diff --git a/src/core/server/ui_settings/integration_tests/doc_exists.ts b/src/core/server/ui_settings/integration_tests/doc_exists.ts index d100b89af9609d..aa6f98ddf2d03a 100644 --- a/src/core/server/ui_settings/integration_tests/doc_exists.ts +++ b/src/core/server/ui_settings/integration_tests/doc_exists.ts @@ -8,7 +8,7 @@ import { getServices, chance } from './lib'; -export const docExistsSuite = (savedObjectsIndex: string) => () => { +export function docExistsSuite() { async function setup(options: any = {}) { const { initialSettings } = options; @@ -16,7 +16,7 @@ export const docExistsSuite = (savedObjectsIndex: string) => () => { // delete the kibana index to ensure we start fresh await callCluster('deleteByQuery', { - index: savedObjectsIndex, + index: kbnServer.config.get('kibana.index'), body: { conflicts: 'proceed', query: { match_all: {} }, @@ -212,4 +212,4 @@ export const docExistsSuite = (savedObjectsIndex: string) => () => { }); }); }); -}; +} diff --git a/src/core/server/ui_settings/integration_tests/doc_missing.ts b/src/core/server/ui_settings/integration_tests/doc_missing.ts index 822ffe398b87d1..501976e3823f1d 100644 --- a/src/core/server/ui_settings/integration_tests/doc_missing.ts +++ b/src/core/server/ui_settings/integration_tests/doc_missing.ts @@ -8,7 +8,7 @@ import { getServices, chance } from './lib'; -export const docMissingSuite = (savedObjectsIndex: string) => () => { +export function docMissingSuite() { // ensure the kibana index has no documents beforeEach(async () => { const { kbnServer, callCluster } = getServices(); @@ -22,7 +22,7 @@ export const docMissingSuite = (savedObjectsIndex: string) => () => { // delete all docs from kibana index to ensure savedConfig is not found await callCluster('deleteByQuery', { - index: savedObjectsIndex, + index: kbnServer.config.get('kibana.index'), body: { query: { match_all: {} }, }, @@ -136,4 +136,4 @@ export const docMissingSuite = (savedObjectsIndex: string) => () => { }); }); }); -}; +} diff --git a/src/core/server/ui_settings/integration_tests/doc_missing_and_index_read_only.ts b/src/core/server/ui_settings/integration_tests/doc_missing_and_index_read_only.ts index 997d51e36abdc0..b2ff1b2f1d4ab8 100644 --- a/src/core/server/ui_settings/integration_tests/doc_missing_and_index_read_only.ts +++ b/src/core/server/ui_settings/integration_tests/doc_missing_and_index_read_only.ts @@ -8,7 +8,7 @@ import { getServices, chance } from './lib'; -export const docMissingAndIndexReadOnlySuite = (savedObjectsIndex: string) => () => { +export function docMissingAndIndexReadOnlySuite() { // ensure the kibana index has no documents beforeEach(async () => { const { kbnServer, callCluster } = getServices(); @@ -22,7 +22,7 @@ export const docMissingAndIndexReadOnlySuite = (savedObjectsIndex: string) => () // delete all docs from kibana index to ensure savedConfig is not found await callCluster('deleteByQuery', { - index: savedObjectsIndex, + index: kbnServer.config.get('kibana.index'), body: { query: { match_all: {} }, }, @@ -30,7 +30,7 @@ export const docMissingAndIndexReadOnlySuite = (savedObjectsIndex: string) => () // set the index to read only await callCluster('indices.putSettings', { - index: savedObjectsIndex, + index: kbnServer.config.get('kibana.index'), body: { index: { blocks: { @@ -42,11 +42,11 @@ export const docMissingAndIndexReadOnlySuite = (savedObjectsIndex: string) => () }); afterEach(async () => { - const { callCluster } = getServices(); + const { kbnServer, callCluster } = getServices(); // disable the read only block await callCluster('indices.putSettings', { - index: savedObjectsIndex, + index: kbnServer.config.get('kibana.index'), body: { index: { blocks: { @@ -142,4 +142,4 @@ export const docMissingAndIndexReadOnlySuite = (savedObjectsIndex: string) => () }); }); }); -}; +} diff --git a/src/core/server/ui_settings/integration_tests/index.test.ts b/src/core/server/ui_settings/integration_tests/index.test.ts index e27e6c4e468742..f415f1d73de7d7 100644 --- a/src/core/server/ui_settings/integration_tests/index.test.ts +++ b/src/core/server/ui_settings/integration_tests/index.test.ts @@ -6,25 +6,20 @@ * Public License, v 1. */ -import { Env } from '@kbn/config'; -import { REPO_ROOT } from '@kbn/dev-utils'; -import { getEnvOptions } from '@kbn/config/target/mocks'; import { startServers, stopServers } from './lib'; + import { docExistsSuite } from './doc_exists'; import { docMissingSuite } from './doc_missing'; import { docMissingAndIndexReadOnlySuite } from './doc_missing_and_index_read_only'; -const kibanaVersion = Env.createDefault(REPO_ROOT, getEnvOptions()).packageInfo.version; -const savedObjectIndex = `.kibana_${kibanaVersion}_001`; - describe('uiSettings/routes', function () { jest.setTimeout(10000); beforeAll(startServers); /* eslint-disable jest/valid-describe */ - describe('doc missing', docMissingSuite(savedObjectIndex)); - describe('doc missing and index readonly', docMissingAndIndexReadOnlySuite(savedObjectIndex)); - describe('doc exists', docExistsSuite(savedObjectIndex)); + describe('doc missing', docMissingSuite); + describe('doc missing and index readonly', docMissingAndIndexReadOnlySuite); + describe('doc exists', docExistsSuite); /* eslint-enable jest/valid-describe */ afterAll(stopServers); }); diff --git a/src/core/server/ui_settings/integration_tests/lib/servers.ts b/src/core/server/ui_settings/integration_tests/lib/servers.ts index f181272030ae1d..b5198b19007d05 100644 --- a/src/core/server/ui_settings/integration_tests/lib/servers.ts +++ b/src/core/server/ui_settings/integration_tests/lib/servers.ts @@ -37,6 +37,9 @@ export async function startServers() { adjustTimeout: (t) => jest.setTimeout(t), settings: { kbn: { + migrations: { + enableV2: false, + }, uiSettings: { overrides: { foo: 'bar', diff --git a/src/core/test_helpers/kbn_server.ts b/src/core/test_helpers/kbn_server.ts index 6fe6819a0981a4..0007e1fcca0a51 100644 --- a/src/core/test_helpers/kbn_server.ts +++ b/src/core/test_helpers/kbn_server.ts @@ -40,7 +40,7 @@ const DEFAULTS_SETTINGS = { }, logging: { silent: true }, plugins: {}, - migrations: { skip: false }, + migrations: { skip: true }, }; const DEFAULT_SETTINGS_WITH_CORE_PLUGINS = { diff --git a/src/dev/code_coverage/ingest_coverage/integration_tests/team_assignment.test.js b/src/dev/code_coverage/ingest_coverage/integration_tests/team_assignment.test.js index bf4c55871598e8..13ef28def40802 100644 --- a/src/dev/code_coverage/ingest_coverage/integration_tests/team_assignment.test.js +++ b/src/dev/code_coverage/ingest_coverage/integration_tests/team_assignment.test.js @@ -8,7 +8,6 @@ import { resolve } from 'path'; import execa from 'execa'; -import expect from '@kbn/expect'; import shell from 'shelljs'; const ROOT_DIR = resolve(__dirname, '../../../../..'); @@ -38,11 +37,14 @@ describe('Team Assignment', () => { describe(`when the codeowners file contains #CC#`, () => { it(`should strip the prefix and still drill down through the fs`, async () => { const { stdout } = await execa('grep', ['tre', teamAssignmentsPath], { cwd: ROOT_DIR }); - expect(stdout).to.be(`x-pack/plugins/code/jest.config.js kibana-tre -x-pack/plugins/code/server/config.ts kibana-tre -x-pack/plugins/code/server/index.ts kibana-tre -x-pack/plugins/code/server/plugin.test.ts kibana-tre -x-pack/plugins/code/server/plugin.ts kibana-tre`); + const lines = stdout.split('\n').filter((line) => !line.includes('/target')); + expect(lines).toEqual([ + 'x-pack/plugins/code/jest.config.js kibana-tre', + 'x-pack/plugins/code/server/config.ts kibana-tre', + 'x-pack/plugins/code/server/index.ts kibana-tre', + 'x-pack/plugins/code/server/plugin.test.ts kibana-tre', + 'x-pack/plugins/code/server/plugin.ts kibana-tre', + ]); }); }); }); diff --git a/src/plugins/data/common/search/types.ts b/src/plugins/data/common/search/types.ts index c1293f44154583..38e963591f25ce 100644 --- a/src/plugins/data/common/search/types.ts +++ b/src/plugins/data/common/search/types.ts @@ -84,11 +84,18 @@ export interface ISearchOptions { * An `AbortSignal` that allows the caller of `search` to abort a search request. */ abortSignal?: AbortSignal; + /** * Use this option to force using a specific server side search strategy. Leave empty to use the default strategy. */ strategy?: string; + /** + * Request the legacy format for the total number of hits. If sending `rest_total_hits_as_int` to + * something other than `true`, this should be set to `false`. + */ + legacyHitsTotal?: boolean; + /** * A session ID, grouping multiple search requests into a single session. */ diff --git a/src/plugins/data/public/public.api.md b/src/plugins/data/public/public.api.md index f533af2db9672e..4f197dd43a83ef 100644 --- a/src/plugins/data/public/public.api.md +++ b/src/plugins/data/public/public.api.md @@ -1641,6 +1641,7 @@ export interface ISearchOptions { abortSignal?: AbortSignal; isRestore?: boolean; isStored?: boolean; + legacyHitsTotal?: boolean; sessionId?: string; strategy?: string; } diff --git a/src/plugins/data/server/index.ts b/src/plugins/data/server/index.ts index 27af11674d061e..370ff180fa5627 100644 --- a/src/plugins/data/server/index.ts +++ b/src/plugins/data/server/index.ts @@ -235,6 +235,7 @@ export { SearchUsage, SessionService, ISessionService, + IScopedSessionService, DataApiRequestHandlerContext, DataRequestHandlerContext, } from './search'; diff --git a/src/plugins/data/server/search/es_search/es_search_strategy.ts b/src/plugins/data/server/search/es_search/es_search_strategy.ts index c176a50627b92e..2d9b16ac8b00bf 100644 --- a/src/plugins/data/server/search/es_search/es_search_strategy.ts +++ b/src/plugins/data/server/search/es_search/es_search_strategy.ts @@ -13,7 +13,7 @@ import type { Logger, SharedGlobalConfig } from 'kibana/server'; import type { ISearchStrategy } from '../types'; import type { SearchUsage } from '../collectors'; import { getDefaultSearchParams, getShardTimeout, shimAbortSignal } from './request_utils'; -import { toKibanaSearchResponse } from './response_utils'; +import { shimHitsTotal, toKibanaSearchResponse } from './response_utils'; import { searchUsageObserver } from '../collectors/usage'; import { getKbnServerError, KbnServerError } from '../../../../kibana_utils/server'; @@ -29,7 +29,7 @@ export const esSearchStrategyProvider = ( * @throws `KbnServerError` * @returns `Observable>` */ - search: (request, { abortSignal }, { esClient, uiSettingsClient }) => { + search: (request, { abortSignal, ...options }, { esClient, uiSettingsClient }) => { // Only default index pattern type is supported here. // See data_enhanced for other type support. if (request.indexType) { @@ -46,7 +46,8 @@ export const esSearchStrategyProvider = ( }; const promise = esClient.asCurrentUser.search>(params); const { body } = await shimAbortSignal(promise, abortSignal); - return toKibanaSearchResponse(body); + const response = shimHitsTotal(body, options); + return toKibanaSearchResponse(response); } catch (e) { throw getKbnServerError(e); } diff --git a/src/plugins/data/server/search/es_search/response_utils.test.ts b/src/plugins/data/server/search/es_search/response_utils.test.ts index 8c973b92c7ffe7..7cb5705ecf8ef2 100644 --- a/src/plugins/data/server/search/es_search/response_utils.test.ts +++ b/src/plugins/data/server/search/es_search/response_utils.test.ts @@ -6,7 +6,7 @@ * Public License, v 1. */ -import { getTotalLoaded, toKibanaSearchResponse } from './response_utils'; +import { getTotalLoaded, toKibanaSearchResponse, shimHitsTotal } from './response_utils'; import { SearchResponse } from 'elasticsearch'; describe('response utils', () => { @@ -55,4 +55,79 @@ describe('response utils', () => { }); }); }); + + describe('shimHitsTotal', () => { + test('returns the total if it is already numeric', () => { + const result = shimHitsTotal({ + hits: { + total: 5, + }, + } as any); + expect(result).toEqual({ + hits: { + total: 5, + }, + }); + }); + + test('returns the total if it is inside `value`', () => { + const result = shimHitsTotal({ + hits: { + total: { + value: 5, + }, + }, + } as any); + expect(result).toEqual({ + hits: { + total: 5, + }, + }); + }); + + test('returns other properties from the response', () => { + const result = shimHitsTotal({ + _shards: {}, + hits: { + hits: [], + total: { + value: 5, + }, + }, + } as any); + expect(result).toEqual({ + _shards: {}, + hits: { + hits: [], + total: 5, + }, + }); + }); + + test('returns the response as-is if `legacyHitsTotal` is `false`', () => { + const result = shimHitsTotal( + { + _shards: {}, + hits: { + hits: [], + total: { + value: 5, + relation: 'eq', + }, + }, + } as any, + { legacyHitsTotal: false } + ); + expect(result).toEqual({ + _shards: {}, + hits: { + hits: [], + total: { + value: 5, + relation: 'eq', + }, + }, + }); + }); + }); }); diff --git a/src/plugins/data/server/search/es_search/response_utils.ts b/src/plugins/data/server/search/es_search/response_utils.ts index d4fa14866fd97d..3417f24cf420ac 100644 --- a/src/plugins/data/server/search/es_search/response_utils.ts +++ b/src/plugins/data/server/search/es_search/response_utils.ts @@ -7,6 +7,7 @@ */ import { SearchResponse } from 'elasticsearch'; +import { ISearchOptions } from '../../../common'; /** * Get the `total`/`loaded` for this response (see `IKibanaSearchResponse`). Note that `skipped` is @@ -31,3 +32,20 @@ export function toKibanaSearchResponse(rawResponse: SearchResponse) { ...getTotalLoaded(rawResponse), }; } + +/** + * Temporary workaround until https://github.com/elastic/kibana/issues/26356 is addressed. + * Since we are setting `track_total_hits` in the request, `hits.total` will be an object + * containing the `value`. + * + * @internal + */ +export function shimHitsTotal( + response: SearchResponse, + { legacyHitsTotal = true }: ISearchOptions = {} +) { + if (!legacyHitsTotal) return response; + const total = (response.hits?.total as any)?.value ?? response.hits?.total; + const hits = { ...response.hits, total }; + return { ...response, hits }; +} diff --git a/src/plugins/data/server/search/index.ts b/src/plugins/data/server/search/index.ts index a3334241100655..301b0989b51839 100644 --- a/src/plugins/data/server/search/index.ts +++ b/src/plugins/data/server/search/index.ts @@ -10,5 +10,4 @@ export * from './types'; export * from './es_search'; export { usageProvider, SearchUsage, searchUsageObserver } from './collectors'; export * from './aggs'; -export { shimHitsTotal } from './routes'; export * from './session'; diff --git a/src/plugins/data/server/search/routes/bsearch.ts b/src/plugins/data/server/search/routes/bsearch.ts index e30b7bdaa84022..ba96726b787c02 100644 --- a/src/plugins/data/server/search/routes/bsearch.ts +++ b/src/plugins/data/server/search/routes/bsearch.ts @@ -6,7 +6,7 @@ * Public License, v 1. */ -import { catchError, first, map } from 'rxjs/operators'; +import { catchError, first } from 'rxjs/operators'; import { CoreStart, KibanaRequest } from 'src/core/server'; import { BfetchServerSetup } from 'src/plugins/bfetch/server'; import { @@ -15,7 +15,6 @@ import { ISearchClient, ISearchOptions, } from '../../../common/search'; -import { shimHitsTotal } from './shim_hits_total'; type GetScopedProider = (coreStart: CoreStart) => (request: KibanaRequest) => ISearchClient; @@ -40,14 +39,6 @@ export function registerBsearchRoute( .search(requestData, options) .pipe( first(), - map((response) => { - return { - ...response, - ...{ - rawResponse: shimHitsTotal(response.rawResponse), - }, - }; - }), catchError((err) => { // Re-throw as object, to get attributes passed to the client // eslint-disable-next-line no-throw-literal diff --git a/src/plugins/data/server/search/routes/call_msearch.ts b/src/plugins/data/server/search/routes/call_msearch.ts index fc30e2f29c3ef2..e6ff5f454079b2 100644 --- a/src/plugins/data/server/search/routes/call_msearch.ts +++ b/src/plugins/data/server/search/routes/call_msearch.ts @@ -12,9 +12,8 @@ import { SearchResponse } from 'elasticsearch'; import { IUiSettingsClient, IScopedClusterClient, SharedGlobalConfig } from 'src/core/server'; import type { MsearchRequestBody, MsearchResponse } from '../../../common/search/search_source'; -import { shimHitsTotal } from './shim_hits_total'; import { getKbnServerError } from '../../../../kibana_utils/server'; -import { getShardTimeout, getDefaultSearchParams, shimAbortSignal } from '..'; +import { getShardTimeout, getDefaultSearchParams, shimAbortSignal, shimHitsTotal } from '..'; /** @internal */ export function convertRequestBody( diff --git a/src/plugins/data/server/search/routes/index.ts b/src/plugins/data/server/search/routes/index.ts index ea20240f6ae19b..25e0353fb4a27a 100644 --- a/src/plugins/data/server/search/routes/index.ts +++ b/src/plugins/data/server/search/routes/index.ts @@ -9,4 +9,3 @@ export * from './call_msearch'; export * from './msearch'; export * from './search'; -export * from './shim_hits_total'; diff --git a/src/plugins/data/server/search/routes/search.ts b/src/plugins/data/server/search/routes/search.ts index 6d2da4c1e63ddb..e556e3ca49ec29 100644 --- a/src/plugins/data/server/search/routes/search.ts +++ b/src/plugins/data/server/search/routes/search.ts @@ -9,7 +9,6 @@ import { first } from 'rxjs/operators'; import { schema } from '@kbn/config-schema'; import { getRequestAbortedSignal } from '../../lib'; -import { shimHitsTotal } from './shim_hits_total'; import { reportServerError } from '../../../../kibana_utils/server'; import type { DataPluginRouter } from '../types'; @@ -27,6 +26,7 @@ export function registerSearchRoute(router: DataPluginRouter): void { body: schema.object( { + legacyHitsTotal: schema.maybe(schema.boolean()), sessionId: schema.maybe(schema.string()), isStored: schema.maybe(schema.boolean()), isRestore: schema.maybe(schema.boolean()), @@ -36,7 +36,13 @@ export function registerSearchRoute(router: DataPluginRouter): void { }, }, async (context, request, res) => { - const { sessionId, isStored, isRestore, ...searchRequest } = request.body; + const { + legacyHitsTotal = true, + sessionId, + isStored, + isRestore, + ...searchRequest + } = request.body; const { strategy, id } = request.params; const abortSignal = getRequestAbortedSignal(request.events.aborted$); @@ -47,6 +53,7 @@ export function registerSearchRoute(router: DataPluginRouter): void { { abortSignal, strategy, + legacyHitsTotal, sessionId, isStored, isRestore, @@ -55,14 +62,7 @@ export function registerSearchRoute(router: DataPluginRouter): void { .pipe(first()) .toPromise(); - return res.ok({ - body: { - ...response, - ...{ - rawResponse: shimHitsTotal(response.rawResponse), - }, - }, - }); + return res.ok({ body: response }); } catch (err) { return reportServerError(res, err); } diff --git a/src/plugins/data/server/search/routes/shim_hits_total.test.ts b/src/plugins/data/server/search/routes/shim_hits_total.test.ts deleted file mode 100644 index 6dcd7c3ff6c70c..00000000000000 --- a/src/plugins/data/server/search/routes/shim_hits_total.test.ts +++ /dev/null @@ -1,58 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License - * and the Server Side Public License, v 1; you may not use this file except in - * compliance with, at your election, the Elastic License or the Server Side - * Public License, v 1. - */ - -import { shimHitsTotal } from './shim_hits_total'; - -describe('shimHitsTotal', () => { - test('returns the total if it is already numeric', () => { - const result = shimHitsTotal({ - hits: { - total: 5, - }, - } as any); - expect(result).toEqual({ - hits: { - total: 5, - }, - }); - }); - - test('returns the total if it is inside `value`', () => { - const result = shimHitsTotal({ - hits: { - total: { - value: 5, - }, - }, - } as any); - expect(result).toEqual({ - hits: { - total: 5, - }, - }); - }); - - test('returns other properties from the response', () => { - const result = shimHitsTotal({ - _shards: {}, - hits: { - hits: [], - total: { - value: 5, - }, - }, - } as any); - expect(result).toEqual({ - _shards: {}, - hits: { - hits: [], - total: 5, - }, - }); - }); -}); diff --git a/src/plugins/data/server/search/routes/shim_hits_total.ts b/src/plugins/data/server/search/routes/shim_hits_total.ts deleted file mode 100644 index 4b56d6394e0dbc..00000000000000 --- a/src/plugins/data/server/search/routes/shim_hits_total.ts +++ /dev/null @@ -1,22 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License - * and the Server Side Public License, v 1; you may not use this file except in - * compliance with, at your election, the Elastic License or the Server Side - * Public License, v 1. - */ - -import { SearchResponse } from 'elasticsearch'; - -/** - * Temporary workaround until https://github.com/elastic/kibana/issues/26356 is addressed. - * Since we are setting `track_total_hits` in the request, `hits.total` will be an object - * containing the `value`. - * - * @internal - */ -export function shimHitsTotal(response: SearchResponse) { - const total = (response.hits?.total as any)?.value ?? response.hits?.total; - const hits = { ...response.hits, total }; - return { ...response, hits }; -} diff --git a/src/plugins/data/server/search/search_service.ts b/src/plugins/data/server/search/search_service.ts index 63593bbe84a088..e9f0edbd4d6c47 100644 --- a/src/plugins/data/server/search/search_service.ts +++ b/src/plugins/data/server/search/search_service.ts @@ -34,7 +34,7 @@ import { AggsService } from './aggs'; import { FieldFormatsStart } from '../field_formats'; import { IndexPatternsServiceStart } from '../index_patterns'; -import { getCallMsearch, registerMsearchRoute, registerSearchRoute, shimHitsTotal } from './routes'; +import { getCallMsearch, registerMsearchRoute, registerSearchRoute } from './routes'; import { ES_SEARCH_STRATEGY, esSearchStrategyProvider } from './es_search'; import { DataPluginStart } from '../plugin'; import { UsageCollectionSetup } from '../../../usage_collection/server'; @@ -62,7 +62,7 @@ import { } from '../../common/search/aggs/buckets/shard_delay'; import { aggShardDelay } from '../../common/search/aggs/buckets/shard_delay_fn'; import { ConfigSchema } from '../../config'; -import { SessionService, IScopedSessionService, ISessionService } from './session'; +import { IScopedSessionService, ISessionService, SessionService } from './session'; import { KbnServerError } from '../../../kibana_utils/server'; import { registerBsearchRoute } from './routes/bsearch'; @@ -209,7 +209,7 @@ export class SearchService implements Plugin { const searchSourceDependencies: SearchSourceDependencies = { getConfig: (key: string): T => uiSettingsCache[key], search: asScoped(request).search, - onResponse: (req, res) => shimHitsTotal(res), + onResponse: (req, res) => res, legacy: { callMsearch: getCallMsearch({ esClient, diff --git a/src/plugins/data/server/server.api.md b/src/plugins/data/server/server.api.md index 9789f3354e9efb..635428f298ab2b 100644 --- a/src/plugins/data/server/server.api.md +++ b/src/plugins/data/server/server.api.md @@ -310,8 +310,6 @@ export const config: PluginConfigDescriptor; // // @public (undocumented) export interface DataApiRequestHandlerContext extends ISearchClient { - // Warning: (ae-forgotten-export) The symbol "IScopedSessionService" needs to be exported by the entry point index.d.ts - // // (undocumented) session: IScopedSessionService; } @@ -912,6 +910,16 @@ export class IndexPatternsService implements Plugin_3(strategy: ISearchStrategy, ...args: Parameters['search']>) => Observable; +} + // Warning: (ae-missing-release-tag) "ISearchOptions" is exported by the package, but it is missing a release tag (@alpha, @beta, @public, or @internal) // // @public (undocumented) @@ -919,6 +927,7 @@ export interface ISearchOptions { abortSignal?: AbortSignal; isRestore?: boolean; isStored?: boolean; + legacyHitsTotal?: boolean; sessionId?: string; strategy?: string; } @@ -1284,7 +1293,7 @@ export class SessionService implements ISessionService { export const shimAbortSignal: (promise: TransportRequestPromise, signal?: AbortSignal | undefined) => TransportRequestPromise; // @internal -export function shimHitsTotal(response: SearchResponse): { +export function shimHitsTotal(response: SearchResponse, { legacyHitsTotal }?: ISearchOptions): { hits: { total: any; max_score: number; @@ -1293,7 +1302,7 @@ export function shimHitsTotal(response: SearchResponse): { _type: string; _id: string; _score: number; - _source: any; + _source: unknown; _version?: number | undefined; _explanation?: import("elasticsearch").Explanation | undefined; fields?: any; @@ -1426,20 +1435,20 @@ export function usageProvider(core: CoreSetup_2): SearchUsage; // src/plugins/data/server/index.ts:100:26 - (ae-forgotten-export) The symbol "TruncateFormat" needs to be exported by the entry point index.d.ts // src/plugins/data/server/index.ts:126:27 - (ae-forgotten-export) The symbol "isFilterable" needs to be exported by the entry point index.d.ts // src/plugins/data/server/index.ts:126:27 - (ae-forgotten-export) The symbol "isNestedField" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:243:20 - (ae-forgotten-export) The symbol "getRequestInspectorStats" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:243:20 - (ae-forgotten-export) The symbol "getResponseInspectorStats" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:243:20 - (ae-forgotten-export) The symbol "tabifyAggResponse" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:243:20 - (ae-forgotten-export) The symbol "tabifyGetColumns" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:245:1 - (ae-forgotten-export) The symbol "CidrMask" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:246:1 - (ae-forgotten-export) The symbol "dateHistogramInterval" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:255:1 - (ae-forgotten-export) The symbol "InvalidEsCalendarIntervalError" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:256:1 - (ae-forgotten-export) The symbol "InvalidEsIntervalFormatError" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:257:1 - (ae-forgotten-export) The symbol "Ipv4Address" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:261:1 - (ae-forgotten-export) The symbol "isValidEsInterval" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:262:1 - (ae-forgotten-export) The symbol "isValidInterval" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:266:1 - (ae-forgotten-export) The symbol "propFilter" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:269:1 - (ae-forgotten-export) The symbol "toAbsoluteDates" needs to be exported by the entry point index.d.ts -// src/plugins/data/server/index.ts:270:1 - (ae-forgotten-export) The symbol "calcAutoIntervalLessThan" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:244:20 - (ae-forgotten-export) The symbol "getRequestInspectorStats" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:244:20 - (ae-forgotten-export) The symbol "getResponseInspectorStats" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:244:20 - (ae-forgotten-export) The symbol "tabifyAggResponse" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:244:20 - (ae-forgotten-export) The symbol "tabifyGetColumns" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:246:1 - (ae-forgotten-export) The symbol "CidrMask" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:247:1 - (ae-forgotten-export) The symbol "dateHistogramInterval" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:256:1 - (ae-forgotten-export) The symbol "InvalidEsCalendarIntervalError" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:257:1 - (ae-forgotten-export) The symbol "InvalidEsIntervalFormatError" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:258:1 - (ae-forgotten-export) The symbol "Ipv4Address" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:262:1 - (ae-forgotten-export) The symbol "isValidEsInterval" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:263:1 - (ae-forgotten-export) The symbol "isValidInterval" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:267:1 - (ae-forgotten-export) The symbol "propFilter" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:270:1 - (ae-forgotten-export) The symbol "toAbsoluteDates" needs to be exported by the entry point index.d.ts +// src/plugins/data/server/index.ts:271:1 - (ae-forgotten-export) The symbol "calcAutoIntervalLessThan" needs to be exported by the entry point index.d.ts // src/plugins/data/server/index_patterns/index_patterns_service.ts:59:14 - (ae-forgotten-export) The symbol "IndexPatternsService" needs to be exported by the entry point index.d.ts // src/plugins/data/server/plugin.ts:79:74 - (ae-forgotten-export) The symbol "DataEnhancements" needs to be exported by the entry point index.d.ts // src/plugins/data/server/search/types.ts:103:5 - (ae-forgotten-export) The symbol "ISearchStartSearchSource" needs to be exported by the entry point index.d.ts diff --git a/src/plugins/es_ui_shared/static/forms/hook_form_lib/components/use_field.test.tsx b/src/plugins/es_ui_shared/static/forms/hook_form_lib/components/use_field.test.tsx index 36841f8b1d521e..4945b7a059e8c0 100644 --- a/src/plugins/es_ui_shared/static/forms/hook_form_lib/components/use_field.test.tsx +++ b/src/plugins/es_ui_shared/static/forms/hook_form_lib/components/use_field.test.tsx @@ -287,4 +287,55 @@ describe('', () => { expect(formHook?.getFormData()).toEqual({ name: 'myName' }); }); }); + + describe('change handlers', () => { + const onError = jest.fn(); + + beforeEach(() => { + jest.resetAllMocks(); + }); + + const getTestComp = (fieldConfig: FieldConfig) => { + const TestComp = () => { + const { form } = useForm(); + + return ( +
+ + + ); + }; + return TestComp; + }; + + const setup = (fieldConfig: FieldConfig) => { + return registerTestBed(getTestComp(fieldConfig), { + memoryRouter: { wrapComponent: false }, + })() as TestBed; + }; + + it('calls onError when validation state changes', async () => { + const { + form: { setInputValue }, + } = setup({ + validations: [ + { + validator: ({ value }) => (value === '1' ? undefined : { message: 'oops!' }), + }, + ], + }); + + expect(onError).toBeCalledTimes(0); + await act(async () => { + setInputValue('myField', '0'); + }); + expect(onError).toBeCalledTimes(1); + expect(onError).toBeCalledWith(['oops!']); + await act(async () => { + setInputValue('myField', '1'); + }); + expect(onError).toBeCalledTimes(2); + expect(onError).toBeCalledWith(null); + }); + }); }); diff --git a/src/plugins/es_ui_shared/static/forms/hook_form_lib/components/use_field.tsx b/src/plugins/es_ui_shared/static/forms/hook_form_lib/components/use_field.tsx index 94c2bc42d28558..cc79ed24b5d0cb 100644 --- a/src/plugins/es_ui_shared/static/forms/hook_form_lib/components/use_field.tsx +++ b/src/plugins/es_ui_shared/static/forms/hook_form_lib/components/use_field.tsx @@ -20,6 +20,7 @@ export interface Props { componentProps?: Record; readDefaultValueOnForm?: boolean; onChange?: (value: I) => void; + onError?: (errors: string[] | null) => void; children?: (field: FieldHook) => JSX.Element | null; [key: string]: any; } @@ -33,6 +34,7 @@ function UseFieldComp(props: Props(props: Props(form, path, fieldConfig, onChange); + const field = useField(form, path, fieldConfig, onChange, onError); // Children prevails over anything else provided. if (children) { diff --git a/src/plugins/es_ui_shared/static/forms/hook_form_lib/hooks/use_field.ts b/src/plugins/es_ui_shared/static/forms/hook_form_lib/hooks/use_field.ts index c396f223e97fde..db7b0b2820a47d 100644 --- a/src/plugins/es_ui_shared/static/forms/hook_form_lib/hooks/use_field.ts +++ b/src/plugins/es_ui_shared/static/forms/hook_form_lib/hooks/use_field.ts @@ -27,7 +27,8 @@ export const useField = ( form: FormHook, path: string, config: FieldConfig & InternalFieldConfig = {}, - valueChangeListener?: (value: I) => void + valueChangeListener?: (value: I) => void, + errorChangeListener?: (errors: string[] | null) => void ) => { const { type = FIELD_TYPES.TEXT, @@ -596,6 +597,15 @@ export const useField = ( }; }, [onValueChange]); + useEffect(() => { + if (!isMounted.current) { + return; + } + if (errorChangeListener) { + errorChangeListener(errors.length ? errors.map((error) => error.message) : null); + } + }, [errors, errorChangeListener]); + useEffect(() => { isMounted.current = true; diff --git a/src/plugins/kibana_overview/tsconfig.json b/src/plugins/kibana_overview/tsconfig.json new file mode 100644 index 00000000000000..ac3ac109cb35f0 --- /dev/null +++ b/src/plugins/kibana_overview/tsconfig.json @@ -0,0 +1,23 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "composite": true, + "outDir": "./target/types", + "emitDeclarationOnly": true, + "declaration": true, + "declarationMap": true + }, + "include": [ + "public/**/*", + "common/**/*", + ], + "references": [ + { "path": "../../core/tsconfig.json" }, + { "path": "../../plugins/navigation/tsconfig.json" }, + { "path": "../../plugins/data/tsconfig.json" }, + { "path": "../../plugins/home/tsconfig.json" }, + { "path": "../../plugins/newsfeed/tsconfig.json" }, + { "path": "../../plugins/usage_collection/tsconfig.json" }, + { "path": "../../plugins/kibana_react/tsconfig.json" }, + ] +} diff --git a/src/plugins/vis_type_table/public/components/table_vis_basic.tsx b/src/plugins/vis_type_table/public/components/table_vis_basic.tsx index 6cea4d09c4e7ff..8bb5186159b7d5 100644 --- a/src/plugins/vis_type_table/public/components/table_vis_basic.tsx +++ b/src/plugins/vis_type_table/public/components/table_vis_basic.tsx @@ -13,15 +13,14 @@ import { orderBy } from 'lodash'; import { IInterpreterRenderHandlers } from 'src/plugins/expressions'; import { createTableVisCell } from './table_vis_cell'; -import { Table } from '../table_vis_response_handler'; -import { TableVisConfig, TableVisUseUiStateProps } from '../types'; -import { useFormattedColumnsAndRows, usePagination } from '../utils'; +import { TableContext, TableVisConfig, TableVisUseUiStateProps } from '../types'; +import { usePagination } from '../utils'; import { TableVisControls } from './table_vis_controls'; import { createGridColumns } from './table_vis_columns'; interface TableVisBasicProps { fireEvent: IInterpreterRenderHandlers['event']; - table: Table; + table: TableContext; visConfig: TableVisConfig; title?: string; uiStateProps: TableVisUseUiStateProps; @@ -35,7 +34,7 @@ export const TableVisBasic = memo( title, uiStateProps: { columnsWidth, sort, setColumnsWidth, setSort }, }: TableVisBasicProps) => { - const { columns, rows } = useFormattedColumnsAndRows(table, visConfig); + const { columns, rows, formattedColumns } = table; // custom sorting is in place until the EuiDataGrid sorting gets rid of flaws -> https://github.com/elastic/eui/issues/4108 const sortedRows = useMemo( @@ -47,13 +46,19 @@ export const TableVisBasic = memo( ); // renderCellValue is a component which renders a cell based on column and row indexes - const renderCellValue = useMemo(() => createTableVisCell(columns, sortedRows), [ - columns, + const renderCellValue = useMemo(() => createTableVisCell(sortedRows, formattedColumns), [ + formattedColumns, sortedRows, ]); // Columns config - const gridColumns = createGridColumns(table, columns, columnsWidth, sortedRows, fireEvent); + const gridColumns = createGridColumns( + columns, + sortedRows, + formattedColumns, + columnsWidth, + fireEvent + ); // Pagination config const pagination = usePagination(visConfig, rows.length); @@ -126,10 +131,9 @@ export const TableVisBasic = memo( additionalControls: ( ), @@ -138,8 +142,7 @@ export const TableVisBasic = memo( renderCellValue={renderCellValue} renderFooterCellValue={ visConfig.showTotal - ? // @ts-expect-error - ({ colIndex }) => columns[colIndex].formattedTotal || null + ? ({ columnId }) => formattedColumns[columnId].formattedTotal || null : undefined } pagination={pagination} diff --git a/src/plugins/vis_type_table/public/components/table_vis_cell.tsx b/src/plugins/vis_type_table/public/components/table_vis_cell.tsx index 0a6aafc84bf263..04df3907c8c9b0 100644 --- a/src/plugins/vis_type_table/public/components/table_vis_cell.tsx +++ b/src/plugins/vis_type_table/public/components/table_vis_cell.tsx @@ -9,17 +9,15 @@ import React from 'react'; import { EuiDataGridCellValueElementProps } from '@elastic/eui'; -import { Table } from '../table_vis_response_handler'; -import { FormattedColumn } from '../types'; +import { DatatableRow } from 'src/plugins/expressions'; +import { FormattedColumns } from '../types'; -export const createTableVisCell = (formattedColumns: FormattedColumn[], rows: Table['rows']) => ({ - // @ts-expect-error - colIndex, +export const createTableVisCell = (rows: DatatableRow[], formattedColumns: FormattedColumns) => ({ rowIndex, columnId, }: EuiDataGridCellValueElementProps) => { const rowValue = rows[rowIndex][columnId]; - const column = formattedColumns[colIndex]; + const column = formattedColumns[columnId]; const content = column.formatter.convert(rowValue, 'html'); const cellContent = ( diff --git a/src/plugins/vis_type_table/public/components/table_vis_columns.tsx b/src/plugins/vis_type_table/public/components/table_vis_columns.tsx index 2610677b2491c1..6b44a2504ff89c 100644 --- a/src/plugins/vis_type_table/public/components/table_vis_columns.tsx +++ b/src/plugins/vis_type_table/public/components/table_vis_columns.tsx @@ -10,9 +10,8 @@ import React from 'react'; import { EuiDataGridColumnCellActionProps, EuiDataGridColumn } from '@elastic/eui'; import { i18n } from '@kbn/i18n'; -import { IInterpreterRenderHandlers } from 'src/plugins/expressions'; -import { Table } from '../table_vis_response_handler'; -import { FormattedColumn, TableVisUiState } from '../types'; +import { DatatableColumn, DatatableRow, IInterpreterRenderHandlers } from 'src/plugins/expressions'; +import { FormattedColumns, TableVisUiState } from '../types'; interface FilterCellData { /** @@ -27,33 +26,24 @@ interface FilterCellData { } export const createGridColumns = ( - table: Table, - columns: FormattedColumn[], + columns: DatatableColumn[], + rows: DatatableRow[], + formattedColumns: FormattedColumns, columnsWidth: TableVisUiState['colWidth'], - rows: Table['rows'], fireEvent: IInterpreterRenderHandlers['event'] ) => { const onFilterClick = (data: FilterCellData, negate: boolean) => { - /** - * Visible column index and the actual one from the source table could be different. - * e.x. a column could be filtered out if it's not a dimension - - * see formattedColumns in use_formatted_columns.ts file, - * or an extra percantage column could be added, which doesn't exist in the raw table - */ - const rawTableActualColumnIndex = table.columns.findIndex( - (c) => c.id === columns[data.column].id - ); fireEvent({ name: 'filterBucket', data: { data: [ { table: { - ...table, + columns, rows, }, ...data, - column: rawTableActualColumnIndex, + column: data.column, }, ], negate, @@ -63,12 +53,13 @@ export const createGridColumns = ( return columns.map( (col, colIndex): EuiDataGridColumn => { - const cellActions = col.filterable + const formattedColumn = formattedColumns[col.id]; + const cellActions = formattedColumn.filterable ? [ ({ rowIndex, columnId, Component, closePopover }: EuiDataGridColumnCellActionProps) => { const rowValue = rows[rowIndex][columnId]; const contentsIsDefined = rowValue !== null && rowValue !== undefined; - const cellContent = col.formatter.convert(rowValue); + const cellContent = formattedColumn.formatter.convert(rowValue); const filterForText = i18n.translate( 'visTypeTable.tableCellFilter.filterForValueText', @@ -105,7 +96,7 @@ export const createGridColumns = ( ({ rowIndex, columnId, Component, closePopover }: EuiDataGridColumnCellActionProps) => { const rowValue = rows[rowIndex][columnId]; const contentsIsDefined = rowValue !== null && rowValue !== undefined; - const cellContent = col.formatter.convert(rowValue); + const cellContent = formattedColumn.formatter.convert(rowValue); const filterOutText = i18n.translate( 'visTypeTable.tableCellFilter.filterOutValueText', @@ -144,8 +135,8 @@ export const createGridColumns = ( const initialWidth = columnsWidth.find((c) => c.colIndex === colIndex); const column: EuiDataGridColumn = { id: col.id, - display: col.title, - displayAsText: col.title, + display: col.name, + displayAsText: col.name, actions: { showHide: false, showMoveLeft: false, diff --git a/src/plugins/vis_type_table/public/components/table_vis_controls.tsx b/src/plugins/vis_type_table/public/components/table_vis_controls.tsx index 1f4f49442957f9..3eda73084e41de 100644 --- a/src/plugins/vis_type_table/public/components/table_vis_controls.tsx +++ b/src/plugins/vis_type_table/public/components/table_vis_controls.tsx @@ -11,81 +11,103 @@ import { EuiButtonEmpty, EuiContextMenuItem, EuiContextMenuPanel, EuiPopover } f import { FormattedMessage } from '@kbn/i18n/react'; import { i18n } from '@kbn/i18n'; -import { DatatableRow } from 'src/plugins/expressions'; +import { DatatableColumn, DatatableRow } from 'src/plugins/expressions'; import { CoreStart } from 'kibana/public'; import { useKibana } from '../../../kibana_react/public'; -import { FormattedColumn } from '../types'; -import { Table } from '../table_vis_response_handler'; -import { exportAsCsv } from '../utils'; +import { exporters } from '../../../data/public'; +import { + CSV_SEPARATOR_SETTING, + CSV_QUOTE_VALUES_SETTING, + downloadFileAs, +} from '../../../share/public'; +import { getFormatService } from '../services'; interface TableVisControlsProps { dataGridAriaLabel: string; filename?: string; - cols: FormattedColumn[]; + columns: DatatableColumn[]; rows: DatatableRow[]; - table: Table; } -export const TableVisControls = memo(({ dataGridAriaLabel, ...props }: TableVisControlsProps) => { - const [isPopoverOpen, setIsPopoverOpen] = useState(false); - const togglePopover = useCallback(() => setIsPopoverOpen((state) => !state), []); - const closePopover = useCallback(() => setIsPopoverOpen(false), []); +export const TableVisControls = memo( + ({ dataGridAriaLabel, filename, columns, rows }: TableVisControlsProps) => { + const [isPopoverOpen, setIsPopoverOpen] = useState(false); + const togglePopover = useCallback(() => setIsPopoverOpen((state) => !state), []); + const closePopover = useCallback(() => setIsPopoverOpen(false), []); - const { - services: { uiSettings }, - } = useKibana(); + const { + services: { uiSettings }, + } = useKibana(); - const onClickExport = useCallback( - (formatted: boolean) => - exportAsCsv(formatted, { - ...props, - uiSettings, - }), - [props, uiSettings] - ); + const onClickExport = useCallback( + (formatted: boolean) => { + const csvSeparator = uiSettings.get(CSV_SEPARATOR_SETTING); + const quoteValues = uiSettings.get(CSV_QUOTE_VALUES_SETTING); - const exportBtnAriaLabel = i18n.translate('visTypeTable.vis.controls.exportButtonAriaLabel', { - defaultMessage: 'Export {dataGridAriaLabel} as CSV', - values: { - dataGridAriaLabel, - }, - }); + const content = exporters.datatableToCSV( + { + type: 'datatable', + columns, + rows, + }, + { + csvSeparator, + quoteValues, + formatFactory: getFormatService().deserialize, + raw: !formatted, + } + ); + downloadFileAs(`${filename || 'unsaved'}.csv`, { content, type: exporters.CSV_MIME_TYPE }); + }, + [columns, rows, filename, uiSettings] + ); - const button = ( - - - - ); + const exportBtnAriaLabel = i18n.translate('visTypeTable.vis.controls.exportButtonAriaLabel', { + defaultMessage: 'Export {dataGridAriaLabel} as CSV', + values: { + dataGridAriaLabel, + }, + }); - const items = [ - onClickExport(false)}> - - , - onClickExport(true)}> - - , - ]; + const button = ( + + + + ); - return ( - - - - ); -}); + const items = [ + onClickExport(false)}> + + , + onClickExport(true)}> + + , + ]; + + return ( + + + + ); + } +); diff --git a/src/plugins/vis_type_table/public/components/table_vis_split.tsx b/src/plugins/vis_type_table/public/components/table_vis_split.tsx index be1a918e22c4bd..3d1cacd732fae7 100644 --- a/src/plugins/vis_type_table/public/components/table_vis_split.tsx +++ b/src/plugins/vis_type_table/public/components/table_vis_split.tsx @@ -9,8 +9,7 @@ import React, { memo } from 'react'; import { IInterpreterRenderHandlers } from 'src/plugins/expressions'; -import { TableGroup } from '../table_vis_response_handler'; -import { TableVisConfig, TableVisUseUiStateProps } from '../types'; +import { TableGroup, TableVisConfig, TableVisUseUiStateProps } from '../types'; import { TableVisBasic } from './table_vis_basic'; interface TableVisSplitProps { @@ -24,11 +23,11 @@ export const TableVisSplit = memo( ({ fireEvent, tables, visConfig, uiStateProps }: TableVisSplitProps) => { return ( <> - {tables.map(({ tables: dataTable, key, title }) => ( -
+ {tables.map(({ table, title }) => ( +
{ diff --git a/src/plugins/vis_type_table/public/table_vis_fn.test.ts b/src/plugins/vis_type_table/public/table_vis_fn.test.ts index ea8030688caeda..31b440ffb642f6 100644 --- a/src/plugins/vis_type_table/public/table_vis_fn.test.ts +++ b/src/plugins/vis_type_table/public/table_vis_fn.test.ts @@ -7,11 +7,11 @@ */ import { createTableVisFn } from './table_vis_fn'; -import { tableVisResponseHandler } from './table_vis_response_handler'; +import { tableVisResponseHandler } from './utils'; import { functionWrapper } from '../../expressions/common/expression_functions/specs/tests/utils'; -jest.mock('./table_vis_response_handler', () => ({ +jest.mock('./utils', () => ({ tableVisResponseHandler: jest.fn().mockReturnValue({ tables: [{ columns: [], rows: [] }], }), @@ -62,6 +62,6 @@ describe('interpreter/functions#table', () => { it('calls response handler with correct values', async () => { await fn(context, { visConfig: JSON.stringify(visConfig) }, undefined); expect(tableVisResponseHandler).toHaveBeenCalledTimes(1); - expect(tableVisResponseHandler).toHaveBeenCalledWith(context, visConfig.dimensions); + expect(tableVisResponseHandler).toHaveBeenCalledWith(context, visConfig); }); }); diff --git a/src/plugins/vis_type_table/public/table_vis_fn.ts b/src/plugins/vis_type_table/public/table_vis_fn.ts index 99fee424b8bea9..3dd8e81fc2ab27 100644 --- a/src/plugins/vis_type_table/public/table_vis_fn.ts +++ b/src/plugins/vis_type_table/public/table_vis_fn.ts @@ -7,10 +7,10 @@ */ import { i18n } from '@kbn/i18n'; -import { tableVisResponseHandler, TableContext } from './table_vis_response_handler'; import { ExpressionFunctionDefinition, Datatable, Render } from '../../expressions/public'; -import { TableVisConfig } from './types'; +import { TableVisData, TableVisConfig } from './types'; import { VIS_TYPE_TABLE } from '../common'; +import { tableVisResponseHandler } from './utils'; export type Input = Datatable; @@ -19,7 +19,7 @@ interface Arguments { } export interface TableVisRenderValue { - visData: TableContext; + visData: TableVisData; visType: typeof VIS_TYPE_TABLE; visConfig: TableVisConfig; } @@ -47,7 +47,7 @@ export const createTableVisFn = (): TableExpressionFunctionDefinition => ({ }, fn(input, args, handlers) { const visConfig = args.visConfig && JSON.parse(args.visConfig); - const convertedData = tableVisResponseHandler(input, visConfig.dimensions); + const convertedData = tableVisResponseHandler(input, visConfig); if (handlers?.inspectorAdapters?.tables) { handlers.inspectorAdapters.tables.logDatatable('default', input); diff --git a/src/plugins/vis_type_table/public/table_vis_response_handler.ts b/src/plugins/vis_type_table/public/table_vis_response_handler.ts deleted file mode 100644 index dbd01f94bd3c56..00000000000000 --- a/src/plugins/vis_type_table/public/table_vis_response_handler.ts +++ /dev/null @@ -1,90 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License - * and the Server Side Public License, v 1; you may not use this file except in - * compliance with, at your election, the Elastic License or the Server Side - * Public License, v 1. - */ - -import { Required } from '@kbn/utility-types'; - -import { getFormatService } from './services'; -import { Input } from './table_vis_fn'; -import { Dimensions } from './types'; - -export interface TableContext { - table?: Table; - tables: TableGroup[]; - direction?: 'row' | 'column'; -} - -export interface TableGroup { - table: Input; - tables: Table[]; - title: string; - name: string; - key: string | number; - column: number; - row: number; -} - -export interface Table { - columns: Input['columns']; - rows: Input['rows']; -} - -export function tableVisResponseHandler(input: Input, dimensions: Dimensions): TableContext { - let table: Table | undefined; - let tables: TableGroup[] = []; - let direction: TableContext['direction']; - - const split = dimensions.splitColumn || dimensions.splitRow; - - if (split) { - tables = []; - direction = dimensions.splitRow ? 'row' : 'column'; - const splitColumnIndex = split[0].accessor; - const splitColumnFormatter = getFormatService().deserialize(split[0].format); - const splitColumn = input.columns[splitColumnIndex]; - const splitMap: { [key: string]: number } = {}; - let splitIndex = 0; - - input.rows.forEach((row, rowIndex) => { - const splitValue: string | number = row[splitColumn.id]; - - if (!splitMap.hasOwnProperty(splitValue)) { - splitMap[splitValue] = splitIndex++; - const tableGroup: Required = { - title: `${splitColumnFormatter.convert(splitValue)}: ${splitColumn.name}`, - name: splitColumn.name, - key: splitValue, - column: splitColumnIndex, - row: rowIndex, - table: input, - tables: [], - }; - - tableGroup.tables.push({ - columns: input.columns, - rows: [], - }); - - tables.push(tableGroup); - } - - const tableIndex = splitMap[splitValue]; - tables[tableIndex].tables[0].rows.push(row); - }); - } else { - table = { - columns: input.columns, - rows: input.rows, - }; - } - - return { - direction, - table, - tables, - }; -} diff --git a/src/plugins/vis_type_table/public/types.ts b/src/plugins/vis_type_table/public/types.ts index 03cf8bb3395d6d..61ba7739b9cb19 100644 --- a/src/plugins/vis_type_table/public/types.ts +++ b/src/plugins/vis_type_table/public/types.ts @@ -7,6 +7,7 @@ */ import { IFieldFormat } from 'src/plugins/data/public'; +import { DatatableColumn, DatatableRow } from 'src/plugins/expressions'; import { SchemaConfig } from 'src/plugins/visualizations/public'; import { TableVisParams } from '../common'; @@ -43,7 +44,6 @@ export interface TableVisConfig extends TableVisParams { } export interface FormattedColumn { - id: string; title: string; formatter: IFieldFormat; formattedTotal?: string | number; @@ -51,3 +51,24 @@ export interface FormattedColumn { sumTotal?: number; total?: number; } + +export interface FormattedColumns { + [key: string]: FormattedColumn; +} + +export interface TableContext { + columns: DatatableColumn[]; + rows: DatatableRow[]; + formattedColumns: FormattedColumns; +} + +export interface TableGroup { + table: TableContext; + title: string; +} + +export interface TableVisData { + table?: TableContext; + tables: TableGroup[]; + direction?: 'row' | 'column'; +} diff --git a/src/plugins/vis_type_table/public/utils/add_percentage_column.ts b/src/plugins/vis_type_table/public/utils/add_percentage_column.ts index 0e3879255dd069..11528c76ee3006 100644 --- a/src/plugins/vis_type_table/public/utils/add_percentage_column.ts +++ b/src/plugins/vis_type_table/public/utils/add_percentage_column.ts @@ -6,48 +6,59 @@ * Public License, v 1. */ +import { findIndex } from 'lodash'; import { i18n } from '@kbn/i18n'; -import { DatatableRow } from 'src/plugins/expressions'; +import { DatatableColumn } from 'src/plugins/expressions'; import { getFormatService } from '../services'; -import { FormattedColumn } from '../types'; -import { Table } from '../table_vis_response_handler'; +import { FormattedColumns, TableContext } from '../types'; -function insertColumn(arr: FormattedColumn[], index: number, col: FormattedColumn) { +function insertColumn(arr: DatatableColumn[], index: number, col: DatatableColumn) { const newArray = [...arr]; newArray.splice(index + 1, 0, col); return newArray; } /** - * @param columns - the formatted columns that will be displayed - * @param title - the title of the column to add to - * @param rows - the row data for the columns - * @param insertAtIndex - the index to insert the percentage column at - * @returns cols and rows for the table to render now included percentage column(s) + * Adds a brand new column with percentages of selected column to existing data table */ -export function addPercentageColumn( - columns: FormattedColumn[], - title: string, - rows: Table['rows'], - insertAtIndex: number -) { - const { id, sumTotal } = columns[insertAtIndex]; - const newId = `${id}-percents`; +export function addPercentageColumn(table: TableContext, name: string) { + const { columns, rows, formattedColumns } = table; + const insertAtIndex = findIndex(columns, { name }); + // column to show percentage for was removed + if (insertAtIndex < 0) return table; + + const { id } = columns[insertAtIndex]; + const { sumTotal } = formattedColumns[id]; + const percentageColumnId = `${id}-percents`; const formatter = getFormatService().deserialize({ id: 'percent' }); - const i18nTitle = i18n.translate('visTypeTable.params.percentageTableColumnName', { + const percentageColumnName = i18n.translate('visTypeTable.params.percentageTableColumnName', { defaultMessage: '{title} percentages', - values: { title }, + values: { title: name }, }); const newCols = insertColumn(columns, insertAtIndex, { - title: i18nTitle, - id: newId, - formatter, - filterable: false, + name: percentageColumnName, + id: percentageColumnId, + meta: { + type: 'number', + params: { id: 'percent' }, + }, }); - const newRows = rows.map((row) => ({ - [newId]: (row[id] as number) / (sumTotal as number), + const newFormattedColumns: FormattedColumns = { + ...formattedColumns, + [percentageColumnId]: { + title: percentageColumnName, + formatter, + filterable: false, + }, + }; + const newRows = rows.map((row) => ({ + [percentageColumnId]: (row[id] as number) / (sumTotal as number), ...row, })); - return { cols: newCols, rows: newRows }; + return { + columns: newCols, + rows: newRows, + formattedColumns: newFormattedColumns, + }; } diff --git a/src/plugins/vis_type_table/public/utils/create_formatted_table.ts b/src/plugins/vis_type_table/public/utils/create_formatted_table.ts new file mode 100644 index 00000000000000..9dbb6c0c76e258 --- /dev/null +++ b/src/plugins/vis_type_table/public/utils/create_formatted_table.ts @@ -0,0 +1,97 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * and the Server Side Public License, v 1; you may not use this file except in + * compliance with, at your election, the Elastic License or the Server Side + * Public License, v 1. + */ + +import { chain } from 'lodash'; +import { Datatable } from 'src/plugins/expressions'; +import { getFormatService } from '../services'; +import { FormattedColumn, FormattedColumns, TableVisConfig, TableContext } from '../types'; +import { AggTypes } from '../../common'; + +export const createFormattedTable = ( + table: Datatable | TableContext, + visConfig: TableVisConfig +) => { + const { buckets, metrics } = visConfig.dimensions; + + const formattedColumns = table.columns.reduce((acc, col, i) => { + const isBucket = buckets.find(({ accessor }) => accessor === i); + const dimension = isBucket || metrics.find(({ accessor }) => accessor === i); + + if (!dimension) return acc; + + const formatter = getFormatService().deserialize(dimension.format); + const formattedColumn: FormattedColumn = { + title: col.name, + formatter, + filterable: !!isBucket, + }; + + const isDate = dimension.format.id === 'date' || dimension.format.params?.id === 'date'; + const allowsNumericalAggregations = formatter.allowsNumericalAggregations; + + if (allowsNumericalAggregations || isDate || visConfig.totalFunc === AggTypes.COUNT) { + const sumOfColumnValues = table.rows.reduce((prev, curr) => { + // some metrics return undefined for some of the values + // derivative is an example of this as it returns undefined in the first row + if (curr[col.id] === undefined) return prev; + return prev + (curr[col.id] as number); + }, 0); + + formattedColumn.sumTotal = sumOfColumnValues; + + switch (visConfig.totalFunc) { + case AggTypes.SUM: { + if (!isDate) { + formattedColumn.formattedTotal = formatter.convert(sumOfColumnValues); + formattedColumn.total = sumOfColumnValues; + } + break; + } + case AggTypes.AVG: { + if (!isDate) { + const total = sumOfColumnValues / table.rows.length; + formattedColumn.formattedTotal = formatter.convert(total); + formattedColumn.total = total; + } + break; + } + case AggTypes.MIN: { + const total = chain(table.rows).map(col.id).min().value() as number; + formattedColumn.formattedTotal = formatter.convert(total); + formattedColumn.total = total; + break; + } + case AggTypes.MAX: { + const total = chain(table.rows).map(col.id).max().value() as number; + formattedColumn.formattedTotal = formatter.convert(total); + formattedColumn.total = total; + break; + } + case AggTypes.COUNT: { + const total = table.rows.length; + formattedColumn.formattedTotal = total; + formattedColumn.total = total; + break; + } + default: + break; + } + } + + acc[col.id] = formattedColumn; + + return acc; + }, {}); + + return { + // filter out columns which are not dimensions + columns: table.columns.filter((col) => formattedColumns[col.id]), + rows: table.rows, + formattedColumns, + }; +}; diff --git a/src/plugins/vis_type_table/public/utils/export_as_csv.ts b/src/plugins/vis_type_table/public/utils/export_as_csv.ts deleted file mode 100644 index 4371a20cfa0dae..00000000000000 --- a/src/plugins/vis_type_table/public/utils/export_as_csv.ts +++ /dev/null @@ -1,64 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License - * and the Server Side Public License, v 1; you may not use this file except in - * compliance with, at your election, the Elastic License or the Server Side - * Public License, v 1. - */ - -import { isObject } from 'lodash'; -// @ts-ignore -import { saveAs } from '@elastic/filesaver'; - -import { CoreStart } from 'kibana/public'; -import { DatatableRow } from 'src/plugins/expressions'; -import { CSV_SEPARATOR_SETTING, CSV_QUOTE_VALUES_SETTING } from '../../../share/public'; -import { FormattedColumn } from '../types'; -import { Table } from '../table_vis_response_handler'; - -const nonAlphaNumRE = /[^a-zA-Z0-9]/; -const allDoubleQuoteRE = /"/g; - -interface ToCsvData { - filename?: string; - cols: FormattedColumn[]; - rows: DatatableRow[]; - table: Table; - uiSettings: CoreStart['uiSettings']; -} - -const toCsv = (formatted: boolean, { cols, rows, table, uiSettings }: ToCsvData) => { - const separator = uiSettings.get(CSV_SEPARATOR_SETTING); - const quoteValues = uiSettings.get(CSV_QUOTE_VALUES_SETTING); - - function escape(val: unknown) { - if (!formatted && isObject(val)) val = val.valueOf(); - val = String(val); - if (quoteValues && nonAlphaNumRE.test(val as string)) { - val = '"' + (val as string).replace(allDoubleQuoteRE, '""') + '"'; - } - return val as string; - } - - const csvRows: string[][] = []; - - for (const row of rows) { - const rowArray: string[] = []; - for (const col of cols) { - const value = row[col.id]; - const formattedValue = formatted ? escape(col.formatter.convert(value)) : escape(value); - rowArray.push(formattedValue); - } - csvRows.push(rowArray); - } - - // add headers to the rows - csvRows.unshift(cols.map(({ title }) => escape(title))); - - return csvRows.map((row) => row.join(separator) + '\r\n').join(''); -}; - -export const exportAsCsv = (formatted: boolean, data: ToCsvData) => { - const csv = new Blob([toCsv(formatted, data)], { type: 'text/plain;charset=utf-8' }); - saveAs(csv, `${data.filename || 'unsaved'}.csv`); -}; diff --git a/src/plugins/vis_type_table/public/utils/index.ts b/src/plugins/vis_type_table/public/utils/index.ts index 6a6dda0d12fa3c..8731b52a7ba30f 100644 --- a/src/plugins/vis_type_table/public/utils/index.ts +++ b/src/plugins/vis_type_table/public/utils/index.ts @@ -7,4 +7,4 @@ */ export * from './use'; -export * from './export_as_csv'; +export * from './table_vis_response_handler'; diff --git a/src/plugins/vis_type_table/public/utils/table_vis_response_handler.ts b/src/plugins/vis_type_table/public/utils/table_vis_response_handler.ts new file mode 100644 index 00000000000000..0a2b8d81808543 --- /dev/null +++ b/src/plugins/vis_type_table/public/utils/table_vis_response_handler.ts @@ -0,0 +1,75 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * and the Server Side Public License, v 1; you may not use this file except in + * compliance with, at your election, the Elastic License or the Server Side + * Public License, v 1. + */ + +import { Datatable } from 'src/plugins/expressions'; +import { getFormatService } from '../services'; +import { TableVisData, TableGroup, TableVisConfig, TableContext } from '../types'; +import { addPercentageColumn } from './add_percentage_column'; +import { createFormattedTable } from './create_formatted_table'; + +/** + * Converts datatable input from response into appropriate format for consuming renderer + */ +export function tableVisResponseHandler(input: Datatable, visConfig: TableVisConfig): TableVisData { + const tables: TableGroup[] = []; + let table: TableContext | undefined; + let direction: TableVisData['direction']; + + const split = visConfig.dimensions.splitColumn || visConfig.dimensions.splitRow; + + if (split) { + direction = visConfig.dimensions.splitRow ? 'row' : 'column'; + const splitColumnIndex = split[0].accessor; + const splitColumnFormatter = getFormatService().deserialize(split[0].format); + const splitColumn = input.columns[splitColumnIndex]; + const columns = input.columns.filter((c, idx) => idx !== splitColumnIndex); + const splitMap: { [key: string]: number } = {}; + let splitIndex = 0; + + input.rows.forEach((row) => { + const splitValue: string | number = row[splitColumn.id]; + + if (!splitMap.hasOwnProperty(splitValue)) { + splitMap[splitValue] = splitIndex++; + const tableGroup: TableGroup = { + title: `${splitColumnFormatter.convert(splitValue)}: ${splitColumn.name}`, + table: { + columns, + rows: [], + formattedColumns: {}, + }, + }; + + tables.push(tableGroup); + } + + const tableIndex = splitMap[splitValue]; + tables[tableIndex].table.rows.push(row); + }); + + tables.forEach((tg) => { + tg.table = createFormattedTable({ ...tg.table, columns: input.columns }, visConfig); + + if (visConfig.percentageCol) { + tg.table = addPercentageColumn(tg.table, visConfig.percentageCol); + } + }); + } else { + table = createFormattedTable(input, visConfig); + + if (visConfig.percentageCol) { + table = addPercentageColumn(table, visConfig.percentageCol); + } + } + + return { + direction, + table, + tables, + }; +} diff --git a/src/plugins/vis_type_table/public/utils/use/index.ts b/src/plugins/vis_type_table/public/utils/use/index.ts index 08daf7f28c0e86..9fcc7915610464 100644 --- a/src/plugins/vis_type_table/public/utils/use/index.ts +++ b/src/plugins/vis_type_table/public/utils/use/index.ts @@ -6,6 +6,5 @@ * Public License, v 1. */ -export * from './use_formatted_columns'; export * from './use_pagination'; export * from './use_ui_state'; diff --git a/src/plugins/vis_type_table/public/utils/use/use_formatted_columns.ts b/src/plugins/vis_type_table/public/utils/use/use_formatted_columns.ts deleted file mode 100644 index 3a733e7a9a4dcf..00000000000000 --- a/src/plugins/vis_type_table/public/utils/use/use_formatted_columns.ts +++ /dev/null @@ -1,115 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License - * and the Server Side Public License, v 1; you may not use this file except in - * compliance with, at your election, the Elastic License or the Server Side - * Public License, v 1. - */ - -import { useMemo } from 'react'; -import { chain, findIndex } from 'lodash'; - -import { AggTypes } from '../../../common'; -import { Table } from '../../table_vis_response_handler'; -import { FormattedColumn, TableVisConfig } from '../../types'; -import { getFormatService } from '../../services'; -import { addPercentageColumn } from '../add_percentage_column'; - -export const useFormattedColumnsAndRows = (table: Table, visConfig: TableVisConfig) => { - const { formattedColumns: columns, formattedRows: rows } = useMemo(() => { - const { buckets, metrics } = visConfig.dimensions; - let formattedRows = table.rows; - - let formattedColumns = table.columns - .map((col, i) => { - const isBucket = buckets.find(({ accessor }) => accessor === i); - const dimension = isBucket || metrics.find(({ accessor }) => accessor === i); - - if (!dimension) return undefined; - - const formatter = getFormatService().deserialize(dimension.format); - const formattedColumn: FormattedColumn = { - id: col.id, - title: col.name, - formatter, - filterable: !!isBucket, - }; - - const isDate = dimension.format.id === 'date' || dimension.format.params?.id === 'date'; - const allowsNumericalAggregations = formatter.allowsNumericalAggregations; - - if (allowsNumericalAggregations || isDate || visConfig.totalFunc === AggTypes.COUNT) { - const sumOfColumnValues = table.rows.reduce((prev, curr) => { - // some metrics return undefined for some of the values - // derivative is an example of this as it returns undefined in the first row - if (curr[col.id] === undefined) return prev; - return prev + (curr[col.id] as number); - }, 0); - - formattedColumn.sumTotal = sumOfColumnValues; - - switch (visConfig.totalFunc) { - case AggTypes.SUM: { - if (!isDate) { - formattedColumn.formattedTotal = formatter.convert(sumOfColumnValues); - formattedColumn.total = sumOfColumnValues; - } - break; - } - case AggTypes.AVG: { - if (!isDate) { - const total = sumOfColumnValues / table.rows.length; - formattedColumn.formattedTotal = formatter.convert(total); - formattedColumn.total = total; - } - break; - } - case AggTypes.MIN: { - const total = chain(table.rows).map(col.id).min().value() as number; - formattedColumn.formattedTotal = formatter.convert(total); - formattedColumn.total = total; - break; - } - case AggTypes.MAX: { - const total = chain(table.rows).map(col.id).max().value() as number; - formattedColumn.formattedTotal = formatter.convert(total); - formattedColumn.total = total; - break; - } - case AggTypes.COUNT: { - const total = table.rows.length; - formattedColumn.formattedTotal = total; - formattedColumn.total = total; - break; - } - default: - break; - } - } - - return formattedColumn; - }) - .filter((column): column is FormattedColumn => !!column); - - if (visConfig.percentageCol) { - const insertAtIndex = findIndex(formattedColumns, { title: visConfig.percentageCol }); - - // column to show percentage for was removed - if (insertAtIndex < 0) return { formattedColumns, formattedRows }; - - const { cols, rows: rowsWithPercentage } = addPercentageColumn( - formattedColumns, - visConfig.percentageCol, - table.rows, - insertAtIndex - ); - - formattedRows = rowsWithPercentage; - formattedColumns = cols; - } - - return { formattedColumns, formattedRows }; - }, [table, visConfig.dimensions, visConfig.percentageCol, visConfig.totalFunc]); - - return { columns, rows }; -}; diff --git a/test/accessibility/apps/kibana_overview.ts b/test/accessibility/apps/kibana_overview.ts index eb0b54ad07aa75..c26a042b10e729 100644 --- a/test/accessibility/apps/kibana_overview.ts +++ b/test/accessibility/apps/kibana_overview.ts @@ -16,7 +16,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { const esArchiver = getService('esArchiver'); before(async () => { - await esArchiver.emptyKibanaIndex(); + await esArchiver.load('empty_kibana'); await PageObjects.common.navigateToApp('kibanaOverview'); }); @@ -25,6 +25,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { useActualUrl: true, }); await PageObjects.home.removeSampleDataSet('flights'); + await esArchiver.unload('empty_kibana'); }); it('Getting started view', async () => { diff --git a/test/api_integration/apis/home/sample_data.ts b/test/api_integration/apis/home/sample_data.ts index ebda93b12dc200..042aff13752676 100644 --- a/test/api_integration/apis/home/sample_data.ts +++ b/test/api_integration/apis/home/sample_data.ts @@ -11,15 +11,11 @@ import { FtrProviderContext } from '../../ftr_provider_context'; export default function ({ getService }: FtrProviderContext) { const supertest = getService('supertest'); - const esArchiver = getService('esArchiver'); const es = getService('es'); const MILLISECOND_IN_WEEK = 1000 * 60 * 60 * 24 * 7; describe('sample data apis', () => { - before(async () => { - await esArchiver.emptyKibanaIndex(); - }); describe('list', () => { it('should return list of sample data sets with installed status', async () => { const resp = await supertest.get(`/api/sample_data`).set('kbn-xsrf', 'kibana').expect(200); diff --git a/test/api_integration/apis/saved_objects/bulk_create.ts b/test/api_integration/apis/saved_objects/bulk_create.ts index d7cdee16214a8d..a548172365b07b 100644 --- a/test/api_integration/apis/saved_objects/bulk_create.ts +++ b/test/api_integration/apis/saved_objects/bulk_create.ts @@ -97,11 +97,10 @@ export default function ({ getService }: FtrProviderContext) { before( async () => // just in case the kibana server has recreated it - await es.indices.delete({ index: '.kibana*' }, { ignore: [404] }) + await es.indices.delete({ index: '.kibana' }, { ignore: [404] }) ); - it('should return 200 with errors', async () => { - await new Promise((resolve) => setTimeout(resolve, 2000)); + it('should return 200 with individual responses', async () => await supertest .post('/api/saved_objects/_bulk_create') .send(BULK_REQUESTS) @@ -110,27 +109,38 @@ export default function ({ getService }: FtrProviderContext) { expect(resp.body).to.eql({ saved_objects: [ { - id: BULK_REQUESTS[0].id, - type: BULK_REQUESTS[0].type, - error: { - error: 'Internal Server Error', - message: 'An internal server error occurred', - statusCode: 500, + type: 'visualization', + id: 'dd7caf20-9efd-11e7-acb3-3dab96693fab', + updated_at: resp.body.saved_objects[0].updated_at, + version: resp.body.saved_objects[0].version, + attributes: { + title: 'An existing visualization', + }, + references: [], + namespaces: ['default'], + migrationVersion: { + visualization: resp.body.saved_objects[0].migrationVersion.visualization, }, + coreMigrationVersion: KIBANA_VERSION, // updated from 1.2.3 to the latest kibana version }, { - id: BULK_REQUESTS[1].id, - type: BULK_REQUESTS[1].type, - error: { - error: 'Internal Server Error', - message: 'An internal server error occurred', - statusCode: 500, + type: 'dashboard', + id: 'a01b2f57-fcfd-4864-b735-09e28f0d815e', + updated_at: resp.body.saved_objects[1].updated_at, + version: resp.body.saved_objects[1].version, + attributes: { + title: 'A great new dashboard', }, + references: [], + namespaces: ['default'], + migrationVersion: { + dashboard: resp.body.saved_objects[1].migrationVersion.dashboard, + }, + coreMigrationVersion: KIBANA_VERSION, }, ], }); - }); - }); + })); }); }); } diff --git a/test/api_integration/apis/saved_objects/bulk_get.ts b/test/api_integration/apis/saved_objects/bulk_get.ts index b9536843d30c95..46631225f8e8a2 100644 --- a/test/api_integration/apis/saved_objects/bulk_get.ts +++ b/test/api_integration/apis/saved_objects/bulk_get.ts @@ -108,7 +108,7 @@ export default function ({ getService }: FtrProviderContext) { before( async () => // just in case the kibana server has recreated it - await es.indices.delete({ index: '.kibana*' }, { ignore: [404] }) + await es.indices.delete({ index: '.kibana' }, { ignore: [404] }) ); it('should return 200 with individual responses', async () => diff --git a/test/api_integration/apis/saved_objects/bulk_update.ts b/test/api_integration/apis/saved_objects/bulk_update.ts index 2cf3ade406a93d..5a2496b6dde819 100644 --- a/test/api_integration/apis/saved_objects/bulk_update.ts +++ b/test/api_integration/apis/saved_objects/bulk_update.ts @@ -235,10 +235,10 @@ export default function ({ getService }: FtrProviderContext) { before( async () => // just in case the kibana server has recreated it - await es.indices.delete({ index: '.kibana*' }, { ignore: [404] }) + await es.indices.delete({ index: '.kibana' }, { ignore: [404] }) ); - it('should return 200 with errors', async () => { + it('should return generic 404', async () => { const response = await supertest .put(`/api/saved_objects/_bulk_update`) .send([ @@ -267,9 +267,9 @@ export default function ({ getService }: FtrProviderContext) { id: 'dd7caf20-9efd-11e7-acb3-3dab96693fab', type: 'visualization', error: { - statusCode: 500, - error: 'Internal Server Error', - message: 'An internal server error occurred', + statusCode: 404, + error: 'Not Found', + message: 'Saved object [visualization/dd7caf20-9efd-11e7-acb3-3dab96693fab] not found', }, }); @@ -277,9 +277,9 @@ export default function ({ getService }: FtrProviderContext) { id: 'be3733a0-9efe-11e7-acb3-3dab96693fab', type: 'dashboard', error: { - statusCode: 500, - error: 'Internal Server Error', - message: 'An internal server error occurred', + statusCode: 404, + error: 'Not Found', + message: 'Saved object [dashboard/be3733a0-9efe-11e7-acb3-3dab96693fab] not found', }, }); }); diff --git a/test/api_integration/apis/saved_objects/create.ts b/test/api_integration/apis/saved_objects/create.ts index 833cb127d0023b..551e082630e8f3 100644 --- a/test/api_integration/apis/saved_objects/create.ts +++ b/test/api_integration/apis/saved_objects/create.ts @@ -82,10 +82,10 @@ export default function ({ getService }: FtrProviderContext) { before( async () => // just in case the kibana server has recreated it - await es.indices.delete({ index: '.kibana*' }, { ignore: [404] }) + await es.indices.delete({ index: '.kibana' }, { ignore: [404] }) ); - it('should return 500 and not auto-create saved objects index', async () => { + it('should return 200 and create kibana index', async () => { await supertest .post(`/api/saved_objects/visualization`) .send({ @@ -93,16 +93,50 @@ export default function ({ getService }: FtrProviderContext) { title: 'My favorite vis', }, }) - .expect(500) + .expect(200) .then((resp) => { + // loose uuid validation + expect(resp.body) + .to.have.property('id') + .match(/^[0-9a-f-]{36}$/); + + // loose ISO8601 UTC time with milliseconds validation + expect(resp.body) + .to.have.property('updated_at') + .match(/^[\d-]{10}T[\d:\.]{12}Z$/); + expect(resp.body).to.eql({ - error: 'Internal Server Error', - message: 'An internal server error occurred.', - statusCode: 500, + id: resp.body.id, + type: 'visualization', + migrationVersion: resp.body.migrationVersion, + coreMigrationVersion: KIBANA_VERSION, + updated_at: resp.body.updated_at, + version: resp.body.version, + attributes: { + title: 'My favorite vis', + }, + references: [], + namespaces: ['default'], }); + expect(resp.body.migrationVersion).to.be.ok(); }); - expect((await es.indices.exists({ index: '.kibana' })).body).to.be(false); + expect((await es.indices.exists({ index: '.kibana' })).body).to.be(true); + }); + + it('result should have the latest coreMigrationVersion', async () => { + await supertest + .post(`/api/saved_objects/visualization`) + .send({ + attributes: { + title: 'My favorite vis', + }, + coreMigrationVersion: '1.2.3', + }) + .expect(200) + .then((resp) => { + expect(resp.body.coreMigrationVersion).to.eql(KIBANA_VERSION); + }); }); }); }); diff --git a/test/api_integration/apis/saved_objects/delete.ts b/test/api_integration/apis/saved_objects/delete.ts index d2dd4454bdf1ed..9ba51b4b91468e 100644 --- a/test/api_integration/apis/saved_objects/delete.ts +++ b/test/api_integration/apis/saved_objects/delete.ts @@ -44,7 +44,7 @@ export default function ({ getService }: FtrProviderContext) { before( async () => // just in case the kibana server has recreated it - await es.indices.delete({ index: '.kibana*' }, { ignore: [404] }) + await es.indices.delete({ index: '.kibana' }, { ignore: [404] }) ); it('returns generic 404 when kibana index is missing', async () => diff --git a/test/api_integration/apis/saved_objects/export.ts b/test/api_integration/apis/saved_objects/export.ts index c0d54300709511..a45191f24d8725 100644 --- a/test/api_integration/apis/saved_objects/export.ts +++ b/test/api_integration/apis/saved_objects/export.ts @@ -534,7 +534,7 @@ export default function ({ getService }: FtrProviderContext) { before( async () => // just in case the kibana server has recreated it - await es.indices.delete({ index: '.kibana*' }, { ignore: [404] }) + await es.indices.delete({ index: '.kibana' }, { ignore: [404] }) ); it('should return empty response', async () => { diff --git a/test/api_integration/apis/saved_objects/find.ts b/test/api_integration/apis/saved_objects/find.ts index 5f549dc6c5780f..7aa4de86baa692 100644 --- a/test/api_integration/apis/saved_objects/find.ts +++ b/test/api_integration/apis/saved_objects/find.ts @@ -40,7 +40,7 @@ export default function ({ getService }: FtrProviderContext) { { type: 'visualization', id: 'dd7caf20-9efd-11e7-acb3-3dab96693fab', - version: 'WzE4LDJd', + version: 'WzIsMV0=', attributes: { title: 'Count of requests', }, @@ -137,7 +137,7 @@ export default function ({ getService }: FtrProviderContext) { { type: 'visualization', id: 'dd7caf20-9efd-11e7-acb3-3dab96693fab', - version: 'WzE4LDJd', + version: 'WzIsMV0=', attributes: { title: 'Count of requests', }, @@ -174,7 +174,7 @@ export default function ({ getService }: FtrProviderContext) { { type: 'visualization', id: 'dd7caf20-9efd-11e7-acb3-3dab96693fab', - version: 'WzE4LDJd', + version: 'WzIsMV0=', attributes: { title: 'Count of requests', }, @@ -209,7 +209,7 @@ export default function ({ getService }: FtrProviderContext) { score: 0, type: 'visualization', updated_at: '2017-09-21T18:51:23.794Z', - version: 'WzIyLDJd', + version: 'WzYsMV0=', }, ], }); @@ -256,7 +256,7 @@ export default function ({ getService }: FtrProviderContext) { migrationVersion: resp.body.saved_objects[0].migrationVersion, coreMigrationVersion: KIBANA_VERSION, updated_at: '2017-09-21T18:51:23.794Z', - version: 'WzE4LDJd', + version: 'WzIsMV0=', }, ], }); @@ -426,11 +426,11 @@ export default function ({ getService }: FtrProviderContext) { })); }); - describe('without kibana index', () => { + describe.skip('without kibana index', () => { before( async () => // just in case the kibana server has recreated it - await es.indices.delete({ index: '.kibana*' }, { ignore: [404] }) + await es.indices.delete({ index: '.kibana' }, { ignore: [404] }) ); it('should return 200 with empty response', async () => diff --git a/test/api_integration/apis/saved_objects/get.ts b/test/api_integration/apis/saved_objects/get.ts index 9bb6e32004c812..ff47b9df218dc4 100644 --- a/test/api_integration/apis/saved_objects/get.ts +++ b/test/api_integration/apis/saved_objects/get.ts @@ -78,7 +78,7 @@ export default function ({ getService }: FtrProviderContext) { before( async () => // just in case the kibana server has recreated it - await es.indices.delete({ index: '.kibana*' }, { ignore: [404] }) + await es.indices.delete({ index: '.kibana' }, { ignore: [404] }) ); it('should return basic 404 without mentioning index', async () => diff --git a/test/api_integration/apis/saved_objects/resolve_import_errors.ts b/test/api_integration/apis/saved_objects/resolve_import_errors.ts index 042741476bb8e0..3686c46b229b12 100644 --- a/test/api_integration/apis/saved_objects/resolve_import_errors.ts +++ b/test/api_integration/apis/saved_objects/resolve_import_errors.ts @@ -13,7 +13,6 @@ import { FtrProviderContext } from '../../ftr_provider_context'; export default function ({ getService }: FtrProviderContext) { const supertest = getService('supertest'); const esArchiver = getService('esArchiver'); - const es = getService('legacyEs'); describe('resolve_import_errors', () => { // mock success results including metadata @@ -35,14 +34,7 @@ export default function ({ getService }: FtrProviderContext) { describe('without kibana index', () => { // Cleanup data that got created in import - before( - async () => - // just in case the kibana server has recreated it - await es.indices.delete({ - index: '.kibana*', - ignore: [404], - }) - ); + after(() => esArchiver.unload('saved_objects/basic')); it('should return 200 and import nothing when empty parameters are passed in', async () => { await supertest @@ -59,7 +51,7 @@ export default function ({ getService }: FtrProviderContext) { }); }); - it('should return 200 with internal server errors', async () => { + it('should return 200 and import everything when overwrite parameters contains all objects', async () => { await supertest .post('/api/saved_objects/_resolve_import_errors') .field( @@ -86,42 +78,12 @@ export default function ({ getService }: FtrProviderContext) { .expect(200) .then((resp) => { expect(resp.body).to.eql({ - successCount: 0, - success: false, - errors: [ - { - ...indexPattern, - ...{ title: indexPattern.meta.title }, - overwrite: true, - error: { - statusCode: 500, - error: 'Internal Server Error', - message: 'An internal server error occurred', - type: 'unknown', - }, - }, - { - ...visualization, - ...{ title: visualization.meta.title }, - overwrite: true, - error: { - statusCode: 500, - error: 'Internal Server Error', - message: 'An internal server error occurred', - type: 'unknown', - }, - }, - { - ...dashboard, - ...{ title: dashboard.meta.title }, - overwrite: true, - error: { - statusCode: 500, - error: 'Internal Server Error', - message: 'An internal server error occurred', - type: 'unknown', - }, - }, + success: true, + successCount: 3, + successResults: [ + { ...indexPattern, overwrite: true }, + { ...visualization, overwrite: true }, + { ...dashboard, overwrite: true }, ], warnings: [], }); diff --git a/test/api_integration/apis/saved_objects/update.ts b/test/api_integration/apis/saved_objects/update.ts index da7285a430fddb..d5346e82ce98c8 100644 --- a/test/api_integration/apis/saved_objects/update.ts +++ b/test/api_integration/apis/saved_objects/update.ts @@ -121,10 +121,10 @@ export default function ({ getService }: FtrProviderContext) { before( async () => // just in case the kibana server has recreated it - await es.indices.delete({ index: '.kibana*' }, { ignore: [404] }) + await es.indices.delete({ index: '.kibana' }, { ignore: [404] }) ); - it('should return 500', async () => + it('should return generic 404', async () => await supertest .put(`/api/saved_objects/visualization/dd7caf20-9efd-11e7-acb3-3dab96693fab`) .send({ @@ -132,12 +132,13 @@ export default function ({ getService }: FtrProviderContext) { title: 'My second favorite vis', }, }) - .expect(500) + .expect(404) .then((resp) => { expect(resp.body).eql({ - statusCode: 500, - error: 'Internal Server Error', - message: 'An internal server error occurred.', + statusCode: 404, + error: 'Not Found', + message: + 'Saved object [visualization/dd7caf20-9efd-11e7-acb3-3dab96693fab] not found', }); })); }); diff --git a/test/api_integration/apis/saved_objects_management/find.ts b/test/api_integration/apis/saved_objects_management/find.ts index 8453b542903a43..acc01c73de674d 100644 --- a/test/api_integration/apis/saved_objects_management/find.ts +++ b/test/api_integration/apis/saved_objects_management/find.ts @@ -42,7 +42,7 @@ export default function ({ getService }: FtrProviderContext) { { type: 'visualization', id: 'dd7caf20-9efd-11e7-acb3-3dab96693fab', - version: 'WzE4LDJd', + version: 'WzIsMV0=', attributes: { title: 'Count of requests', }, @@ -184,7 +184,7 @@ export default function ({ getService }: FtrProviderContext) { before( async () => // just in case the kibana server has recreated it - await es.indices.delete({ index: '.kibana*' }, { ignore: [404] }) + await es.indices.delete({ index: '.kibana' }, { ignore: [404] }) ); it('should return 200 with empty response', async () => diff --git a/test/api_integration/apis/saved_objects_management/get.ts b/test/api_integration/apis/saved_objects_management/get.ts index 70e1faa9fd22b9..bc05d7e392bb98 100644 --- a/test/api_integration/apis/saved_objects_management/get.ts +++ b/test/api_integration/apis/saved_objects_management/get.ts @@ -45,7 +45,7 @@ export default function ({ getService }: FtrProviderContext) { before( async () => // just in case the kibana server has recreated it - await es.indices.delete({ index: '.kibana*' }, { ignore: [404] }) + await es.indices.delete({ index: '.kibana' }, { ignore: [404] }) ); it('should return 404 for object that no longer exists', async () => diff --git a/test/api_integration/apis/search/search.ts b/test/api_integration/apis/search/search.ts index e43c4493093061..155705f81fa8a9 100644 --- a/test/api_integration/apis/search/search.ts +++ b/test/api_integration/apis/search/search.ts @@ -17,7 +17,6 @@ export default function ({ getService }: FtrProviderContext) { describe('search', () => { before(async () => { - await esArchiver.emptyKibanaIndex(); await esArchiver.loadIfNeeded('../../../functional/fixtures/es_archiver/logstash_functional'); }); diff --git a/test/api_integration/apis/telemetry/opt_in.ts b/test/api_integration/apis/telemetry/opt_in.ts index ba5f46c38211f9..f03b33e61965e6 100644 --- a/test/api_integration/apis/telemetry/opt_in.ts +++ b/test/api_integration/apis/telemetry/opt_in.ts @@ -14,13 +14,10 @@ import { FtrProviderContext } from '../../ftr_provider_context'; export default function optInTest({ getService }: FtrProviderContext) { const supertest = getService('supertest'); const kibanaServer = getService('kibanaServer'); - const esArchiver = getService('esArchiver'); - describe('/api/telemetry/v2/optIn API', () => { let defaultAttributes: TelemetrySavedObjectAttributes; let kibanaVersion: any; before(async () => { - await esArchiver.emptyKibanaIndex(); const kibanaVersionAccessor = kibanaServer.version; kibanaVersion = await kibanaVersionAccessor.get(); defaultAttributes = diff --git a/test/api_integration/apis/telemetry/telemetry_local.ts b/test/api_integration/apis/telemetry/telemetry_local.ts index 650846015a4a2f..25d29a807bdadf 100644 --- a/test/api_integration/apis/telemetry/telemetry_local.ts +++ b/test/api_integration/apis/telemetry/telemetry_local.ts @@ -177,7 +177,6 @@ export default function ({ getService }: FtrProviderContext) { describe('basic behaviour', () => { let savedObjectIds: string[] = []; before('create application usage entries', async () => { - await esArchiver.emptyKibanaIndex(); savedObjectIds = await Promise.all([ createSavedObject(), createSavedObject('appView1'), diff --git a/test/api_integration/apis/ui_counters/ui_counters.ts b/test/api_integration/apis/ui_counters/ui_counters.ts index 8d60c79c9698d9..1cf16fe433bf9d 100644 --- a/test/api_integration/apis/ui_counters/ui_counters.ts +++ b/test/api_integration/apis/ui_counters/ui_counters.ts @@ -13,7 +13,6 @@ import { FtrProviderContext } from '../../ftr_provider_context'; export default function ({ getService }: FtrProviderContext) { const supertest = getService('supertest'); - const esArchiver = getService('esArchiver'); const es = getService('es'); const createUiCounterEvent = (eventName: string, type: UiCounterMetricType, count = 1) => ({ @@ -24,10 +23,6 @@ export default function ({ getService }: FtrProviderContext) { }); describe('UI Counters API', () => { - before(async () => { - await esArchiver.emptyKibanaIndex(); - }); - const dayDate = moment().format('DDMMYYYY'); it('stores ui counter events in savedObjects', async () => { diff --git a/test/api_integration/apis/ui_metric/ui_metric.ts b/test/api_integration/apis/ui_metric/ui_metric.ts index e3b3b2ec4c542a..d330cb037d1a1c 100644 --- a/test/api_integration/apis/ui_metric/ui_metric.ts +++ b/test/api_integration/apis/ui_metric/ui_metric.ts @@ -13,7 +13,6 @@ import { FtrProviderContext } from '../../ftr_provider_context'; export default function ({ getService }: FtrProviderContext) { const supertest = getService('supertest'); - const esArchiver = getService('esArchiver'); const es = getService('es'); const createStatsMetric = ( @@ -35,10 +34,6 @@ export default function ({ getService }: FtrProviderContext) { }); describe('ui_metric savedObject data', () => { - before(async () => { - await esArchiver.emptyKibanaIndex(); - }); - it('increments the count field in the document defined by the {app}/{action_type} path', async () => { const reportManager = new ReportManager(); const uiStatsMetric = createStatsMetric('myEvent'); diff --git a/test/common/config.js b/test/common/config.js index 8a42e6c87b214d..b6d12444b7017d 100644 --- a/test/common/config.js +++ b/test/common/config.js @@ -61,6 +61,8 @@ export default function () { ...(!!process.env.CODE_COVERAGE ? [`--plugin-path=${path.join(__dirname, 'fixtures', 'plugins', 'coverage')}`] : []), + // Disable v2 migrations in tests for now + '--migrations.enableV2=false', ], }, services, diff --git a/test/common/services/kibana_server/extend_es_archiver.js b/test/common/services/kibana_server/extend_es_archiver.js index 1d76bc44737678..5390b43a87187d 100644 --- a/test/common/services/kibana_server/extend_es_archiver.js +++ b/test/common/services/kibana_server/extend_es_archiver.js @@ -6,7 +6,7 @@ * Public License, v 1. */ -const ES_ARCHIVER_LOAD_METHODS = ['load', 'loadIfNeeded', 'unload', 'emptyKibanaIndex']; +const ES_ARCHIVER_LOAD_METHODS = ['load', 'loadIfNeeded', 'unload']; const KIBANA_INDEX = '.kibana'; export function extendEsArchiver({ esArchiver, kibanaServer, retry, defaults }) { @@ -25,7 +25,7 @@ export function extendEsArchiver({ esArchiver, kibanaServer, retry, defaults }) const statsKeys = Object.keys(stats); const kibanaKeys = statsKeys.filter( // this also matches stats keys like '.kibana_1' and '.kibana_2,.kibana_1' - (key) => key.includes(KIBANA_INDEX) && stats[key].created + (key) => key.includes(KIBANA_INDEX) && (stats[key].created || stats[key].deleted) ); // if the kibana index was created by the esArchiver then update the uiSettings diff --git a/test/functional/apps/management/_import_objects.ts b/test/functional/apps/management/_import_objects.ts index e18f2a74854441..754406938e47b6 100644 --- a/test/functional/apps/management/_import_objects.ts +++ b/test/functional/apps/management/_import_objects.ts @@ -27,9 +27,9 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { describe.skip('import objects', function describeIndexTests() { describe('.ndjson file', () => { beforeEach(async function () { - await esArchiver.load('management'); await kibanaServer.uiSettings.replace({}); await PageObjects.settings.navigateTo(); + await esArchiver.load('management'); await PageObjects.settings.clickKibanaSavedObjects(); }); @@ -213,9 +213,10 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { describe('.json file', () => { beforeEach(async function () { - await esArchiver.load('saved_objects_imports'); + // delete .kibana index and then wait for Kibana to re-create it await kibanaServer.uiSettings.replace({}); await PageObjects.settings.navigateTo(); + await esArchiver.load('saved_objects_imports'); await PageObjects.settings.clickKibanaSavedObjects(); }); diff --git a/test/functional/apps/management/_index_pattern_filter.js b/test/functional/apps/management/_index_pattern_filter.js index 91ea13348d6115..eae53682b6ccf6 100644 --- a/test/functional/apps/management/_index_pattern_filter.js +++ b/test/functional/apps/management/_index_pattern_filter.js @@ -12,11 +12,10 @@ export default function ({ getService, getPageObjects }) { const kibanaServer = getService('kibanaServer'); const retry = getService('retry'); const PageObjects = getPageObjects(['settings']); - const esArchiver = getService('esArchiver'); describe('index pattern filter', function describeIndexTests() { before(async function () { - await esArchiver.emptyKibanaIndex(); + // delete .kibana index and then wait for Kibana to re-create it await kibanaServer.uiSettings.replace({}); await PageObjects.settings.navigateTo(); await PageObjects.settings.clickKibanaIndexPatterns(); diff --git a/test/functional/apps/management/_index_patterns_empty.ts b/test/functional/apps/management/_index_patterns_empty.ts index 4e86de6d706537..3b89e05d4b582c 100644 --- a/test/functional/apps/management/_index_patterns_empty.ts +++ b/test/functional/apps/management/_index_patterns_empty.ts @@ -19,7 +19,7 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) { describe('index pattern empty view', () => { before(async () => { - await esArchiver.emptyKibanaIndex(); + await esArchiver.load('empty_kibana'); await esArchiver.unload('logstash_functional'); await esArchiver.unload('makelogs'); await kibanaServer.uiSettings.replace({}); @@ -27,6 +27,7 @@ export default function ({ getPageObjects, getService }: FtrProviderContext) { }); after(async () => { + await esArchiver.unload('empty_kibana'); await esArchiver.loadIfNeeded('makelogs'); // @ts-expect-error await es.transport.request({ diff --git a/test/functional/apps/management/_mgmt_import_saved_objects.js b/test/functional/apps/management/_mgmt_import_saved_objects.js index 87eca2c7a5a652..45a18d59327643 100644 --- a/test/functional/apps/management/_mgmt_import_saved_objects.js +++ b/test/functional/apps/management/_mgmt_import_saved_objects.js @@ -18,13 +18,14 @@ export default function ({ getService, getPageObjects }) { describe('mgmt saved objects', function describeIndexTests() { beforeEach(async function () { - await esArchiver.emptyKibanaIndex(); + await esArchiver.load('empty_kibana'); await esArchiver.load('discover'); await PageObjects.settings.navigateTo(); }); afterEach(async function () { await esArchiver.unload('discover'); + await esArchiver.load('empty_kibana'); }); it('should import saved objects mgmt', async function () { diff --git a/test/functional/apps/management/_test_huge_fields.js b/test/functional/apps/management/_test_huge_fields.js index c1fca31e695cb0..2ab619276d2b91 100644 --- a/test/functional/apps/management/_test_huge_fields.js +++ b/test/functional/apps/management/_test_huge_fields.js @@ -20,7 +20,6 @@ export default function ({ getService, getPageObjects }) { const EXPECTED_FIELD_COUNT = '10006'; before(async function () { await security.testUser.setRoles(['kibana_admin', 'test_testhuge_reader']); - await esArchiver.emptyKibanaIndex(); await esArchiver.loadIfNeeded('large_fields'); await PageObjects.settings.createIndexPattern('testhuge', 'date'); }); diff --git a/test/functional/apps/management/index.ts b/test/functional/apps/management/index.ts index 06e652f9f3e59a..ca898538750279 100644 --- a/test/functional/apps/management/index.ts +++ b/test/functional/apps/management/index.ts @@ -14,11 +14,13 @@ export default function ({ getService, loadTestFile }: FtrProviderContext) { describe('management', function () { before(async () => { await esArchiver.unload('logstash_functional'); + await esArchiver.load('empty_kibana'); await esArchiver.loadIfNeeded('makelogs'); }); after(async () => { await esArchiver.unload('makelogs'); + await esArchiver.unload('empty_kibana'); }); describe('', function () { diff --git a/test/functional/apps/visualize/input_control_vis/input_control_range.ts b/test/functional/apps/visualize/input_control_vis/input_control_range.ts index 613b1a162eb632..9b48e78246b37b 100644 --- a/test/functional/apps/visualize/input_control_vis/input_control_range.ts +++ b/test/functional/apps/visualize/input_control_vis/input_control_range.ts @@ -12,6 +12,7 @@ import { FtrProviderContext } from '../../../ftr_provider_context'; export default function ({ getService, getPageObjects }: FtrProviderContext) { const esArchiver = getService('esArchiver'); + const kibanaServer = getService('kibanaServer'); const find = getService('find'); const security = getService('security'); const { visualize, visEditor } = getPageObjects(['visualize', 'visEditor']); @@ -52,6 +53,7 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { await esArchiver.loadIfNeeded('logstash_functional'); await esArchiver.loadIfNeeded('long_window_logstash'); await esArchiver.load('visualize'); + await kibanaServer.uiSettings.replace({ defaultIndex: 'logstash-*' }); await security.testUser.restoreDefaults(); }); }); diff --git a/test/plugin_functional/test_suites/core_plugins/applications.ts b/test/plugin_functional/test_suites/core_plugins/applications.ts index 52924d8c932805..e72d032f63469a 100644 --- a/test/plugin_functional/test_suites/core_plugins/applications.ts +++ b/test/plugin_functional/test_suites/core_plugins/applications.ts @@ -19,7 +19,6 @@ export default function ({ getService, getPageObjects }: PluginFunctionalProvide const find = getService('find'); const retry = getService('retry'); const deployment = getService('deployment'); - const esArchiver = getService('esArchiver'); const loadingScreenNotShown = async () => expect(await testSubjects.exists('kbnLoadingMessage')).to.be(false); @@ -51,7 +50,6 @@ export default function ({ getService, getPageObjects }: PluginFunctionalProvide describe('ui applications', function describeIndexTests() { before(async () => { - await esArchiver.emptyKibanaIndex(); await PageObjects.common.navigateToApp('foo'); }); diff --git a/test/plugin_functional/test_suites/data_plugin/index_patterns.ts b/test/plugin_functional/test_suites/data_plugin/index_patterns.ts index 0cd53a5e1b7647..ba12e2df16d411 100644 --- a/test/plugin_functional/test_suites/data_plugin/index_patterns.ts +++ b/test/plugin_functional/test_suites/data_plugin/index_patterns.ts @@ -12,12 +12,8 @@ import '../../plugins/core_provider_plugin/types'; export default function ({ getService }: PluginFunctionalProviderContext) { const supertest = getService('supertest'); - const esArchiver = getService('esArchiver'); describe('index patterns', function () { - before(async () => { - await esArchiver.emptyKibanaIndex(); - }); let indexPatternId = ''; it('can create an index pattern', async () => { diff --git a/test/plugin_functional/test_suites/saved_objects_management/import_warnings.ts b/test/plugin_functional/test_suites/saved_objects_management/import_warnings.ts index b60e4b4a1d8b7f..71663b19b35cb7 100644 --- a/test/plugin_functional/test_suites/saved_objects_management/import_warnings.ts +++ b/test/plugin_functional/test_suites/saved_objects_management/import_warnings.ts @@ -10,15 +10,10 @@ import path from 'path'; import expect from '@kbn/expect'; import { PluginFunctionalProviderContext } from '../../services'; -export default function ({ getPageObjects, getService }: PluginFunctionalProviderContext) { +export default function ({ getPageObjects }: PluginFunctionalProviderContext) { const PageObjects = getPageObjects(['common', 'settings', 'header', 'savedObjects']); - const esArchiver = getService('esArchiver'); describe('import warnings', () => { - before(async () => { - await esArchiver.emptyKibanaIndex(); - }); - beforeEach(async () => { await PageObjects.settings.navigateTo(); await PageObjects.settings.clickKibanaSavedObjects(); diff --git a/test/scripts/jenkins_build_load_testing.sh b/test/scripts/jenkins_build_load_testing.sh index aeb584b106498c..659321f1d39751 100755 --- a/test/scripts/jenkins_build_load_testing.sh +++ b/test/scripts/jenkins_build_load_testing.sh @@ -1,5 +1,13 @@ #!/usr/bin/env bash +while getopts s: flag +do + case "${flag}" in + s) simulations=${OPTARG};; + esac +done +echo "Simulation classes: $simulations"; + cd "$KIBANA_DIR" source src/dev/ci_setup/setup_env.sh @@ -25,6 +33,7 @@ echo " -> test setup" source test/scripts/jenkins_test_setup_xpack.sh echo " -> run gatling load testing" +export GATLING_SIMULATIONS="$simulations" node scripts/functional_tests \ - --kibana-install-dir "$KIBANA_INSTALL_DIR" \ - --config test/load/config.ts + --kibana-install-dir "$KIBANA_INSTALL_DIR" \ + --config test/load/config.ts diff --git a/test/security_functional/insecure_cluster_warning.ts b/test/security_functional/insecure_cluster_warning.ts index 2f7656b743a51c..181c4cf2b46b77 100644 --- a/test/security_functional/insecure_cluster_warning.ts +++ b/test/security_functional/insecure_cluster_warning.ts @@ -31,7 +31,6 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { before(async () => { await browser.setLocalStorageItem('insecureClusterWarningVisibility', ''); await esArchiver.unload('hamlet'); - await esArchiver.emptyKibanaIndex(); }); it('should not warn when the cluster contains no user data', async () => { diff --git a/test/tsconfig.json b/test/tsconfig.json index 1dc58f7b25c241..c3acf94f8c2679 100644 --- a/test/tsconfig.json +++ b/test/tsconfig.json @@ -26,6 +26,7 @@ { "path": "../src/plugins/expressions/tsconfig.json" }, { "path": "../src/plugins/home/tsconfig.json" }, { "path": "../src/plugins/inspector/tsconfig.json" }, + { "path": "../src/plugins/kibana_overview/tsconfig.json" }, { "path": "../src/plugins/kibana_react/tsconfig.json" }, { "path": "../src/plugins/kibana_usage_collection/tsconfig.json" }, { "path": "../src/plugins/kibana_utils/tsconfig.json" }, diff --git a/tsconfig.json b/tsconfig.json index 21760919c89e93..f6e0fbc8d9e971 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -24,6 +24,7 @@ "src/plugins/input_control_vis/**/*", "src/plugins/inspector/**/*", "src/plugins/kibana_legacy/**/*", + "src/plugins/kibana_overview/**/*", "src/plugins/kibana_react/**/*", "src/plugins/kibana_usage_collection/**/*", "src/plugins/kibana_utils/**/*", @@ -84,6 +85,7 @@ { "path": "./src/plugins/home/tsconfig.json" }, { "path": "./src/plugins/inspector/tsconfig.json" }, { "path": "./src/plugins/kibana_legacy/tsconfig.json" }, + { "path": "./src/plugins/kibana_overview/tsconfig.json" }, { "path": "./src/plugins/kibana_react/tsconfig.json" }, { "path": "./src/plugins/kibana_usage_collection/tsconfig.json" }, { "path": "./src/plugins/kibana_utils/tsconfig.json" }, diff --git a/tsconfig.refs.json b/tsconfig.refs.json index 1d08e764709cad..17b1fc5dc1fe91 100644 --- a/tsconfig.refs.json +++ b/tsconfig.refs.json @@ -17,6 +17,7 @@ { "path": "./src/plugins/home/tsconfig.json" }, { "path": "./src/plugins/inspector/tsconfig.json" }, { "path": "./src/plugins/kibana_legacy/tsconfig.json" }, + { "path": "./src/plugins/kibana_overview/tsconfig.json" }, { "path": "./src/plugins/kibana_react/tsconfig.json" }, { "path": "./src/plugins/kibana_usage_collection/tsconfig.json" }, { "path": "./src/plugins/kibana_utils/tsconfig.json" }, diff --git a/x-pack/plugins/actions/README.md b/x-pack/plugins/actions/README.md index 9472cbf400a6a4..1eb94af4dddf8c 100644 --- a/x-pack/plugins/actions/README.md +++ b/x-pack/plugins/actions/README.md @@ -70,12 +70,14 @@ Table of Contents - [`params`](#params-6) - [`subActionParams (pushToService)`](#subactionparams-pushtoservice) - [`subActionParams (getFields)`](#subactionparams-getfields) + - [`subActionParams (getIncident)`](#subactionparams-getincident) + - [`subActionParams (getChoices)`](#subactionparams-getchoices) - [Jira](#jira) - [`config`](#config-7) - [`secrets`](#secrets-7) - [`params`](#params-7) - [`subActionParams (pushToService)`](#subactionparams-pushtoservice-1) - - [`subActionParams (getIncident)`](#subactionparams-getincident) + - [`subActionParams (getIncident)`](#subactionparams-getincident-1) - [`subActionParams (issueTypes)`](#subactionparams-issuetypes) - [`subActionParams (fieldsByIssueType)`](#subactionparams-fieldsbyissuetype) - [`subActionParams (issues)`](#subactionparams-issues) @@ -347,17 +349,18 @@ const result = await actionsClient.execute({ Kibana ships with a set of built-in action types: -| Type | Id | Description | -| ------------------------------- | ------------- | ------------------------------------------------------------------ | -| [Server log](#server-log) | `.server-log` | Logs messages to the Kibana log using Kibana's logger | -| [Email](#email) | `.email` | Sends an email using SMTP | -| [Slack](#slack) | `.slack` | Posts a message to a slack channel | -| [Index](#index) | `.index` | Indexes document(s) into Elasticsearch | -| [Webhook](#webhook) | `.webhook` | Send a payload to a web service using HTTP POST or PUT | -| [PagerDuty](#pagerduty) | `.pagerduty` | Trigger, resolve, or acknowlege an incident to a PagerDuty service | -| [ServiceNow](#servicenow) | `.servicenow` | Create or update an incident to a ServiceNow instance | -| [Jira](#jira) | `.jira` | Create or update an issue to a Jira instance | -| [IBM Resilient](#ibm-resilient) | `.resilient` | Create or update an incident to a IBM Resilient instance | +| Type | Id | Description | +| ------------------------------- | ----------------- | ------------------------------------------------------------------ | +| [Server log](#server-log) | `.server-log` | Logs messages to the Kibana log using Kibana's logger | +| [Email](#email) | `.email` | Sends an email using SMTP | +| [Slack](#slack) | `.slack` | Posts a message to a slack channel | +| [Index](#index) | `.index` | Indexes document(s) into Elasticsearch | +| [Webhook](#webhook) | `.webhook` | Send a payload to a web service using HTTP POST or PUT | +| [PagerDuty](#pagerduty) | `.pagerduty` | Trigger, resolve, or acknowlege an incident to a PagerDuty service | +| [ServiceNow ITSM](#servicenow) | `.servicenow` | Create or update an incident to a ServiceNow ITSM instance | +| [ServiceNow SIR](#servicenow) | `.servicenow-sir` | Create or update an incident to a ServiceNow SIR instance | +| [Jira](#jira) | `.jira` | Create or update an issue to a Jira instance | +| [IBM Resilient](#ibm-resilient) | `.resilient` | Create or update an incident to a IBM Resilient instance | --- @@ -549,9 +552,11 @@ For more details see [PagerDuty v2 event parameters](https://v2.developer.pagerd ## ServiceNow -ID: `.servicenow` +ServiceNow ITSM ID: `.servicenow` -The ServiceNow action uses the [V2 Table API](https://developer.servicenow.com/app.do#!/rest_api_doc?v=orlando&id=c_TableAPI) to create and update ServiceNow incidents. +ServiceNow SIR ID: `.servicenow-sir` + +The ServiceNow actions use the [V2 Table API](https://developer.servicenow.com/app.do#!/rest_api_doc?v=orlando&id=c_TableAPI) to create and update ServiceNow incidents. Both action types use the same `config`, `secrets`, and `params` schema. ### `config` @@ -568,10 +573,10 @@ The ServiceNow action uses the [V2 Table API](https://developer.servicenow.com/a ### `params` -| Property | Description | Type | -| --------------- | --------------------------------------------------------------------- | ------ | -| subAction | The sub action to perform. It can be `getFields`, and `pushToService` | string | -| subActionParams | The parameters of the sub action | object | +| Property | Description | Type | +| --------------- | -------------------------------------------------------------------------------------------------- | ------ | +| subAction | The sub action to perform. It can be `pushToService`, `getFields`, `getIncident`, and `getChoices` | string | +| subActionParams | The parameters of the sub action | object | #### `subActionParams (pushToService)` @@ -595,6 +600,19 @@ The following table describes the properties of the `incident` object. No parameters for `getFields` sub-action. Provide an empty object `{}`. +#### `subActionParams (getIncident)` + +| Property | Description | Type | +| ---------- | ------------------------------------- | ------ | +| externalId | The id of the incident in ServiceNow. | string | + + +#### `subActionParams (getChoices)` + +| Property | Description | Type | +| -------- | ------------------------------------------------------------ | -------- | +| fields | An array of fields. Example: `[priority, category, impact]`. | string[] | + --- ## Jira diff --git a/x-pack/plugins/actions/server/builtin_action_types/index.ts b/x-pack/plugins/actions/server/builtin_action_types/index.ts index 3a01b875ec4a0e..21161ff8ad0dd6 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/index.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/index.ts @@ -14,7 +14,7 @@ import { getActionType as getPagerDutyActionType } from './pagerduty'; import { getActionType as getServerLogActionType } from './server_log'; import { getActionType as getSlackActionType } from './slack'; import { getActionType as getWebhookActionType } from './webhook'; -import { getActionType as getServiceNowActionType } from './servicenow'; +import { getServiceNowITSMActionType, getServiceNowSIRActionType } from './servicenow'; import { getActionType as getJiraActionType } from './jira'; import { getActionType as getResilientActionType } from './resilient'; import { getActionType as getTeamsActionType } from './teams'; @@ -38,7 +38,8 @@ export { } from './webhook'; export { ActionParamsType as ServiceNowActionParams, - ActionTypeId as ServiceNowActionTypeId, + ServiceNowITSMActionTypeId, + ServiceNowSIRActionTypeId, } from './servicenow'; export { ActionParamsType as JiraActionParams, ActionTypeId as JiraActionTypeId } from './jira'; export { @@ -66,7 +67,8 @@ export function registerBuiltInActionTypes({ actionTypeRegistry.register(getServerLogActionType({ logger })); actionTypeRegistry.register(getSlackActionType({ logger, configurationUtilities })); actionTypeRegistry.register(getWebhookActionType({ logger, configurationUtilities })); - actionTypeRegistry.register(getServiceNowActionType({ logger, configurationUtilities })); + actionTypeRegistry.register(getServiceNowITSMActionType({ logger, configurationUtilities })); + actionTypeRegistry.register(getServiceNowSIRActionType({ logger, configurationUtilities })); actionTypeRegistry.register(getJiraActionType({ logger, configurationUtilities })); actionTypeRegistry.register(getResilientActionType({ logger, configurationUtilities })); actionTypeRegistry.register(getTeamsActionType({ logger, configurationUtilities })); diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.test.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.test.ts index 772cd16cc4d512..ef5de9fc487bc2 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.test.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.test.ts @@ -5,7 +5,7 @@ */ import { Logger } from '../../../../../../src/core/server'; -import { externalServiceMock, apiParams, serviceNowCommonFields } from './mocks'; +import { externalServiceMock, apiParams, serviceNowCommonFields, serviceNowChoices } from './mocks'; import { ExternalService } from './types'; import { api } from './api'; let mockedLogger: jest.Mocked; @@ -235,4 +235,14 @@ describe('api', () => { expect(res).toEqual(serviceNowCommonFields); }); }); + + describe('getChoices', () => { + test('it returns the fields correctly', async () => { + const res = await api.getChoices({ + externalService, + params: { fields: ['priority'] }, + }); + expect(res).toEqual(serviceNowChoices); + }); + }); }); diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.ts index 9981a8431a736a..7f5747277a4e91 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/api.ts @@ -5,6 +5,8 @@ */ import { ExternalServiceApi, + GetChoicesHandlerArgs, + GetChoicesResponse, GetCommonFieldsHandlerArgs, GetCommonFieldsResponse, GetIncidentApiHandlerArgs, @@ -71,7 +73,16 @@ const getFieldsHandler = async ({ return res; }; +const getChoicesHandler = async ({ + externalService, + params, +}: GetChoicesHandlerArgs): Promise => { + const res = await externalService.getChoices(params.fields); + return res; +}; + export const api: ExternalServiceApi = { + getChoices: getChoicesHandler, getFields: getFieldsHandler, getIncident: getIncidentHandler, handshake: handshakeHandler, diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/index.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/index.ts index 107d86f111deb1..fd4991e5f7e984 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/index.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/index.ts @@ -11,7 +11,8 @@ import { validate } from './validators'; import { ExternalIncidentServiceConfiguration, ExternalIncidentServiceSecretConfiguration, - ExecutorParamsSchema, + ExecutorParamsSchemaITSM, + ExecutorParamsSchemaSIR, } from './schema'; import { ActionsConfigurationUtilities } from '../../actions_config'; import { ActionType, ActionTypeExecutorOptions, ActionTypeExecutorResult } from '../../types'; @@ -27,18 +28,26 @@ import { PushToServiceResponse, ExecutorSubActionCommonFieldsParams, ServiceNowExecutorResultData, + ExecutorSubActionGetChoicesParams, } from './types'; -export type ActionParamsType = TypeOf; +export type ActionParamsType = + | TypeOf + | TypeOf; interface GetActionTypeParams { logger: Logger; configurationUtilities: ActionsConfigurationUtilities; } -export const ActionTypeId = '.servicenow'; +const serviceNowITSMTable = 'incident'; +const serviceNowSIRTable = 'sn_si_incident'; + +export const ServiceNowITSMActionTypeId = '.servicenow'; +export const ServiceNowSIRActionTypeId = '.servicenow-sir'; + // action type definition -export function getActionType( +export function getServiceNowITSMActionType( params: GetActionTypeParams ): ActionType< ServiceNowPublicConfigurationType, @@ -48,9 +57,9 @@ export function getActionType( > { const { logger, configurationUtilities } = params; return { - id: ActionTypeId, + id: ServiceNowITSMActionTypeId, minimumLicenseRequired: 'platinum', - name: i18n.NAME, + name: i18n.SERVICENOW_ITSM, validate: { config: schema.object(ExternalIncidentServiceConfiguration, { validate: curry(validate.config)(configurationUtilities), @@ -58,19 +67,46 @@ export function getActionType( secrets: schema.object(ExternalIncidentServiceSecretConfiguration, { validate: curry(validate.secrets)(configurationUtilities), }), - params: ExecutorParamsSchema, + params: ExecutorParamsSchemaITSM, }, - executor: curry(executor)({ logger, configurationUtilities }), + executor: curry(executor)({ logger, configurationUtilities, table: serviceNowITSMTable }), + }; +} + +export function getServiceNowSIRActionType( + params: GetActionTypeParams +): ActionType< + ServiceNowPublicConfigurationType, + ServiceNowSecretConfigurationType, + ExecutorParams, + PushToServiceResponse | {} +> { + const { logger, configurationUtilities } = params; + return { + id: ServiceNowSIRActionTypeId, + minimumLicenseRequired: 'platinum', + name: i18n.SERVICENOW_SIR, + validate: { + config: schema.object(ExternalIncidentServiceConfiguration, { + validate: curry(validate.config)(configurationUtilities), + }), + secrets: schema.object(ExternalIncidentServiceSecretConfiguration, { + validate: curry(validate.secrets)(configurationUtilities), + }), + params: ExecutorParamsSchemaSIR, + }, + executor: curry(executor)({ logger, configurationUtilities, table: serviceNowSIRTable }), }; } // action executor -const supportedSubActions: string[] = ['getFields', 'pushToService']; +const supportedSubActions: string[] = ['getFields', 'pushToService', 'getChoices', 'getIncident']; async function executor( { logger, configurationUtilities, - }: { logger: Logger; configurationUtilities: ActionsConfigurationUtilities }, + table, + }: { logger: Logger; configurationUtilities: ActionsConfigurationUtilities; table: string }, execOptions: ActionTypeExecutorOptions< ServiceNowPublicConfigurationType, ServiceNowSecretConfigurationType, @@ -82,6 +118,7 @@ async function executor( let data: ServiceNowExecutorResultData | null = null; const externalService = createExternalService( + table, { config, secrets, @@ -122,5 +159,13 @@ async function executor( }); } + if (subAction === 'getChoices') { + const getChoicesParams = subActionParams as ExecutorSubActionGetChoicesParams; + data = await api.getChoices({ + externalService, + params: getChoicesParams, + }); + } + return { status: 'ok', data: data ?? {}, actionId }; } diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/mocks.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/mocks.ts index 9d9b1e164e7ddf..f958cdb73ebfc5 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/mocks.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/mocks.ts @@ -4,7 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -import { ExternalService, PushToServiceApiParams, ExecutorSubActionPushParams } from './types'; +import { ExternalService, ExecutorSubActionPushParams } from './types'; export const serviceNowCommonFields = [ { @@ -33,8 +33,43 @@ export const serviceNowCommonFields = [ element: 'sys_updated_by', }, ]; + +export const serviceNowChoices = [ + { + dependent_value: '', + label: '1 - Critical', + value: '1', + element: 'priority', + }, + { + dependent_value: '', + label: '2 - High', + value: '2', + element: 'priority', + }, + { + dependent_value: '', + label: '3 - Moderate', + value: '3', + element: 'priority', + }, + { + dependent_value: '', + label: '4 - Low', + value: '4', + element: 'priority', + }, + { + dependent_value: '', + label: '5 - Planning', + value: '5', + element: 'priority', + }, +]; + const createMock = (): jest.Mocked => { const service = { + getChoices: jest.fn().mockImplementation(() => Promise.resolve(serviceNowChoices)), getFields: jest.fn().mockImplementation(() => Promise.resolve(serviceNowCommonFields)), getIncident: jest.fn().mockImplementation(() => Promise.resolve({ @@ -89,8 +124,6 @@ const executorParams: ExecutorSubActionPushParams = { ], }; -const apiParams: PushToServiceApiParams = { - ...executorParams, -}; +const apiParams = executorParams; export { externalServiceMock, executorParams, apiParams }; diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/schema.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/schema.ts index 1c05fa93f2362e..5c7de935223a88 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/schema.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/schema.ts @@ -28,25 +28,48 @@ export const ExecutorSubActionSchema = schema.oneOf([ schema.literal('getIncident'), schema.literal('pushToService'), schema.literal('handshake'), + schema.literal('getChoices'), ]); -export const ExecutorSubActionPushParamsSchema = schema.object({ +const CommentsSchema = schema.nullable( + schema.arrayOf( + schema.object({ + comment: schema.string(), + commentId: schema.string(), + }) + ) +); + +const CommonAttributes = { + short_description: schema.string(), + description: schema.nullable(schema.string()), + externalId: schema.nullable(schema.string()), +}; + +// Schema for ServiceNow Incident Management (ITSM) +export const ExecutorSubActionPushParamsSchemaITSM = schema.object({ incident: schema.object({ - short_description: schema.string(), - description: schema.nullable(schema.string()), - externalId: schema.nullable(schema.string()), + ...CommonAttributes, severity: schema.nullable(schema.string()), urgency: schema.nullable(schema.string()), impact: schema.nullable(schema.string()), }), - comments: schema.nullable( - schema.arrayOf( - schema.object({ - comment: schema.string(), - commentId: schema.string(), - }) - ) - ), + comments: CommentsSchema, +}); + +// Schema for ServiceNow Security Incident Response (SIR) +export const ExecutorSubActionPushParamsSchemaSIR = schema.object({ + incident: schema.object({ + ...CommonAttributes, + category: schema.nullable(schema.string()), + dest_ip: schema.nullable(schema.string()), + malware_hash: schema.nullable(schema.string()), + malware_url: schema.nullable(schema.string()), + priority: schema.nullable(schema.string()), + source_ip: schema.nullable(schema.string()), + subcategory: schema.nullable(schema.string()), + }), + comments: CommentsSchema, }); export const ExecutorSubActionGetIncidentParamsSchema = schema.object({ @@ -56,8 +79,36 @@ export const ExecutorSubActionGetIncidentParamsSchema = schema.object({ // Reserved for future implementation export const ExecutorSubActionHandshakeParamsSchema = schema.object({}); export const ExecutorSubActionCommonFieldsParamsSchema = schema.object({}); +export const ExecutorSubActionGetChoicesParamsSchema = schema.object({ + fields: schema.arrayOf(schema.string()), +}); + +// Executor parameters for ServiceNow Incident Management (ITSM) +export const ExecutorParamsSchemaITSM = schema.oneOf([ + schema.object({ + subAction: schema.literal('getFields'), + subActionParams: ExecutorSubActionCommonFieldsParamsSchema, + }), + schema.object({ + subAction: schema.literal('getIncident'), + subActionParams: ExecutorSubActionGetIncidentParamsSchema, + }), + schema.object({ + subAction: schema.literal('handshake'), + subActionParams: ExecutorSubActionHandshakeParamsSchema, + }), + schema.object({ + subAction: schema.literal('pushToService'), + subActionParams: ExecutorSubActionPushParamsSchemaITSM, + }), + schema.object({ + subAction: schema.literal('getChoices'), + subActionParams: ExecutorSubActionGetChoicesParamsSchema, + }), +]); -export const ExecutorParamsSchema = schema.oneOf([ +// Executor parameters for ServiceNow Security Incident Response (SIR) +export const ExecutorParamsSchemaSIR = schema.oneOf([ schema.object({ subAction: schema.literal('getFields'), subActionParams: ExecutorSubActionCommonFieldsParamsSchema, @@ -72,6 +123,10 @@ export const ExecutorParamsSchema = schema.oneOf([ }), schema.object({ subAction: schema.literal('pushToService'), - subActionParams: ExecutorSubActionPushParamsSchema, + subActionParams: ExecutorSubActionPushParamsSchemaSIR, + }), + schema.object({ + subAction: schema.literal('getChoices'), + subActionParams: ExecutorSubActionGetChoicesParamsSchema, }), ]); diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.test.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.test.ts index 4ef0e7da166e65..18f3a2f3ff379c 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.test.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.test.ts @@ -12,7 +12,7 @@ import { ExternalService } from './types'; import { Logger } from '../../../../../../src/core/server'; import { loggingSystemMock } from '../../../../../../src/core/server/mocks'; import { actionsConfigMock } from '../../actions_config.mock'; -import { serviceNowCommonFields } from './mocks'; +import { serviceNowCommonFields, serviceNowChoices } from './mocks'; const logger = loggingSystemMock.create().get() as jest.Mocked; jest.mock('axios'); @@ -29,12 +29,14 @@ axios.create = jest.fn(() => axios); const requestMock = utils.request as jest.Mock; const patchMock = utils.patch as jest.Mock; const configurationUtilities = actionsConfigMock.create(); +const table = 'incident'; describe('ServiceNow service', () => { let service: ExternalService; - beforeAll(() => { + beforeEach(() => { service = createExternalService( + table, { // The trailing slash at the end of the url is intended. // All API calls need to have the trailing slash removed. @@ -54,6 +56,7 @@ describe('ServiceNow service', () => { test('throws without url', () => { expect(() => createExternalService( + table, { config: { apiUrl: null }, secrets: { username: 'admin', password: 'admin' }, @@ -67,6 +70,7 @@ describe('ServiceNow service', () => { test('throws without username', () => { expect(() => createExternalService( + table, { config: { apiUrl: 'test.com' }, secrets: { username: '', password: 'admin' }, @@ -80,6 +84,7 @@ describe('ServiceNow service', () => { test('throws without password', () => { expect(() => createExternalService( + table, { config: { apiUrl: 'test.com' }, secrets: { username: '', password: undefined }, @@ -114,6 +119,30 @@ describe('ServiceNow service', () => { }); }); + test('it should call request with correct arguments when table changes', async () => { + service = createExternalService( + 'sn_si_incident', + { + config: { apiUrl: 'https://dev102283.service-now.com/' }, + secrets: { username: 'admin', password: 'admin' }, + }, + logger, + configurationUtilities + ); + + requestMock.mockImplementation(() => ({ + data: { result: { sys_id: '1', number: 'INC01' } }, + })); + + await service.getIncident('1'); + expect(requestMock).toHaveBeenCalledWith({ + axios, + logger, + configurationUtilities, + url: 'https://dev102283.service-now.com/api/now/v2/table/sn_si_incident/1', + }); + }); + test('it should throw an error', async () => { requestMock.mockImplementation(() => { throw new Error('An error has occurred'); @@ -122,6 +151,17 @@ describe('ServiceNow service', () => { 'Unable to get incident with id 1. Error: An error has occurred' ); }); + + test('it should throw an error when instance is not alive', async () => { + requestMock.mockImplementation(() => ({ + status: 200, + data: {}, + request: { connection: { servername: 'Developer instance' } }, + })); + await expect(service.getIncident('1')).rejects.toThrow( + 'There is an issue with your Service Now Instance. Please check Developer instance.' + ); + }); }); describe('createIncident', () => { @@ -161,6 +201,39 @@ describe('ServiceNow service', () => { }); }); + test('it should call request with correct arguments when table changes', async () => { + service = createExternalService( + 'sn_si_incident', + { + config: { apiUrl: 'https://dev102283.service-now.com/' }, + secrets: { username: 'admin', password: 'admin' }, + }, + logger, + configurationUtilities + ); + + requestMock.mockImplementation(() => ({ + data: { result: { sys_id: '1', number: 'INC01', sys_created_on: '2020-03-10 12:24:20' } }, + })); + + const res = await service.createIncident({ + incident: { short_description: 'title', description: 'desc' }, + }); + + expect(requestMock).toHaveBeenCalledWith({ + axios, + logger, + configurationUtilities, + url: 'https://dev102283.service-now.com/api/now/v2/table/sn_si_incident', + method: 'post', + data: { short_description: 'title', description: 'desc' }, + }); + + expect(res.url).toEqual( + 'https://dev102283.service-now.com/nav_to.do?uri=sn_si_incident.do?sys_id=1' + ); + }); + test('it should throw an error', async () => { requestMock.mockImplementation(() => { throw new Error('An error has occurred'); @@ -174,6 +247,17 @@ describe('ServiceNow service', () => { '[Action][ServiceNow]: Unable to create incident. Error: An error has occurred' ); }); + + test('it should throw an error when instance is not alive', async () => { + requestMock.mockImplementation(() => ({ + status: 200, + data: {}, + request: { connection: { servername: 'Developer instance' } }, + })); + await expect(service.getIncident('1')).rejects.toThrow( + 'There is an issue with your Service Now Instance. Please check Developer instance.' + ); + }); }); describe('updateIncident', () => { @@ -214,6 +298,39 @@ describe('ServiceNow service', () => { }); }); + test('it should call request with correct arguments when table changes', async () => { + service = createExternalService( + 'sn_si_incident', + { + config: { apiUrl: 'https://dev102283.service-now.com/' }, + secrets: { username: 'admin', password: 'admin' }, + }, + logger, + configurationUtilities + ); + + patchMock.mockImplementation(() => ({ + data: { result: { sys_id: '1', number: 'INC01', sys_updated_on: '2020-03-10 12:24:20' } }, + })); + + const res = await service.updateIncident({ + incidentId: '1', + incident: { short_description: 'title', description: 'desc' }, + }); + + expect(patchMock).toHaveBeenCalledWith({ + axios, + logger, + configurationUtilities, + url: 'https://dev102283.service-now.com/api/now/v2/table/sn_si_incident/1', + data: { short_description: 'title', description: 'desc' }, + }); + + expect(res.url).toEqual( + 'https://dev102283.service-now.com/nav_to.do?uri=sn_si_incident.do?sys_id=1' + ); + }); + test('it should throw an error', async () => { patchMock.mockImplementation(() => { throw new Error('An error has occurred'); @@ -228,6 +345,7 @@ describe('ServiceNow service', () => { '[Action][ServiceNow]: Unable to update incident with id 1. Error: An error has occurred' ); }); + test('it creates the comment correctly', async () => { patchMock.mockImplementation(() => ({ data: { result: { sys_id: '11', number: 'INC011', sys_updated_on: '2020-03-10 12:24:20' } }, @@ -245,6 +363,17 @@ describe('ServiceNow service', () => { url: 'https://dev102283.service-now.com/nav_to.do?uri=incident.do?sys_id=11', }); }); + + test('it should throw an error when instance is not alive', async () => { + requestMock.mockImplementation(() => ({ + status: 200, + data: {}, + request: { connection: { servername: 'Developer instance' } }, + })); + await expect(service.getIncident('1')).rejects.toThrow( + 'There is an issue with your Service Now Instance. Please check Developer instance.' + ); + }); }); describe('getFields', () => { @@ -259,9 +388,10 @@ describe('ServiceNow service', () => { logger, configurationUtilities, url: - 'https://dev102283.service-now.com/api/now/v2/table/sys_dictionary?sysparm_query=name=task^internal_type=string&active=true&array=false&read_only=false&sysparm_fields=max_length,element,column_label,mandatory', + 'https://dev102283.service-now.com/api/now/v2/table/sys_dictionary?sysparm_query=name=task^ORname=incident^internal_type=string&active=true&array=false&read_only=false&sysparm_fields=max_length,element,column_label,mandatory', }); }); + test('it returns common fields correctly', async () => { requestMock.mockImplementation(() => ({ data: { result: serviceNowCommonFields }, @@ -270,6 +400,31 @@ describe('ServiceNow service', () => { expect(res).toEqual(serviceNowCommonFields); }); + test('it should call request with correct arguments when table changes', async () => { + service = createExternalService( + 'sn_si_incident', + { + config: { apiUrl: 'https://dev102283.service-now.com/' }, + secrets: { username: 'admin', password: 'admin' }, + }, + logger, + configurationUtilities + ); + + requestMock.mockImplementation(() => ({ + data: { result: serviceNowCommonFields }, + })); + await service.getFields(); + + expect(requestMock).toHaveBeenCalledWith({ + axios, + logger, + configurationUtilities, + url: + 'https://dev102283.service-now.com/api/now/v2/table/sys_dictionary?sysparm_query=name=task^ORname=sn_si_incident^internal_type=string&active=true&array=false&read_only=false&sysparm_fields=max_length,element,column_label,mandatory', + }); + }); + test('it should throw an error', async () => { requestMock.mockImplementation(() => { throw new Error('An error has occurred'); @@ -278,5 +433,87 @@ describe('ServiceNow service', () => { '[Action][ServiceNow]: Unable to get fields. Error: An error has occurred' ); }); + + test('it should throw an error when instance is not alive', async () => { + requestMock.mockImplementation(() => ({ + status: 200, + data: {}, + request: { connection: { servername: 'Developer instance' } }, + })); + await expect(service.getIncident('1')).rejects.toThrow( + 'There is an issue with your Service Now Instance. Please check Developer instance.' + ); + }); + }); + + describe('getChoices', () => { + test('it should call request with correct arguments', async () => { + requestMock.mockImplementation(() => ({ + data: { result: serviceNowChoices }, + })); + await service.getChoices(['priority', 'category']); + + expect(requestMock).toHaveBeenCalledWith({ + axios, + logger, + configurationUtilities, + url: + 'https://dev102283.service-now.com/api/now/v2/table/sys_choice?sysparm_query=name=task^ORname=incident^element=priority^ORelement=category&sysparm_fields=label,value,dependent_value,element', + }); + }); + + test('it returns common fields correctly', async () => { + requestMock.mockImplementation(() => ({ + data: { result: serviceNowChoices }, + })); + const res = await service.getChoices(['priority']); + expect(res).toEqual(serviceNowChoices); + }); + + test('it should call request with correct arguments when table changes', async () => { + service = createExternalService( + 'sn_si_incident', + { + config: { apiUrl: 'https://dev102283.service-now.com/' }, + secrets: { username: 'admin', password: 'admin' }, + }, + logger, + configurationUtilities + ); + + requestMock.mockImplementation(() => ({ + data: { result: serviceNowChoices }, + })); + + await service.getChoices(['priority', 'category']); + + expect(requestMock).toHaveBeenCalledWith({ + axios, + logger, + configurationUtilities, + url: + 'https://dev102283.service-now.com/api/now/v2/table/sys_choice?sysparm_query=name=task^ORname=sn_si_incident^element=priority^ORelement=category&sysparm_fields=label,value,dependent_value,element', + }); + }); + + test('it should throw an error', async () => { + requestMock.mockImplementation(() => { + throw new Error('An error has occurred'); + }); + await expect(service.getChoices(['priority'])).rejects.toThrow( + '[Action][ServiceNow]: Unable to get choices. Error: An error has occurred' + ); + }); + + test('it should throw an error when instance is not alive', async () => { + requestMock.mockImplementation(() => ({ + status: 200, + data: {}, + request: { connection: { servername: 'Developer instance' } }, + })); + await expect(service.getIncident('1')).rejects.toThrow( + 'There is an issue with your Service Now Instance. Please check Developer instance.' + ); + }); }); }); diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.ts index 108fe06bcbcaa0..7c7723c98a070c 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/service.ts @@ -15,13 +15,10 @@ import { request, getErrorMessage, addTimeZoneToDate, patch } from '../lib/axios import { ActionsConfigurationUtilities } from '../../actions_config'; const API_VERSION = 'v2'; -const INCIDENT_URL = `api/now/${API_VERSION}/table/incident`; const SYS_DICTIONARY = `api/now/${API_VERSION}/table/sys_dictionary`; -// Based on: https://docs.servicenow.com/bundle/orlando-platform-user-interface/page/use/navigation/reference/r_NavigatingByURLExamples.html -const VIEW_INCIDENT_URL = `nav_to.do?uri=incident.do?sys_id=`; - export const createExternalService = ( + table: string, { config, secrets }: ExternalServiceCredentials, logger: Logger, configurationUtilities: ActionsConfigurationUtilities @@ -30,24 +27,36 @@ export const createExternalService = ( const { username, password } = secrets as ServiceNowSecretConfigurationType; if (!url || !username || !password) { - throw Error(`[Action]${i18n.NAME}: Wrong configuration.`); + throw Error(`[Action]${i18n.SERVICENOW}: Wrong configuration.`); } const urlWithoutTrailingSlash = url.endsWith('/') ? url.slice(0, -1) : url; - const incidentUrl = `${urlWithoutTrailingSlash}/${INCIDENT_URL}`; - const fieldsUrl = `${urlWithoutTrailingSlash}/${SYS_DICTIONARY}?sysparm_query=name=task^internal_type=string&active=true&array=false&read_only=false&sysparm_fields=max_length,element,column_label,mandatory`; + const incidentUrl = `${urlWithoutTrailingSlash}/api/now/${API_VERSION}/table/${table}`; + const fieldsUrl = `${urlWithoutTrailingSlash}/${SYS_DICTIONARY}?sysparm_query=name=task^ORname=${table}^internal_type=string&active=true&array=false&read_only=false&sysparm_fields=max_length,element,column_label,mandatory`; + const choicesUrl = `${urlWithoutTrailingSlash}/api/now/${API_VERSION}/table/sys_choice`; const axiosInstance = axios.create({ auth: { username, password }, }); const getIncidentViewURL = (id: string) => { - return `${urlWithoutTrailingSlash}/${VIEW_INCIDENT_URL}${id}`; + // Based on: https://docs.servicenow.com/bundle/orlando-platform-user-interface/page/use/navigation/reference/r_NavigatingByURLExamples.html + return `${urlWithoutTrailingSlash}/nav_to.do?uri=${table}.do?sys_id=${id}`; + }; + + const getChoicesURL = (fields: string[]) => { + const elements = fields + .slice(1) + .reduce((acc, field) => `${acc}^ORelement=${field}`, `element=${fields[0]}`); + + return `${choicesUrl}?sysparm_query=name=task^ORname=${table}^${elements}&sysparm_fields=label,value,dependent_value,element`; }; const checkInstance = (res: AxiosResponse) => { if (res.status === 200 && res.data.result == null) { throw new Error( - `There is an issue with your Service Now Instance. Please check ${res.request.connection.servername}` + `There is an issue with your Service Now Instance. Please check ${ + res.request?.connection?.servername ?? '' + }.` ); } }; @@ -64,7 +73,10 @@ export const createExternalService = ( return { ...res.data.result }; } catch (error) { throw new Error( - getErrorMessage(i18n.NAME, `Unable to get incident with id ${id}. Error: ${error.message}`) + getErrorMessage( + i18n.SERVICENOW, + `Unable to get incident with id ${id}. Error: ${error.message}` + ) ); } }; @@ -82,7 +94,10 @@ export const createExternalService = ( return res.data.result.length > 0 ? { ...res.data.result } : undefined; } catch (error) { throw new Error( - getErrorMessage(i18n.NAME, `Unable to find incidents by query. Error: ${error.message}`) + getErrorMessage( + i18n.SERVICENOW, + `Unable to find incidents by query. Error: ${error.message}` + ) ); } }; @@ -106,7 +121,7 @@ export const createExternalService = ( }; } catch (error) { throw new Error( - getErrorMessage(i18n.NAME, `Unable to create incident. Error: ${error.message}`) + getErrorMessage(i18n.SERVICENOW, `Unable to create incident. Error: ${error.message}`) ); } }; @@ -130,7 +145,7 @@ export const createExternalService = ( } catch (error) { throw new Error( getErrorMessage( - i18n.NAME, + i18n.SERVICENOW, `Unable to update incident with id ${incidentId}. Error: ${error.message}` ) ); @@ -148,7 +163,26 @@ export const createExternalService = ( checkInstance(res); return res.data.result.length > 0 ? res.data.result : []; } catch (error) { - throw new Error(getErrorMessage(i18n.NAME, `Unable to get fields. Error: ${error.message}`)); + throw new Error( + getErrorMessage(i18n.SERVICENOW, `Unable to get fields. Error: ${error.message}`) + ); + } + }; + + const getChoices = async (fields: string[]) => { + try { + const res = await request({ + axios: axiosInstance, + url: getChoicesURL(fields), + logger, + configurationUtilities, + }); + checkInstance(res); + return res.data.result; + } catch (error) { + throw new Error( + getErrorMessage(i18n.SERVICENOW, `Unable to get choices. Error: ${error.message}`) + ); } }; @@ -158,5 +192,6 @@ export const createExternalService = ( getFields, getIncident, updateIncident, + getChoices, }; }; diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/translations.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/translations.ts index 287fe8cacda79e..84fe538e0a63a0 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/translations.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/translations.ts @@ -6,10 +6,18 @@ import { i18n } from '@kbn/i18n'; -export const NAME = i18n.translate('xpack.actions.builtin.servicenowTitle', { +export const SERVICENOW = i18n.translate('xpack.actions.builtin.serviceNowTitle', { defaultMessage: 'ServiceNow', }); +export const SERVICENOW_ITSM = i18n.translate('xpack.actions.builtin.serviceNowITSMTitle', { + defaultMessage: 'ServiceNow ITSM', +}); + +export const SERVICENOW_SIR = i18n.translate('xpack.actions.builtin.serviceNowSIRTitle', { + defaultMessage: 'ServiceNow SIR', +}); + export const ALLOWED_HOSTS_ERROR = (message: string) => i18n.translate('xpack.actions.builtin.configuration.apiAllowedHostsError', { defaultMessage: 'error configuring connector action: {message}', diff --git a/x-pack/plugins/actions/server/builtin_action_types/servicenow/types.ts b/x-pack/plugins/actions/server/builtin_action_types/servicenow/types.ts index 9868f5d1bea06d..c74d1fbffd759d 100644 --- a/x-pack/plugins/actions/server/builtin_action_types/servicenow/types.ts +++ b/x-pack/plugins/actions/server/builtin_action_types/servicenow/types.ts @@ -8,13 +8,16 @@ import { TypeOf } from '@kbn/config-schema'; import { - ExecutorParamsSchema, + ExecutorParamsSchemaITSM, ExecutorSubActionCommonFieldsParamsSchema, ExecutorSubActionGetIncidentParamsSchema, ExecutorSubActionHandshakeParamsSchema, - ExecutorSubActionPushParamsSchema, + ExecutorSubActionPushParamsSchemaITSM, ExternalIncidentServiceConfigurationSchema, ExternalIncidentServiceSecretConfigurationSchema, + ExecutorParamsSchemaSIR, + ExecutorSubActionPushParamsSchemaSIR, + ExecutorSubActionGetChoicesParamsSchema, } from './schema'; import { ActionsConfigurationUtilities } from '../../actions_config'; import { Logger } from '../../../../../../src/core/server'; @@ -30,14 +33,29 @@ export type ExecutorSubActionCommonFieldsParams = TypeOf< typeof ExecutorSubActionCommonFieldsParamsSchema >; -export type ServiceNowExecutorResultData = PushToServiceResponse | GetCommonFieldsResponse; +export type ExecutorSubActionGetChoicesParams = TypeOf< + typeof ExecutorSubActionGetChoicesParamsSchema +>; + +export type ServiceNowExecutorResultData = + | PushToServiceResponse + | GetCommonFieldsResponse + | GetChoicesResponse; export interface CreateCommentRequest { [key: string]: string; } -export type ExecutorParams = TypeOf; -export type ExecutorSubActionPushParams = TypeOf; +export type ExecutorParams = + | TypeOf + | TypeOf; + +export type ExecutorSubActionPushParamsITSM = TypeOf; +export type ExecutorSubActionPushParamsSIR = TypeOf; + +export type ExecutorSubActionPushParams = + | ExecutorSubActionPushParamsITSM + | ExecutorSubActionPushParamsSIR; export interface ExternalServiceCredentials { config: Record; @@ -62,14 +80,17 @@ export interface PushToServiceResponse extends ExternalServiceIncidentResponse { export type ExternalServiceParams = Record; export interface ExternalService { - getFields: () => Promise; + getChoices: (fields: string[]) => Promise; getIncident: (id: string) => Promise; + getFields: () => Promise; createIncident: (params: ExternalServiceParams) => Promise; updateIncident: (params: ExternalServiceParams) => Promise; findIncidents: (params?: Record) => Promise; } export type PushToServiceApiParams = ExecutorSubActionPushParams; +export type PushToServiceApiParamsITSM = ExecutorSubActionPushParamsITSM; +export type PushToServiceApiParamsSIR = ExecutorSubActionPushParamsSIR; export interface ExternalServiceApiHandlerArgs { externalService: ExternalService; @@ -83,7 +104,17 @@ export type ExecutorSubActionHandshakeParams = TypeOf< typeof ExecutorSubActionHandshakeParamsSchema >; -export type Incident = Omit; +export type ServiceNowITSMIncident = Omit< + TypeOf['incident'], + 'externalId' +>; + +export type ServiceNowSIRIncident = Omit< + TypeOf['incident'], + 'externalId' +>; + +export type Incident = ServiceNowITSMIncident | ServiceNowSIRIncident; export interface PushToServiceApiHandlerArgs extends ExternalServiceApiHandlerArgs { params: PushToServiceApiParams; @@ -104,13 +135,29 @@ export interface ExternalServiceFields { max_length: string; element: string; } + +export interface ExternalServiceChoices { + value: string; + label: string; + dependent_value: string; + element: string; +} + export type GetCommonFieldsResponse = ExternalServiceFields[]; +export type GetChoicesResponse = ExternalServiceChoices[]; + export interface GetCommonFieldsHandlerArgs { externalService: ExternalService; params: ExecutorSubActionCommonFieldsParams; } +export interface GetChoicesHandlerArgs { + externalService: ExternalService; + params: ExecutorSubActionGetChoicesParams; +} + export interface ExternalServiceApi { + getChoices: (args: GetChoicesHandlerArgs) => Promise; getFields: (args: GetCommonFieldsHandlerArgs) => Promise; handshake: (args: HandshakeApiHandlerArgs) => Promise; pushToService: (args: PushToServiceApiHandlerArgs) => Promise; diff --git a/x-pack/plugins/actions/server/index.ts b/x-pack/plugins/actions/server/index.ts index 4e59dfd0998110..b573bcfc109142 100644 --- a/x-pack/plugins/actions/server/index.ts +++ b/x-pack/plugins/actions/server/index.ts @@ -35,7 +35,8 @@ export type { SlackActionParams, WebhookActionTypeId, WebhookActionParams, - ServiceNowActionTypeId, + ServiceNowITSMActionTypeId, + ServiceNowSIRActionTypeId, ServiceNowActionParams, JiraActionTypeId, JiraActionParams, diff --git a/x-pack/plugins/apm/public/application/action_menu/anomaly_detection_setup_link.tsx b/x-pack/plugins/apm/public/application/action_menu/anomaly_detection_setup_link.tsx index 8a1d73c8189445..a221f4bfb05a98 100644 --- a/x-pack/plugins/apm/public/application/action_menu/anomaly_detection_setup_link.tsx +++ b/x-pack/plugins/apm/public/application/action_menu/anomaly_detection_setup_link.tsx @@ -57,7 +57,9 @@ export function AnomalyDetectionSetupLink() { export function MissingJobsAlert({ environment }: { environment?: string }) { const { data = DEFAULT_DATA, status } = useFetcher( (callApmApi) => - callApmApi({ endpoint: `GET /api/apm/settings/anomaly-detection/jobs` }), + callApmApi({ + endpoint: `GET /api/apm/settings/anomaly-detection/jobs`, + }), [], { preservePreviousData: false, showToastOnError: false } ); diff --git a/x-pack/plugins/apm/public/components/alerting/error_count_alert_trigger/index.tsx b/x-pack/plugins/apm/public/components/alerting/error_count_alert_trigger/index.tsx index d7375d14e17cf9..3b68eccbd9dc40 100644 --- a/x-pack/plugins/apm/public/components/alerting/error_count_alert_trigger/index.tsx +++ b/x-pack/plugins/apm/public/components/alerting/error_count_alert_trigger/index.tsx @@ -13,7 +13,6 @@ import { asInteger } from '../../../../common/utils/formatters'; import { useUrlParams } from '../../../context/url_params_context/use_url_params'; import { useEnvironmentsFetcher } from '../../../hooks/use_environments_fetcher'; import { useFetcher } from '../../../hooks/use_fetcher'; -import { callApmApi } from '../../../services/rest/createCallApmApi'; import { ChartPreview } from '../chart_preview'; import { EnvironmentField, IsAboveField, ServiceField } from '../fields'; import { getAbsoluteTimeRange } from '../helper'; @@ -46,20 +45,23 @@ export function ErrorCountAlertTrigger(props: Props) { const { threshold, windowSize, windowUnit, environment } = alertParams; - const { data } = useFetcher(() => { - if (windowSize && windowUnit) { - return callApmApi({ - endpoint: 'GET /api/apm/alerts/chart_preview/transaction_error_count', - params: { - query: { - ...getAbsoluteTimeRange(windowSize, windowUnit), - environment, - serviceName, + const { data } = useFetcher( + (callApmApi) => { + if (windowSize && windowUnit) { + return callApmApi({ + endpoint: 'GET /api/apm/alerts/chart_preview/transaction_error_count', + params: { + query: { + ...getAbsoluteTimeRange(windowSize, windowUnit), + environment, + serviceName, + }, }, - }, - }); - } - }, [windowSize, windowUnit, environment, serviceName]); + }); + } + }, + [windowSize, windowUnit, environment, serviceName] + ); const defaults = { threshold: 25, diff --git a/x-pack/plugins/apm/public/components/alerting/transaction_duration_alert_trigger/index.tsx b/x-pack/plugins/apm/public/components/alerting/transaction_duration_alert_trigger/index.tsx index 7c0a74f2e1b60c..4d28cdaec3782b 100644 --- a/x-pack/plugins/apm/public/components/alerting/transaction_duration_alert_trigger/index.tsx +++ b/x-pack/plugins/apm/public/components/alerting/transaction_duration_alert_trigger/index.tsx @@ -8,7 +8,6 @@ import { i18n } from '@kbn/i18n'; import { map } from 'lodash'; import React from 'react'; import { useParams } from 'react-router-dom'; -import { useFetcher } from '../../../../../observability/public'; import { ForLastExpression } from '../../../../../triggers_actions_ui/public'; import { ENVIRONMENT_ALL } from '../../../../common/environment_filter_values'; import { getDurationFormatter } from '../../../../common/utils/formatters'; @@ -16,7 +15,7 @@ import { TimeSeries } from '../../../../typings/timeseries'; import { useApmServiceContext } from '../../../context/apm_service/use_apm_service_context'; import { useUrlParams } from '../../../context/url_params_context/use_url_params'; import { useEnvironmentsFetcher } from '../../../hooks/use_environments_fetcher'; -import { callApmApi } from '../../../services/rest/createCallApmApi'; +import { useFetcher } from '../../../hooks/use_fetcher'; import { getMaxY, getResponseTimeTickFormatter, @@ -88,29 +87,32 @@ export function TransactionDurationAlertTrigger(props: Props) { windowUnit, } = alertParams; - const { data } = useFetcher(() => { - if (windowSize && windowUnit) { - return callApmApi({ - endpoint: 'GET /api/apm/alerts/chart_preview/transaction_duration', - params: { - query: { - ...getAbsoluteTimeRange(windowSize, windowUnit), - aggregationType, - environment, - serviceName, - transactionType: alertParams.transactionType, + const { data } = useFetcher( + (callApmApi) => { + if (windowSize && windowUnit) { + return callApmApi({ + endpoint: 'GET /api/apm/alerts/chart_preview/transaction_duration', + params: { + query: { + ...getAbsoluteTimeRange(windowSize, windowUnit), + aggregationType, + environment, + serviceName, + transactionType: alertParams.transactionType, + }, }, - }, - }); - } - }, [ - aggregationType, - environment, - serviceName, - alertParams.transactionType, - windowSize, - windowUnit, - ]); + }); + } + }, + [ + aggregationType, + environment, + serviceName, + alertParams.transactionType, + windowSize, + windowUnit, + ] + ); const maxY = getMaxY([ { data: data ?? [] } as TimeSeries<{ x: number; y: number | null }>, diff --git a/x-pack/plugins/apm/public/components/alerting/transaction_error_rate_alert_trigger/index.tsx b/x-pack/plugins/apm/public/components/alerting/transaction_error_rate_alert_trigger/index.tsx index e06f39ec10220c..58adbb7b172c85 100644 --- a/x-pack/plugins/apm/public/components/alerting/transaction_error_rate_alert_trigger/index.tsx +++ b/x-pack/plugins/apm/public/components/alerting/transaction_error_rate_alert_trigger/index.tsx @@ -12,7 +12,6 @@ import { useApmServiceContext } from '../../../context/apm_service/use_apm_servi import { useUrlParams } from '../../../context/url_params_context/use_url_params'; import { useEnvironmentsFetcher } from '../../../hooks/use_environments_fetcher'; import { useFetcher } from '../../../hooks/use_fetcher'; -import { callApmApi } from '../../../services/rest/createCallApmApi'; import { ChartPreview } from '../chart_preview'; import { EnvironmentField, @@ -54,27 +53,30 @@ export function TransactionErrorRateAlertTrigger(props: Props) { const thresholdAsPercent = (threshold ?? 0) / 100; - const { data } = useFetcher(() => { - if (windowSize && windowUnit) { - return callApmApi({ - endpoint: 'GET /api/apm/alerts/chart_preview/transaction_error_rate', - params: { - query: { - ...getAbsoluteTimeRange(windowSize, windowUnit), - environment, - serviceName, - transactionType: alertParams.transactionType, + const { data } = useFetcher( + (callApmApi) => { + if (windowSize && windowUnit) { + return callApmApi({ + endpoint: 'GET /api/apm/alerts/chart_preview/transaction_error_rate', + params: { + query: { + ...getAbsoluteTimeRange(windowSize, windowUnit), + environment, + serviceName, + transactionType: alertParams.transactionType, + }, }, - }, - }); - } - }, [ - alertParams.transactionType, - environment, - serviceName, - windowSize, - windowUnit, - ]); + }); + } + }, + [ + alertParams.transactionType, + environment, + serviceName, + windowSize, + windowUnit, + ] + ); if (serviceName && !transactionTypes.length) { return null; diff --git a/x-pack/plugins/apm/public/components/app/Correlations/ErrorCorrelations.tsx b/x-pack/plugins/apm/public/components/app/Correlations/ErrorCorrelations.tsx index 25973b9bda3888..891a2081804cc3 100644 --- a/x-pack/plugins/apm/public/components/app/Correlations/ErrorCorrelations.tsx +++ b/x-pack/plugins/apm/public/components/app/Correlations/ErrorCorrelations.tsx @@ -25,10 +25,7 @@ import { } from '@elastic/eui'; import { useUrlParams } from '../../../context/url_params_context/use_url_params'; import { FETCH_STATUS, useFetcher } from '../../../hooks/use_fetcher'; -import { - APIReturnType, - callApmApi, -} from '../../../services/rest/createCallApmApi'; +import { APIReturnType } from '../../../services/rest/createCallApmApi'; import { px } from '../../../style/variables'; import { SignificantTermsTable } from './SignificantTermsTable'; import { ChartContainer } from '../../shared/charts/chart_container'; @@ -65,32 +62,35 @@ export function ErrorCorrelations() { const { urlParams, uiFilters } = useUrlParams(); const { transactionName, transactionType, start, end } = urlParams; - const { data, status } = useFetcher(() => { - if (start && end) { - return callApmApi({ - endpoint: 'GET /api/apm/correlations/failed_transactions', - params: { - query: { - serviceName, - transactionName, - transactionType, - start, - end, - uiFilters: JSON.stringify(uiFilters), - fieldNames: fieldNames.map((field) => field.label).join(','), + const { data, status } = useFetcher( + (callApmApi) => { + if (start && end) { + return callApmApi({ + endpoint: 'GET /api/apm/correlations/failed_transactions', + params: { + query: { + serviceName, + transactionName, + transactionType, + start, + end, + uiFilters: JSON.stringify(uiFilters), + fieldNames: fieldNames.map((field) => field.label).join(','), + }, }, - }, - }); - } - }, [ - serviceName, - start, - end, - transactionName, - transactionType, - uiFilters, - fieldNames, - ]); + }); + } + }, + [ + serviceName, + start, + end, + transactionName, + transactionType, + uiFilters, + fieldNames, + ] + ); return ( <> diff --git a/x-pack/plugins/apm/public/components/app/Correlations/LatencyCorrelations.tsx b/x-pack/plugins/apm/public/components/app/Correlations/LatencyCorrelations.tsx index 438303110fbc46..493c6e04ffbb1a 100644 --- a/x-pack/plugins/apm/public/components/app/Correlations/LatencyCorrelations.tsx +++ b/x-pack/plugins/apm/public/components/app/Correlations/LatencyCorrelations.tsx @@ -26,10 +26,7 @@ import { import { getDurationFormatter } from '../../../../common/utils/formatters'; import { useUrlParams } from '../../../context/url_params_context/use_url_params'; import { FETCH_STATUS, useFetcher } from '../../../hooks/use_fetcher'; -import { - APIReturnType, - callApmApi, -} from '../../../services/rest/createCallApmApi'; +import { APIReturnType } from '../../../services/rest/createCallApmApi'; import { SignificantTermsTable } from './SignificantTermsTable'; import { ChartContainer } from '../../shared/charts/chart_container'; @@ -65,34 +62,37 @@ export function LatencyCorrelations() { const { urlParams, uiFilters } = useUrlParams(); const { transactionName, transactionType, start, end } = urlParams; - const { data, status } = useFetcher(() => { - if (start && end) { - return callApmApi({ - endpoint: 'GET /api/apm/correlations/slow_transactions', - params: { - query: { - serviceName, - transactionName, - transactionType, - start, - end, - uiFilters: JSON.stringify(uiFilters), - durationPercentile, - fieldNames: fieldNames.map((field) => field.label).join(','), + const { data, status } = useFetcher( + (callApmApi) => { + if (start && end) { + return callApmApi({ + endpoint: 'GET /api/apm/correlations/slow_transactions', + params: { + query: { + serviceName, + transactionName, + transactionType, + start, + end, + uiFilters: JSON.stringify(uiFilters), + durationPercentile, + fieldNames: fieldNames.map((field) => field.label).join(','), + }, }, - }, - }); - } - }, [ - serviceName, - start, - end, - transactionName, - transactionType, - uiFilters, - durationPercentile, - fieldNames, - ]); + }); + } + }, + [ + serviceName, + start, + end, + transactionName, + transactionType, + uiFilters, + durationPercentile, + fieldNames, + ] + ); return ( <> diff --git a/x-pack/plugins/apm/public/components/app/ErrorGroupDetails/index.tsx b/x-pack/plugins/apm/public/components/app/ErrorGroupDetails/index.tsx index 95ebd5d4036de2..9501474e3be65f 100644 --- a/x-pack/plugins/apm/public/components/app/ErrorGroupDetails/index.tsx +++ b/x-pack/plugins/apm/public/components/app/ErrorGroupDetails/index.tsx @@ -23,7 +23,6 @@ import { useTrackPageview } from '../../../../../observability/public'; import { NOT_AVAILABLE_LABEL } from '../../../../common/i18n'; import { useFetcher } from '../../../hooks/use_fetcher'; import { useUrlParams } from '../../../context/url_params_context/use_url_params'; -import { callApmApi } from '../../../services/rest/createCallApmApi'; import { fontFamilyCode, fontSizes, px, units } from '../../../style/variables'; import { ApmHeader } from '../../shared/ApmHeader'; import { SearchBar } from '../../shared/search_bar'; @@ -70,24 +69,27 @@ export function ErrorGroupDetails({ location, match }: ErrorGroupDetailsProps) { const { urlParams, uiFilters } = useUrlParams(); const { start, end } = urlParams; - const { data: errorGroupData } = useFetcher(() => { - if (start && end) { - return callApmApi({ - endpoint: 'GET /api/apm/services/{serviceName}/errors/{groupId}', - params: { - path: { - serviceName, - groupId, + const { data: errorGroupData } = useFetcher( + (callApmApi) => { + if (start && end) { + return callApmApi({ + endpoint: 'GET /api/apm/services/{serviceName}/errors/{groupId}', + params: { + path: { + serviceName, + groupId, + }, + query: { + start, + end, + uiFilters: JSON.stringify(uiFilters), + }, }, - query: { - start, - end, - uiFilters: JSON.stringify(uiFilters), - }, - }, - }); - } - }, [serviceName, start, end, groupId, uiFilters]); + }); + } + }, + [serviceName, start, end, groupId, uiFilters] + ); const { errorDistributionData } = useErrorGroupDistributionFetcher({ serviceName, diff --git a/x-pack/plugins/apm/public/components/app/ErrorGroupOverview/index.tsx b/x-pack/plugins/apm/public/components/app/ErrorGroupOverview/index.tsx index 71cb8e0e016022..af8c667df6ca2d 100644 --- a/x-pack/plugins/apm/public/components/app/ErrorGroupOverview/index.tsx +++ b/x-pack/plugins/apm/public/components/app/ErrorGroupOverview/index.tsx @@ -18,7 +18,6 @@ import { useTrackPageview } from '../../../../../observability/public'; import { Projection } from '../../../../common/projections'; import { useFetcher } from '../../../hooks/use_fetcher'; import { useUrlParams } from '../../../context/url_params_context/use_url_params'; -import { callApmApi } from '../../../services/rest/createCallApmApi'; import { LocalUIFilters } from '../../shared/LocalUIFilters'; import { SearchBar } from '../../shared/search_bar'; import { ErrorDistribution } from '../ErrorGroupDetails/Distribution'; @@ -37,27 +36,30 @@ function ErrorGroupOverview({ serviceName }: ErrorGroupOverviewProps) { groupId: undefined, }); - const { data: errorGroupListData } = useFetcher(() => { - const normalizedSortDirection = sortDirection === 'asc' ? 'asc' : 'desc'; + const { data: errorGroupListData } = useFetcher( + (callApmApi) => { + const normalizedSortDirection = sortDirection === 'asc' ? 'asc' : 'desc'; - if (start && end) { - return callApmApi({ - endpoint: 'GET /api/apm/services/{serviceName}/errors', - params: { - path: { - serviceName, + if (start && end) { + return callApmApi({ + endpoint: 'GET /api/apm/services/{serviceName}/errors', + params: { + path: { + serviceName, + }, + query: { + start, + end, + sortField, + sortDirection: normalizedSortDirection, + uiFilters: JSON.stringify(uiFilters), + }, }, - query: { - start, - end, - sortField, - sortDirection: normalizedSortDirection, - uiFilters: JSON.stringify(uiFilters), - }, - }, - }); - } - }, [serviceName, start, end, sortField, sortDirection, uiFilters]); + }); + } + }, + [serviceName, start, end, sortField, sortDirection, uiFilters] + ); useTrackPageview({ app: 'apm', diff --git a/x-pack/plugins/apm/public/components/app/RumDashboard/ux_overview_fetchers.ts b/x-pack/plugins/apm/public/components/app/RumDashboard/ux_overview_fetchers.ts index 7ce9d3f25354cc..0468fbabcdb41d 100644 --- a/x-pack/plugins/apm/public/components/app/RumDashboard/ux_overview_fetchers.ts +++ b/x-pack/plugins/apm/public/components/app/RumDashboard/ux_overview_fetchers.ts @@ -21,6 +21,7 @@ export const fetchUxOverviewDate = async ({ }: FetchDataParams): Promise => { const data = await callApmApi({ endpoint: 'GET /api/apm/rum-client/web-core-vitals', + signal: null, params: { query: { start: new Date(absoluteTime.start).toISOString(), @@ -41,6 +42,7 @@ export async function hasRumData({ }: HasDataParams): Promise { return await callApmApi({ endpoint: 'GET /api/apm/observability_overview/has_rum_data', + signal: null, params: { query: { start: new Date(absoluteTime.start).toISOString(), diff --git a/x-pack/plugins/apm/public/components/app/ServiceMap/index.tsx b/x-pack/plugins/apm/public/components/app/ServiceMap/index.tsx index 6f8d0589031833..463c9f36fb2fed 100644 --- a/x-pack/plugins/apm/public/components/app/ServiceMap/index.tsx +++ b/x-pack/plugins/apm/public/components/app/ServiceMap/index.tsx @@ -17,7 +17,6 @@ import { FETCH_STATUS, useFetcher } from '../../../hooks/use_fetcher'; import { useLicenseContext } from '../../../context/license/use_license_context'; import { useTheme } from '../../../hooks/use_theme'; import { useUrlParams } from '../../../context/url_params_context/use_url_params'; -import { callApmApi } from '../../../services/rest/createCallApmApi'; import { DatePicker } from '../../shared/DatePicker'; import { LicensePrompt } from '../../shared/LicensePrompt'; import { Controls } from './Controls'; @@ -86,28 +85,31 @@ export function ServiceMap({ const license = useLicenseContext(); const { urlParams } = useUrlParams(); - const { data = { elements: [] }, status, error } = useFetcher(() => { - // When we don't have a license or a valid license, don't make the request. - if (!license || !isActivePlatinumLicense(license)) { - return; - } - - const { start, end, environment } = urlParams; - if (start && end) { - return callApmApi({ - isCachable: false, - endpoint: 'GET /api/apm/service-map', - params: { - query: { - start, - end, - environment, - serviceName, + const { data = { elements: [] }, status, error } = useFetcher( + (callApmApi) => { + // When we don't have a license or a valid license, don't make the request. + if (!license || !isActivePlatinumLicense(license)) { + return; + } + + const { start, end, environment } = urlParams; + if (start && end) { + return callApmApi({ + isCachable: false, + endpoint: 'GET /api/apm/service-map', + params: { + query: { + start, + end, + environment, + serviceName, + }, }, - }, - }); - } - }, [license, serviceName, urlParams]); + }); + } + }, + [license, serviceName, urlParams] + ); const { ref, height } = useRefDimensions(); diff --git a/x-pack/plugins/apm/public/components/app/Settings/AgentConfigurations/AgentConfigurationCreateEdit/SettingsPage/saveConfig.ts b/x-pack/plugins/apm/public/components/app/Settings/AgentConfigurations/AgentConfigurationCreateEdit/SettingsPage/saveConfig.ts index e15a57ff7539eb..a9c334f414db23 100644 --- a/x-pack/plugins/apm/public/components/app/Settings/AgentConfigurations/AgentConfigurationCreateEdit/SettingsPage/saveConfig.ts +++ b/x-pack/plugins/apm/public/components/app/Settings/AgentConfigurations/AgentConfigurationCreateEdit/SettingsPage/saveConfig.ts @@ -26,6 +26,7 @@ export async function saveConfig({ try { await callApmApi({ endpoint: 'PUT /api/apm/settings/agent-configuration', + signal: null, params: { query: { overwrite: isEditMode }, body: { diff --git a/x-pack/plugins/apm/public/components/app/Settings/AgentConfigurations/List/ConfirmDeleteModal.tsx b/x-pack/plugins/apm/public/components/app/Settings/AgentConfigurations/List/ConfirmDeleteModal.tsx index 958aafa8159df7..09251efe8b9773 100644 --- a/x-pack/plugins/apm/public/components/app/Settings/AgentConfigurations/List/ConfirmDeleteModal.tsx +++ b/x-pack/plugins/apm/public/components/app/Settings/AgentConfigurations/List/ConfirmDeleteModal.tsx @@ -73,6 +73,7 @@ async function deleteConfig( try { await callApmApi({ endpoint: 'DELETE /api/apm/settings/agent-configuration', + signal: null, params: { body: { service: { diff --git a/x-pack/plugins/apm/public/components/app/Settings/ApmIndices/index.tsx b/x-pack/plugins/apm/public/components/app/Settings/ApmIndices/index.tsx index 8c10b96c51ce2f..29fca54547e7ab 100644 --- a/x-pack/plugins/apm/public/components/app/Settings/ApmIndices/index.tsx +++ b/x-pack/plugins/apm/public/components/app/Settings/ApmIndices/index.tsx @@ -74,6 +74,7 @@ async function saveApmIndices({ }) { await callApmApi({ endpoint: 'POST /api/apm/settings/apm-indices/save', + signal: null, params: { body: apmIndices, }, diff --git a/x-pack/plugins/apm/public/components/app/Settings/CustomizeUI/CustomLink/CreateEditCustomLinkFlyout/DeleteButton.tsx b/x-pack/plugins/apm/public/components/app/Settings/CustomizeUI/CustomLink/CreateEditCustomLinkFlyout/DeleteButton.tsx index ffcb85384642ac..9257d5d78b6b3d 100644 --- a/x-pack/plugins/apm/public/components/app/Settings/CustomizeUI/CustomLink/CreateEditCustomLinkFlyout/DeleteButton.tsx +++ b/x-pack/plugins/apm/public/components/app/Settings/CustomizeUI/CustomLink/CreateEditCustomLinkFlyout/DeleteButton.tsx @@ -48,6 +48,7 @@ async function deleteConfig( try { await callApmApi({ endpoint: 'DELETE /api/apm/settings/custom_links/{id}', + signal: null, params: { path: { id: customLinkId }, }, diff --git a/x-pack/plugins/apm/public/components/app/Settings/CustomizeUI/CustomLink/CreateEditCustomLinkFlyout/LinkPreview.tsx b/x-pack/plugins/apm/public/components/app/Settings/CustomizeUI/CustomLink/CreateEditCustomLinkFlyout/LinkPreview.tsx index 25fd8f7ad3caf5..626aca6218ea39 100644 --- a/x-pack/plugins/apm/public/components/app/Settings/CustomizeUI/CustomLink/CreateEditCustomLinkFlyout/LinkPreview.tsx +++ b/x-pack/plugins/apm/public/components/app/Settings/CustomizeUI/CustomLink/CreateEditCustomLinkFlyout/LinkPreview.tsx @@ -31,6 +31,7 @@ interface Props { const fetchTransaction = debounce( async (filters: Filter[], callback: (transaction: Transaction) => void) => { const transaction = await callApmApi({ + signal: null, endpoint: 'GET /api/apm/settings/custom_links/transaction', params: { query: convertFiltersToQuery(filters) }, }); diff --git a/x-pack/plugins/apm/public/components/app/Settings/CustomizeUI/CustomLink/CreateEditCustomLinkFlyout/saveCustomLink.ts b/x-pack/plugins/apm/public/components/app/Settings/CustomizeUI/CustomLink/CreateEditCustomLinkFlyout/saveCustomLink.ts index cb1eaf6bca3f04..2d172d652ad810 100644 --- a/x-pack/plugins/apm/public/components/app/Settings/CustomizeUI/CustomLink/CreateEditCustomLinkFlyout/saveCustomLink.ts +++ b/x-pack/plugins/apm/public/components/app/Settings/CustomizeUI/CustomLink/CreateEditCustomLinkFlyout/saveCustomLink.ts @@ -35,6 +35,7 @@ export async function saveCustomLink({ if (id) { await callApmApi({ endpoint: 'PUT /api/apm/settings/custom_links/{id}', + signal: null, params: { path: { id }, body: customLink, @@ -43,6 +44,7 @@ export async function saveCustomLink({ } else { await callApmApi({ endpoint: 'POST /api/apm/settings/custom_links', + signal: null, params: { body: customLink, }, diff --git a/x-pack/plugins/apm/public/components/app/Settings/anomaly_detection/create_jobs.ts b/x-pack/plugins/apm/public/components/app/Settings/anomaly_detection/create_jobs.ts index 7106a4c48ef70e..dc73bf12ff4b89 100644 --- a/x-pack/plugins/apm/public/components/app/Settings/anomaly_detection/create_jobs.ts +++ b/x-pack/plugins/apm/public/components/app/Settings/anomaly_detection/create_jobs.ts @@ -28,6 +28,7 @@ export async function createJobs({ try { await callApmApi({ endpoint: 'POST /api/apm/settings/anomaly-detection/jobs', + signal: null, params: { body: { environments }, }, diff --git a/x-pack/plugins/apm/public/components/app/service_inventory/use_anomaly_detection_jobs_fetcher.ts b/x-pack/plugins/apm/public/components/app/service_inventory/use_anomaly_detection_jobs_fetcher.ts index 901841ac4d593f..eacb09bde70ac8 100644 --- a/x-pack/plugins/apm/public/components/app/service_inventory/use_anomaly_detection_jobs_fetcher.ts +++ b/x-pack/plugins/apm/public/components/app/service_inventory/use_anomaly_detection_jobs_fetcher.ts @@ -8,7 +8,9 @@ import { useFetcher } from '../../../hooks/use_fetcher'; export function useAnomalyDetectionJobsFetcher() { const { data, status } = useFetcher( (callApmApi) => - callApmApi({ endpoint: `GET /api/apm/settings/anomaly-detection/jobs` }), + callApmApi({ + endpoint: `GET /api/apm/settings/anomaly-detection/jobs`, + }), [], { showToastOnError: false } ); diff --git a/x-pack/plugins/apm/public/components/app/service_overview/service_overview_dependencies_table/index.tsx b/x-pack/plugins/apm/public/components/app/service_overview/service_overview_dependencies_table/index.tsx index 1bd7310e3251df..e93b29025426d0 100644 --- a/x-pack/plugins/apm/public/components/app/service_overview/service_overview_dependencies_table/index.tsx +++ b/x-pack/plugins/apm/public/components/app/service_overview/service_overview_dependencies_table/index.tsx @@ -26,7 +26,6 @@ import { import { ServiceDependencyItem } from '../../../../../server/lib/services/get_service_dependencies'; import { useUrlParams } from '../../../../context/url_params_context/use_url_params'; import { FETCH_STATUS, useFetcher } from '../../../../hooks/use_fetcher'; -import { callApmApi } from '../../../../services/rest/createCallApmApi'; import { px, unit } from '../../../../style/variables'; import { AgentIcon } from '../../../shared/AgentIcon'; import { SparkPlot } from '../../../shared/charts/spark_plot'; @@ -167,26 +166,29 @@ export function ServiceOverviewDependenciesTable({ serviceName }: Props) { }, ]; - const { data = [], status } = useFetcher(() => { - if (!start || !end) { - return; - } + const { data = [], status } = useFetcher( + (callApmApi) => { + if (!start || !end) { + return; + } - return callApmApi({ - endpoint: 'GET /api/apm/services/{serviceName}/dependencies', - params: { - path: { - serviceName, + return callApmApi({ + endpoint: 'GET /api/apm/services/{serviceName}/dependencies', + params: { + path: { + serviceName, + }, + query: { + start, + end, + environment: environment || ENVIRONMENT_ALL.value, + numBuckets: 20, + }, }, - query: { - start, - end, - environment: environment || ENVIRONMENT_ALL.value, - numBuckets: 20, - }, - }, - }); - }, [start, end, serviceName, environment]); + }); + }, + [start, end, serviceName, environment] + ); // need top-level sortable fields for the managed table const items = data.map((item) => ({ diff --git a/x-pack/plugins/apm/public/components/app/service_overview/service_overview_errors_table/index.tsx b/x-pack/plugins/apm/public/components/app/service_overview/service_overview_errors_table/index.tsx index d14ef648c22d3d..c523728025674d 100644 --- a/x-pack/plugins/apm/public/components/app/service_overview/service_overview_errors_table/index.tsx +++ b/x-pack/plugins/apm/public/components/app/service_overview/service_overview_errors_table/index.tsx @@ -16,7 +16,6 @@ import { asInteger } from '../../../../../common/utils/formatters'; import { useApmServiceContext } from '../../../../context/apm_service/use_apm_service_context'; import { useUrlParams } from '../../../../context/url_params_context/use_url_params'; import { FETCH_STATUS, useFetcher } from '../../../../hooks/use_fetcher'; -import { callApmApi } from '../../../../services/rest/createCallApmApi'; import { px, unit } from '../../../../style/variables'; import { SparkPlot } from '../../../shared/charts/spark_plot'; import { ErrorDetailLink } from '../../../shared/Links/apm/ErrorDetailLink'; @@ -140,50 +139,53 @@ export function ServiceOverviewErrorsTable({ serviceName }: Props) { }, }, status, - } = useFetcher(() => { - if (!start || !end || !transactionType) { - return; - } + } = useFetcher( + (callApmApi) => { + if (!start || !end || !transactionType) { + return; + } - return callApmApi({ - endpoint: 'GET /api/apm/services/{serviceName}/error_groups', - params: { - path: { serviceName }, - query: { - start, - end, - uiFilters: JSON.stringify(uiFilters), - size: PAGE_SIZE, - numBuckets: 20, - pageIndex: tableOptions.pageIndex, - sortField: tableOptions.sort.field, - sortDirection: tableOptions.sort.direction, - transactionType, - }, - }, - }).then((response) => { - return { - items: response.error_groups, - totalItemCount: response.total_error_groups, - tableOptions: { - pageIndex: tableOptions.pageIndex, - sort: { - field: tableOptions.sort.field, - direction: tableOptions.sort.direction, + return callApmApi({ + endpoint: 'GET /api/apm/services/{serviceName}/error_groups', + params: { + path: { serviceName }, + query: { + start, + end, + uiFilters: JSON.stringify(uiFilters), + size: PAGE_SIZE, + numBuckets: 20, + pageIndex: tableOptions.pageIndex, + sortField: tableOptions.sort.field, + sortDirection: tableOptions.sort.direction, + transactionType, }, }, - }; - }); - }, [ - start, - end, - serviceName, - uiFilters, - tableOptions.pageIndex, - tableOptions.sort.field, - tableOptions.sort.direction, - transactionType, - ]); + }).then((response) => { + return { + items: response.error_groups, + totalItemCount: response.total_error_groups, + tableOptions: { + pageIndex: tableOptions.pageIndex, + sort: { + field: tableOptions.sort.field, + direction: tableOptions.sort.direction, + }, + }, + }; + }); + }, + [ + start, + end, + serviceName, + uiFilters, + tableOptions.pageIndex, + tableOptions.sort.field, + tableOptions.sort.direction, + transactionType, + ] + ); const { items, diff --git a/x-pack/plugins/apm/public/components/app/service_overview/service_overview_instances_chart_and_table.tsx b/x-pack/plugins/apm/public/components/app/service_overview/service_overview_instances_chart_and_table.tsx index f7c2891bb3e659..a0528b7220cc1e 100644 --- a/x-pack/plugins/apm/public/components/app/service_overview/service_overview_instances_chart_and_table.tsx +++ b/x-pack/plugins/apm/public/components/app/service_overview/service_overview_instances_chart_and_table.tsx @@ -9,7 +9,6 @@ import React from 'react'; import { useApmServiceContext } from '../../../context/apm_service/use_apm_service_context'; import { useUrlParams } from '../../../context/url_params_context/use_url_params'; import { useFetcher } from '../../../hooks/use_fetcher'; -import { callApmApi } from '../../../services/rest/createCallApmApi'; import { InstancesLatencyDistributionChart } from '../../shared/charts/instances_latency_distribution_chart'; import { ServiceOverviewInstancesTable } from './service_overview_instances_table'; @@ -29,28 +28,31 @@ export function ServiceOverviewInstancesChartAndTable({ uiFilters, } = useUrlParams(); - const { data = [], status } = useFetcher(() => { - if (!start || !end || !transactionType) { - return; - } + const { data = [], status } = useFetcher( + (callApmApi) => { + if (!start || !end || !transactionType) { + return; + } - return callApmApi({ - endpoint: - 'GET /api/apm/services/{serviceName}/service_overview_instances', - params: { - path: { - serviceName, + return callApmApi({ + endpoint: + 'GET /api/apm/services/{serviceName}/service_overview_instances', + params: { + path: { + serviceName, + }, + query: { + start, + end, + transactionType, + uiFilters: JSON.stringify(uiFilters), + numBuckets: 20, + }, }, - query: { - start, - end, - transactionType, - uiFilters: JSON.stringify(uiFilters), - numBuckets: 20, - }, - }, - }); - }, [start, end, serviceName, transactionType, uiFilters]); + }); + }, + [start, end, serviceName, transactionType, uiFilters] + ); return ( <> diff --git a/x-pack/plugins/apm/public/components/app/service_overview/service_overview_throughput_chart.tsx b/x-pack/plugins/apm/public/components/app/service_overview/service_overview_throughput_chart.tsx index b79e011bde488a..fae00822bb966a 100644 --- a/x-pack/plugins/apm/public/components/app/service_overview/service_overview_throughput_chart.tsx +++ b/x-pack/plugins/apm/public/components/app/service_overview/service_overview_throughput_chart.tsx @@ -13,7 +13,6 @@ import { useFetcher } from '../../../hooks/use_fetcher'; import { useTheme } from '../../../hooks/use_theme'; import { useUrlParams } from '../../../context/url_params_context/use_url_params'; import { useApmServiceContext } from '../../../context/apm_service/use_apm_service_context'; -import { callApmApi } from '../../../services/rest/createCallApmApi'; import { TimeseriesChart } from '../../shared/charts/timeseries_chart'; export function ServiceOverviewThroughputChart({ @@ -27,24 +26,27 @@ export function ServiceOverviewThroughputChart({ const { transactionType } = useApmServiceContext(); const { start, end } = urlParams; - const { data, status } = useFetcher(() => { - if (serviceName && transactionType && start && end) { - return callApmApi({ - endpoint: 'GET /api/apm/services/{serviceName}/throughput', - params: { - path: { - serviceName, + const { data, status } = useFetcher( + (callApmApi) => { + if (serviceName && transactionType && start && end) { + return callApmApi({ + endpoint: 'GET /api/apm/services/{serviceName}/throughput', + params: { + path: { + serviceName, + }, + query: { + start, + end, + transactionType, + uiFilters: JSON.stringify(uiFilters), + }, }, - query: { - start, - end, - transactionType, - uiFilters: JSON.stringify(uiFilters), - }, - }, - }); - } - }, [serviceName, start, end, uiFilters, transactionType]); + }); + } + }, + [serviceName, start, end, uiFilters, transactionType] + ); return ( diff --git a/x-pack/plugins/apm/public/components/app/service_overview/service_overview_transactions_table/index.tsx b/x-pack/plugins/apm/public/components/app/service_overview/service_overview_transactions_table/index.tsx index c77e80d0176dee..069c4466d28a09 100644 --- a/x-pack/plugins/apm/public/components/app/service_overview/service_overview_transactions_table/index.tsx +++ b/x-pack/plugins/apm/public/components/app/service_overview/service_overview_transactions_table/index.tsx @@ -23,10 +23,7 @@ import { import { useApmServiceContext } from '../../../../context/apm_service/use_apm_service_context'; import { useUrlParams } from '../../../../context/url_params_context/use_url_params'; import { FETCH_STATUS, useFetcher } from '../../../../hooks/use_fetcher'; -import { - APIReturnType, - callApmApi, -} from '../../../../services/rest/createCallApmApi'; +import { APIReturnType } from '../../../../services/rest/createCallApmApi'; import { px, unit } from '../../../../style/variables'; import { SparkPlot } from '../../../shared/charts/spark_plot'; import { ImpactBar } from '../../../shared/ImpactBar'; @@ -110,53 +107,56 @@ export function ServiceOverviewTransactionsTable(props: Props) { }, }, status, - } = useFetcher(() => { - if (!start || !end || !latencyAggregationType || !transactionType) { - return; - } + } = useFetcher( + (callApmApi) => { + if (!start || !end || !latencyAggregationType || !transactionType) { + return; + } - return callApmApi({ - endpoint: - 'GET /api/apm/services/{serviceName}/transactions/groups/overview', - params: { - path: { serviceName }, - query: { - start, - end, - uiFilters: JSON.stringify(uiFilters), - size: PAGE_SIZE, - numBuckets: 20, - pageIndex: tableOptions.pageIndex, - sortField: tableOptions.sort.field, - sortDirection: tableOptions.sort.direction, - transactionType, - latencyAggregationType: latencyAggregationType as LatencyAggregationType, - }, - }, - }).then((response) => { - return { - items: response.transactionGroups, - totalItemCount: response.totalTransactionGroups, - tableOptions: { - pageIndex: tableOptions.pageIndex, - sort: { - field: tableOptions.sort.field, - direction: tableOptions.sort.direction, + return callApmApi({ + endpoint: + 'GET /api/apm/services/{serviceName}/transactions/groups/overview', + params: { + path: { serviceName }, + query: { + start, + end, + uiFilters: JSON.stringify(uiFilters), + size: PAGE_SIZE, + numBuckets: 20, + pageIndex: tableOptions.pageIndex, + sortField: tableOptions.sort.field, + sortDirection: tableOptions.sort.direction, + transactionType, + latencyAggregationType: latencyAggregationType as LatencyAggregationType, }, }, - }; - }); - }, [ - serviceName, - start, - end, - uiFilters, - tableOptions.pageIndex, - tableOptions.sort.field, - tableOptions.sort.direction, - transactionType, - latencyAggregationType, - ]); + }).then((response) => { + return { + items: response.transactionGroups, + totalItemCount: response.totalTransactionGroups, + tableOptions: { + pageIndex: tableOptions.pageIndex, + sort: { + field: tableOptions.sort.field, + direction: tableOptions.sort.direction, + }, + }, + }; + }); + }, + [ + serviceName, + start, + end, + uiFilters, + tableOptions.pageIndex, + tableOptions.sort.field, + tableOptions.sort.direction, + transactionType, + latencyAggregationType, + ] + ); const { items, diff --git a/x-pack/plugins/apm/public/components/shared/charts/transaction_error_rate_chart/index.tsx b/x-pack/plugins/apm/public/components/shared/charts/transaction_error_rate_chart/index.tsx index d712fa27c75ac4..a16edfee5fb3d3 100644 --- a/x-pack/plugins/apm/public/components/shared/charts/transaction_error_rate_chart/index.tsx +++ b/x-pack/plugins/apm/public/components/shared/charts/transaction_error_rate_chart/index.tsx @@ -12,7 +12,6 @@ import { asPercent } from '../../../../../common/utils/formatters'; import { useFetcher } from '../../../../hooks/use_fetcher'; import { useTheme } from '../../../../hooks/use_theme'; import { useUrlParams } from '../../../../context/url_params_context/use_url_params'; -import { callApmApi } from '../../../../services/rest/createCallApmApi'; import { TimeseriesChart } from '../timeseries_chart'; import { useApmServiceContext } from '../../../../context/apm_service/use_apm_service_context'; @@ -35,26 +34,29 @@ export function TransactionErrorRateChart({ const { transactionType } = useApmServiceContext(); const { start, end, transactionName } = urlParams; - const { data, status } = useFetcher(() => { - if (transactionType && serviceName && start && end) { - return callApmApi({ - endpoint: - 'GET /api/apm/services/{serviceName}/transactions/charts/error_rate', - params: { - path: { - serviceName, + const { data, status } = useFetcher( + (callApmApi) => { + if (transactionType && serviceName && start && end) { + return callApmApi({ + endpoint: + 'GET /api/apm/services/{serviceName}/transactions/charts/error_rate', + params: { + path: { + serviceName, + }, + query: { + start, + end, + transactionType, + transactionName, + uiFilters: JSON.stringify(uiFilters), + }, }, - query: { - start, - end, - transactionType, - transactionName, - uiFilters: JSON.stringify(uiFilters), - }, - }, - }); - } - }, [serviceName, start, end, uiFilters, transactionType, transactionName]); + }); + } + }, + [serviceName, start, end, uiFilters, transactionType, transactionName] + ); const errorRates = data?.transactionErrorRate || []; diff --git a/x-pack/plugins/apm/public/context/annotations/annotations_context.tsx b/x-pack/plugins/apm/public/context/annotations/annotations_context.tsx index 77285f976d8508..9d1b6d70f97382 100644 --- a/x-pack/plugins/apm/public/context/annotations/annotations_context.tsx +++ b/x-pack/plugins/apm/public/context/annotations/annotations_context.tsx @@ -9,7 +9,6 @@ import { useParams } from 'react-router-dom'; import { Annotation } from '../../../common/annotations'; import { useFetcher } from '../../hooks/use_fetcher'; import { useUrlParams } from '../url_params_context/use_url_params'; -import { callApmApi } from '../../services/rest/createCallApmApi'; export const AnnotationsContext = createContext({ annotations: [] } as { annotations: Annotation[]; @@ -27,23 +26,26 @@ export function AnnotationsContextProvider({ const { start, end } = urlParams; const { environment } = uiFilters; - const { data = INITIAL_STATE } = useFetcher(() => { - if (start && end && serviceName) { - return callApmApi({ - endpoint: 'GET /api/apm/services/{serviceName}/annotation/search', - params: { - path: { - serviceName, + const { data = INITIAL_STATE } = useFetcher( + (callApmApi) => { + if (start && end && serviceName) { + return callApmApi({ + endpoint: 'GET /api/apm/services/{serviceName}/annotation/search', + params: { + path: { + serviceName, + }, + query: { + start, + end, + environment, + }, }, - query: { - start, - end, - environment, - }, - }, - }); - } - }, [start, end, environment, serviceName]); + }); + } + }, + [start, end, environment, serviceName] + ); return ; } diff --git a/x-pack/plugins/apm/public/hooks/useLocalUIFilters.ts b/x-pack/plugins/apm/public/hooks/useLocalUIFilters.ts index dabdf41c63f048..fbdee617864df2 100644 --- a/x-pack/plugins/apm/public/hooks/useLocalUIFilters.ts +++ b/x-pack/plugins/apm/public/hooks/useLocalUIFilters.ts @@ -16,7 +16,6 @@ import { } from '../../server/lib/ui_filters/local_ui_filters/config'; import { fromQuery, toQuery } from '../components/shared/Links/url_helpers'; import { removeUndefinedProps } from '../context/url_params_context/helpers'; -import { useCallApi } from './useCallApi'; import { useFetcher } from './use_fetcher'; import { useUrlParams } from '../context/url_params_context/use_url_params'; import { LocalUIFilterName } from '../../common/ui_filter'; @@ -43,7 +42,6 @@ export function useLocalUIFilters({ }) { const history = useHistory(); const { uiFilters, urlParams } = useUrlParams(); - const callApi = useCallApi(); const values = pickKeys(uiFilters, ...filterNames); @@ -69,30 +67,34 @@ export function useLocalUIFilters({ }); }; - const { data = getInitialData(filterNames), status } = useFetcher(() => { - if (shouldFetch) { - return callApi({ - method: 'GET', - pathname: `/api/apm/ui_filters/local_filters/${projection}`, - query: { - uiFilters: JSON.stringify(uiFilters), - start: urlParams.start, - end: urlParams.end, - filterNames: JSON.stringify(filterNames), - ...params, - }, - }); - } - }, [ - callApi, - projection, - uiFilters, - urlParams.start, - urlParams.end, - filterNames, - params, - shouldFetch, - ]); + const { data = getInitialData(filterNames), status } = useFetcher( + (callApmApi) => { + if (shouldFetch && urlParams.start && urlParams.end) { + return callApmApi({ + endpoint: `GET /api/apm/ui_filters/local_filters/${projection}` as const, + params: { + query: { + uiFilters: JSON.stringify(uiFilters), + start: urlParams.start, + end: urlParams.end, + // type expects string constants, but we have to send it as json + filterNames: JSON.stringify(filterNames) as any, + ...params, + }, + }, + }); + } + }, + [ + projection, + uiFilters, + urlParams.start, + urlParams.end, + filterNames, + params, + shouldFetch, + ] + ); const filters = data.map((filter) => ({ ...filter, diff --git a/x-pack/plugins/apm/public/hooks/use_environments_fetcher.tsx b/x-pack/plugins/apm/public/hooks/use_environments_fetcher.tsx index 1ad151b8c7e907..38a8610b82acbe 100644 --- a/x-pack/plugins/apm/public/hooks/use_environments_fetcher.tsx +++ b/x-pack/plugins/apm/public/hooks/use_environments_fetcher.tsx @@ -10,7 +10,6 @@ import { ENVIRONMENT_ALL, ENVIRONMENT_NOT_DEFINED, } from '../../common/environment_filter_values'; -import { callApmApi } from '../services/rest/createCallApmApi'; function getEnvironmentOptions(environments: string[]) { const environmentOptions = environments @@ -32,20 +31,23 @@ export function useEnvironmentsFetcher({ start?: string; end?: string; }) { - const { data: environments = [], status = 'loading' } = useFetcher(() => { - if (start && end) { - return callApmApi({ - endpoint: 'GET /api/apm/ui_filters/environments', - params: { - query: { - start, - end, - serviceName, + const { data: environments = [], status = 'loading' } = useFetcher( + (callApmApi) => { + if (start && end) { + return callApmApi({ + endpoint: 'GET /api/apm/ui_filters/environments', + params: { + query: { + start, + end, + serviceName, + }, }, - }, - }); - } - }, [start, end, serviceName]); + }); + } + }, + [start, end, serviceName] + ); const environmentOptions = useMemo( () => getEnvironmentOptions(environments), diff --git a/x-pack/plugins/apm/public/hooks/use_fetcher.tsx b/x-pack/plugins/apm/public/hooks/use_fetcher.tsx index 2b58f30a9ec649..27427d80adc962 100644 --- a/x-pack/plugins/apm/public/hooks/use_fetcher.tsx +++ b/x-pack/plugins/apm/public/hooks/use_fetcher.tsx @@ -8,7 +8,10 @@ import { i18n } from '@kbn/i18n'; import React, { useEffect, useMemo, useState } from 'react'; import { IHttpFetchError } from 'src/core/public'; import { toMountPoint } from '../../../../../src/plugins/kibana_react/public'; -import { APMClient, callApmApi } from '../services/rest/createCallApmApi'; +import { + callApmApi, + AutoAbortedAPMClient, +} from '../services/rest/createCallApmApi'; import { useApmPluginContext } from '../context/apm_plugin/use_apm_plugin_context'; export enum FETCH_STATUS { @@ -39,6 +42,14 @@ function getDetailsFromErrorResponse(error: IHttpFetchError) { ); } +const createAutoAbortedAPMClient = ( + signal: AbortSignal +): AutoAbortedAPMClient => { + return ((options: Parameters[0]) => { + return callApmApi({ ...options, signal }); + }) as AutoAbortedAPMClient; +}; + // fetcher functions can return undefined OR a promise. Previously we had a more simple type // but it led to issues when using object destructuring with default values type InferResponseType = Exclude extends Promise< @@ -48,7 +59,7 @@ type InferResponseType = Exclude extends Promise< : unknown; export function useFetcher( - fn: (callApmApi: APMClient) => TReturn, + fn: (callApmApi: AutoAbortedAPMClient) => TReturn, fnDeps: any[], options: { preservePreviousData?: boolean; @@ -66,10 +77,16 @@ export function useFetcher( const [counter, setCounter] = useState(0); useEffect(() => { - let didCancel = false; + let controller: AbortController = new AbortController(); async function doFetch() { - const promise = fn(callApmApi); + controller.abort(); + + controller = new AbortController(); + + const signal = controller.signal; + + const promise = fn(createAutoAbortedAPMClient(signal)); // if `fn` doesn't return a promise it is a signal that data fetching was not initiated. // This can happen if the data fetching is conditional (based on certain inputs). // In these cases it is not desirable to invoke the global loading spinner, or change the status to success @@ -85,7 +102,11 @@ export function useFetcher( try { const data = await promise; - if (!didCancel) { + // when http fetches are aborted, the promise will be rejected + // and this code is never reached. For async operations that are + // not cancellable, we need to check whether the signal was + // aborted before updating the result. + if (!signal.aborted) { setResult({ data, status: FETCH_STATUS.SUCCESS, @@ -95,7 +116,7 @@ export function useFetcher( } catch (e) { const err = e as Error | IHttpFetchError; - if (!didCancel) { + if (!signal.aborted) { const errorDetails = 'response' in err ? getDetailsFromErrorResponse(err) : err.message; @@ -130,7 +151,7 @@ export function useFetcher( doFetch(); return () => { - didCancel = true; + controller.abort(); }; /* eslint-disable react-hooks/exhaustive-deps */ }, [ diff --git a/x-pack/plugins/apm/public/services/rest/apm_observability_overview_fetchers.ts b/x-pack/plugins/apm/public/services/rest/apm_observability_overview_fetchers.ts index a0ed51be685c72..ac98bbab5775ba 100644 --- a/x-pack/plugins/apm/public/services/rest/apm_observability_overview_fetchers.ts +++ b/x-pack/plugins/apm/public/services/rest/apm_observability_overview_fetchers.ts @@ -20,6 +20,7 @@ export const fetchObservabilityOverviewPageData = async ({ }: FetchDataParams): Promise => { const data = await callApmApi({ endpoint: 'GET /api/apm/observability_overview', + signal: null, params: { query: { start: new Date(absoluteTime.start).toISOString(), @@ -59,5 +60,6 @@ export const fetchObservabilityOverviewPageData = async ({ export async function hasData() { return await callApmApi({ endpoint: 'GET /api/apm/observability_overview/has_data', + signal: null, }); } diff --git a/x-pack/plugins/apm/public/services/rest/callApi.ts b/x-pack/plugins/apm/public/services/rest/callApi.ts index 4ee12908b7c792..d14cbc5f6d63ef 100644 --- a/x-pack/plugins/apm/public/services/rest/callApi.ts +++ b/x-pack/plugins/apm/public/services/rest/callApi.ts @@ -87,5 +87,6 @@ function isCachable(fetchOptions: FetchOptions) { // order the options object to make sure that two objects with the same arguments, produce produce the // same cache key regardless of the order of properties function getCacheKey(options: FetchOptions) { - return hash(options); + const { pathname, method, body, query, headers } = options; + return hash({ pathname, method, body, query, headers }); } diff --git a/x-pack/plugins/apm/public/services/rest/createCallApmApi.ts b/x-pack/plugins/apm/public/services/rest/createCallApmApi.ts index 2760ed558865ac..b77233982ffc5d 100644 --- a/x-pack/plugins/apm/public/services/rest/createCallApmApi.ts +++ b/x-pack/plugins/apm/public/services/rest/createCallApmApi.ts @@ -12,11 +12,14 @@ import { APMAPI } from '../../../server/routes/create_apm_api'; import { Client } from '../../../server/routes/typings'; export type APMClient = Client; +export type AutoAbortedAPMClient = Client; + export type APMClientOptions = Omit< FetchOptions, - 'query' | 'body' | 'pathname' + 'query' | 'body' | 'pathname' | 'signal' > & { endpoint: string; + signal: AbortSignal | null; params?: { body?: any; query?: any; diff --git a/x-pack/plugins/apm/public/services/rest/index_pattern.ts b/x-pack/plugins/apm/public/services/rest/index_pattern.ts index 6ec542ab6baf31..cea3bcc0b68cc3 100644 --- a/x-pack/plugins/apm/public/services/rest/index_pattern.ts +++ b/x-pack/plugins/apm/public/services/rest/index_pattern.ts @@ -9,11 +9,13 @@ import { callApmApi } from './createCallApmApi'; export const createStaticIndexPattern = async () => { return await callApmApi({ endpoint: 'POST /api/apm/index_pattern/static', + signal: null, }); }; export const getApmIndexPatternTitle = async () => { return await callApmApi({ endpoint: 'GET /api/apm/index_pattern/title', + signal: null, }); }; diff --git a/x-pack/plugins/apm/server/lib/helpers/create_es_client/create_apm_event_client/index.test.ts b/x-pack/plugins/apm/server/lib/helpers/create_es_client/create_apm_event_client/index.test.ts index f58e04061254d5..87cb60a5431932 100644 --- a/x-pack/plugins/apm/server/lib/helpers/create_es_client/create_apm_event_client/index.test.ts +++ b/x-pack/plugins/apm/server/lib/helpers/create_es_client/create_apm_event_client/index.test.ts @@ -65,10 +65,12 @@ describe('createApmEventClient', () => { await new Promise((resolve) => { setTimeout(() => { + incomingRequest.on('abort', () => { + setTimeout(() => { + resolve(undefined); + }, 0); + }); incomingRequest.abort(); - setTimeout(() => { - resolve(undefined); - }, 0); }, 50); }); diff --git a/x-pack/plugins/apm/server/lib/helpers/setup_request.test.ts b/x-pack/plugins/apm/server/lib/helpers/setup_request.test.ts index f00941d6e68007..47a13185ff90fc 100644 --- a/x-pack/plugins/apm/server/lib/helpers/setup_request.test.ts +++ b/x-pack/plugins/apm/server/lib/helpers/setup_request.test.ts @@ -92,7 +92,7 @@ function getMockRequest() { url: '', events: { aborted$: { - subscribe: jest.fn(), + subscribe: jest.fn().mockReturnValue({ unsubscribe: jest.fn() }), }, }, } as unknown) as KibanaRequest; diff --git a/x-pack/plugins/apm/server/lib/services/get_service_dependencies/index.ts b/x-pack/plugins/apm/server/lib/services/get_service_dependencies/index.ts index 2b209f8f6a80a9..8d6b9bfc1a4e64 100644 --- a/x-pack/plugins/apm/server/lib/services/get_service_dependencies/index.ts +++ b/x-pack/plugins/apm/server/lib/services/get_service_dependencies/index.ts @@ -196,19 +196,26 @@ export async function getServiceDependencies({ }); const latencySums = metricsByResolvedAddress - .map((metrics) => metrics.latency.value) + .map( + (metric) => (metric.latency.value ?? 0) * (metric.throughput.value ?? 0) + ) .filter(isFiniteNumber); const minLatencySum = Math.min(...latencySums); const maxLatencySum = Math.max(...latencySums); - return metricsByResolvedAddress.map((metric) => ({ - ...metric, - impact: - metric.latency.value === null - ? 0 - : ((metric.latency.value - minLatencySum) / + return metricsByResolvedAddress.map((metric) => { + const impact = + isFiniteNumber(metric.latency.value) && + isFiniteNumber(metric.throughput.value) + ? ((metric.latency.value * metric.throughput.value - minLatencySum) / (maxLatencySum - minLatencySum)) * - 100, - })); + 100 + : 0; + + return { + ...metric, + impact, + }; + }); } diff --git a/x-pack/plugins/apm/server/routes/create_api/index.ts b/x-pack/plugins/apm/server/routes/create_api/index.ts index 721badf7fc025b..6f6ec4f06b6cbe 100644 --- a/x-pack/plugins/apm/server/routes/create_api/index.ts +++ b/x-pack/plugins/apm/server/routes/create_api/index.ts @@ -10,6 +10,7 @@ import * as t from 'io-ts'; import { PathReporter } from 'io-ts/lib/PathReporter'; import { isLeft } from 'fp-ts/lib/Either'; import { KibanaResponseFactory, RouteRegistrar } from 'src/core/server'; +import { RequestAbortedError } from '@elastic/elasticsearch/lib/errors'; import { merge } from '../../../common/runtime_types/merge'; import { strictKeysRt } from '../../../common/runtime_types/strict_keys_rt'; import { APMConfig } from '../..'; @@ -132,6 +133,15 @@ export function createApi() { if (Boom.isBoom(error)) { return convertBoomToKibanaResponse(error, response); } + + if (error instanceof RequestAbortedError) { + return response.custom({ + statusCode: 499, + body: { + message: 'Client closed request', + }, + }); + } throw error; } } diff --git a/x-pack/plugins/apm/server/routes/typings.ts b/x-pack/plugins/apm/server/routes/typings.ts index 7d7a5c3b0dab32..4cc3c747b201fc 100644 --- a/x-pack/plugins/apm/server/routes/typings.ts +++ b/x-pack/plugins/apm/server/routes/typings.ts @@ -131,15 +131,20 @@ type MaybeOptional }> = RequiredKeys< ? { params?: T['params'] } : { params: T['params'] }; -export type Client = < - TEndpoint extends keyof TRouteState & string ->( - options: Omit & { +export type Client< + TRouteState, + TOptions extends { abortable: boolean } = { abortable: true } +> = ( + options: Omit< + FetchOptions, + 'query' | 'body' | 'pathname' | 'method' | 'signal' + > & { forceCache?: boolean; endpoint: TEndpoint; } & (TRouteState[TEndpoint] extends { params: t.Any } ? MaybeOptional<{ params: t.TypeOf }> - : {}) + : {}) & + (TOptions extends { abortable: true } ? { signal: AbortSignal | null } : {}) ) => Promise< TRouteState[TEndpoint] extends { ret: any } ? TRouteState[TEndpoint]['ret'] diff --git a/x-pack/plugins/case/common/api/connectors/mappings.ts b/x-pack/plugins/case/common/api/connectors/mappings.ts index f8e9830fed7c1b..b9f84d406a184a 100644 --- a/x-pack/plugins/case/common/api/connectors/mappings.ts +++ b/x-pack/plugins/case/common/api/connectors/mappings.ts @@ -16,8 +16,8 @@ import { Incident as ResilientIncident, } from '../../../../actions/server/builtin_action_types/resilient/types'; import { - PushToServiceApiParams as ServiceNowPushToServiceApiParams, - Incident as ServiceNowIncident, + PushToServiceApiParamsITSM as ServiceNowITSMPushToServiceApiParams, + ServiceNowITSMIncident, } from '../../../../actions/server/builtin_action_types/servicenow/types'; import { ResilientFieldsRT } from './resilient'; import { ServiceNowFieldsRT } from './servicenow'; @@ -33,13 +33,13 @@ export interface ElasticUser { export { JiraPushToServiceApiParams, ResilientPushToServiceApiParams, - ServiceNowPushToServiceApiParams, + ServiceNowITSMPushToServiceApiParams, }; -export type Incident = JiraIncident | ResilientIncident | ServiceNowIncident; +export type Incident = JiraIncident | ResilientIncident | ServiceNowITSMIncident; export type PushToServiceApiParams = | JiraPushToServiceApiParams | ResilientPushToServiceApiParams - | ServiceNowPushToServiceApiParams; + | ServiceNowITSMPushToServiceApiParams; const ActionTypeRT = rt.union([ rt.literal('append'), diff --git a/x-pack/plugins/case/server/routes/api/cases/configure/utils.ts b/x-pack/plugins/case/server/routes/api/cases/configure/utils.ts index 89109af4cecb9d..9e903b66459a99 100644 --- a/x-pack/plugins/case/server/routes/api/cases/configure/utils.ts +++ b/x-pack/plugins/case/server/routes/api/cases/configure/utils.ts @@ -19,7 +19,7 @@ import { PrepareFieldsForTransformArgs, PushToServiceApiParams, ResilientPushToServiceApiParams, - ServiceNowPushToServiceApiParams, + ServiceNowITSMPushToServiceApiParams, SimpleComment, Transformer, TransformerArgs, @@ -105,7 +105,11 @@ export const serviceFormatter = ( thirdPartyName: 'Resilient', }; case ConnectorTypes.servicenow: - const { severity, urgency, impact } = params as ServiceNowPushToServiceApiParams['incident']; + const { + severity, + urgency, + impact, + } = params as ServiceNowITSMPushToServiceApiParams['incident']; return { incident: { severity, urgency, impact }, thirdPartyName: 'ServiceNow', diff --git a/x-pack/plugins/code/tsconfig.json b/x-pack/plugins/code/tsconfig.json new file mode 100644 index 00000000000000..9c0b0ed21330f0 --- /dev/null +++ b/x-pack/plugins/code/tsconfig.json @@ -0,0 +1,16 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "composite": true, + "outDir": "./target/types", + "emitDeclarationOnly": true, + "declaration": true, + "declarationMap": true + }, + "include": [ + "server/**/*" + ], + "references": [ + { "path": "../../../src/core/tsconfig.json" }, + ] +} diff --git a/x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts b/x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts index 694d9807b5a4d3..dc1fa13d32e27e 100644 --- a/x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts +++ b/x-pack/plugins/data_enhanced/server/search/es_search_strategy.ts @@ -23,6 +23,7 @@ import { getTotalLoaded, searchUsageObserver, shimAbortSignal, + shimHitsTotal, } from '../../../../../src/plugins/data/server'; import type { IAsyncSearchOptions } from '../../common'; import { pollSearch } from '../../common'; @@ -63,7 +64,8 @@ export const enhancedEsSearchStrategyProvider = ( ? client.get({ ...params, id }) : client.submit(params); const { body } = await shimAbortSignal(promise, options.abortSignal); - return toAsyncKibanaSearchResponse(body); + const response = shimHitsTotal(body.response, options); + return toAsyncKibanaSearchResponse({ ...body, response }); }; const cancel = async () => { @@ -108,7 +110,7 @@ export const enhancedEsSearchStrategyProvider = ( const esResponse = await shimAbortSignal(promise, options?.abortSignal); const response = esResponse.body as SearchResponse; return { - rawResponse: response, + rawResponse: shimHitsTotal(response, options), ...getTotalLoaded(response), }; } catch (e) { diff --git a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/constants.ts b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/constants.ts index 48b8a06b2549ca..5e106a7f42f577 100644 --- a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/constants.ts +++ b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/constants.ts @@ -311,3 +311,157 @@ export const SOURCE_NAME_LABEL = i18n.translate( defaultMessage: 'Source name', } ); + +export const ORG_SOURCES_LINK = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.org.link', + { + defaultMessage: 'Add an organization content source', + } +); + +export const ORG_SOURCES_HEADER_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.org.title', + { + defaultMessage: 'Organization sources', + } +); + +export const ORG_SOURCES_HEADER_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.org.description', + { + defaultMessage: + 'Organization sources are available to the entire organization and can be assigned to specific user groups.', + } +); + +export const PRIVATE_LINK_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.link', + { + defaultMessage: 'Add a private content source', + } +); + +export const PRIVATE_CAN_CREATE_PAGE_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.canCreate.title', + { + defaultMessage: 'Manage private content sources', + } +); + +export const PRIVATE_VIEW_ONLY_PAGE_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.vewOnly.title', + { + defaultMessage: 'Review Group Sources', + } +); + +export const PRIVATE_VIEW_ONLY_PAGE_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.vewOnly.description', + { + defaultMessage: 'Review the status of all sources shared with your Group.', + } +); + +export const PRIVATE_CAN_CREATE_PAGE_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.canCreate.description', + { + defaultMessage: + 'Review the status of all connected private sources, and manage private sources for your account.', + } +); + +export const PRIVATE_HEADER_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.header.title', + { + defaultMessage: 'My private content sources', + } +); + +export const PRIVATE_HEADER_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.header.description', + { + defaultMessage: 'Private content sources are available only to you.', + } +); + +export const PRIVATE_SHARED_SOURCES_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.privateShared.header.title', + { + defaultMessage: 'Shared content sources', + } +); + +export const PRIVATE_EMPTY_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.private.empty.title', + { + defaultMessage: 'You have no private sources', + } +); +export const SHARED_EMPTY_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.shared.empty.title', + { + defaultMessage: 'No content source available', + } +); + +export const SHARED_EMPTY_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.shared.empty.description', + { + defaultMessage: + 'Once content sources are shared with you, they will be displayed here, and available via the search experience.', + } +); + +export const AND = i18n.translate('xpack.enterpriseSearch.workplaceSearch.and', { + defaultMessage: 'and', +}); + +export const LICENSE_CALLOUT_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.licenseCallout.title', + { + defaultMessage: 'Private Sources are no longer available', + } +); + +export const LICENSE_CALLOUT_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.licenseCallout.description', + { + defaultMessage: 'Contact your search experience administrator for more information.', + } +); + +export const SOURCE_DISABLED_CALLOUT_TITLE = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.sourceDisabled.title', + { + defaultMessage: 'Content source is disabled', + } +); + +export const SOURCE_DISABLED_CALLOUT_DESCRIPTION = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.sourceDisabled.description', + { + defaultMessage: + 'Your organization’s license level has changed. Your data is safe, but document-level permissions are no longer supported and searching of this source has been disabled. Upgrade to a Platinum license to re-enable this source.', + } +); + +export const SOURCE_DISABLED_CALLOUT_BUTTON = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.sourceDisabled.button', + { + defaultMessage: 'Explore Platinum license', + } +); + +export const DOCUMENT_PERMISSIONS_LINK = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.documentPermissionsLink', + { + defaultMessage: 'Learn more about document-level permission configuration', + } +); + +export const UNDERSTAND_BUTTON = i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sources.understandButton', + { + defaultMessage: 'I understand', + } +); diff --git a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/organization_sources.tsx b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/organization_sources.tsx index fdb536dd797714..3081301fe0a9f1 100644 --- a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/organization_sources.tsx +++ b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/organization_sources.tsx @@ -12,6 +12,12 @@ import { Link, Redirect } from 'react-router-dom'; import { EuiButton } from '@elastic/eui'; import { ADD_SOURCE_PATH, getSourcesPath } from '../../routes'; +import { + ORG_SOURCES_LINK, + ORG_SOURCES_HEADER_TITLE, + ORG_SOURCES_HEADER_DESCRIPTION, +} from './constants'; + import { Loading } from '../../../shared/loading'; import { ContentSection } from '../../components/shared/content_section'; import { SourcesTable } from '../../components/shared/sources_table'; @@ -21,11 +27,6 @@ import { SourcesLogic } from './sources_logic'; import { SourcesView } from './sources_view'; -const ORG_LINK_TITLE = 'Add an organization content source'; -const ORG_HEADER_TITLE = 'Organization sources'; -const ORG_HEADER_DESCRIPTION = - 'Organization sources are available to the entire organization and can be assigned to specific user groups.'; - export const OrganizationSources: React.FC = () => { const { initializeSources, setSourceSearchability, resetSourcesState } = useActions(SourcesLogic); @@ -40,28 +41,22 @@ export const OrganizationSources: React.FC = () => { if (contentSources.length === 0) return ; - const linkTitle = ORG_LINK_TITLE; - const headerTitle = ORG_HEADER_TITLE; - const headerDescription = ORG_HEADER_DESCRIPTION; - const sectionTitle = ''; - const sectionDescription = ''; - return ( - {linkTitle} + {ORG_SOURCES_LINK} } - description={headerDescription} + description={ORG_SOURCES_HEADER_DESCRIPTION} alignItems="flexStart" /> - + { const { hasPlatinumLicense } = useValues(LicensingLogic); const { initializeSources, setSourceSearchability, resetSourcesState } = useActions(SourcesLogic); @@ -112,7 +119,7 @@ export const PrivateSources: React.FC = () => { - You have no private sources} /> + {PRIVATE_EMPTY_TITLE}} /> @@ -124,13 +131,8 @@ export const PrivateSources: React.FC = () => { No content source available} - body={ -

- Once content sources are shared with you, they will be displayed here, and available - via the search experience. -

- } + title={

{SHARED_EMPTY_TITLE}

} + body={

{SHARED_EMPTY_DESCRIPTION}

} /> @@ -140,16 +142,21 @@ export const PrivateSources: React.FC = () => { const hasPrivateSources = privateContentSources?.length > 0; const privateSources = hasPrivateSources ? privateSourcesTable : privateSourcesEmptyState; - const groupsSentence = `${groups.slice(0, groups.length - 1).join(', ')}, and ${groups.slice( + const groupsSentence = `${groups.slice(0, groups.length - 1).join(', ')}, ${AND} ${groups.slice( -1 )}`; const sharedSources = ( @@ -157,8 +164,8 @@ export const PrivateSources: React.FC = () => { const licenseCallout = ( <> - -

Contact your search experience administrator for more information.

+ +

{LICENSE_CALLOUT_DESCRIPTION}

 diff --git a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/source_router.tsx b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/source_router.tsx index f46743778a1683..67995a4920925d 100644 --- a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/source_router.tsx +++ b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/source_router.tsx @@ -17,6 +17,12 @@ import { SendWorkplaceSearchTelemetry as SendTelemetry } from '../../../shared/t import { NAV } from '../../constants'; +import { + SOURCE_DISABLED_CALLOUT_TITLE, + SOURCE_DISABLED_CALLOUT_DESCRIPTION, + SOURCE_DISABLED_CALLOUT_BUTTON, +} from './constants'; + import { ENT_SEARCH_LICENSE_MANAGEMENT, REINDEX_JOB_PATH, @@ -80,14 +86,10 @@ export const SourceRouter: React.FC = () => { const callout = ( <> - -

- Your organization’s license level has changed. Your data is safe, but document-level - permissions are no longer supported and searching of this source has been disabled. - Upgrade to a Platinum license to re-enable this source. -

+ +

{SOURCE_DISABLED_CALLOUT_DESCRIPTION}

- Explore Platinum license + {SOURCE_DISABLED_CALLOUT_BUTTON} diff --git a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/sources_view.tsx b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/sources_view.tsx index 9e6c8f5b7319ee..f8a2d345c85130 100644 --- a/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/sources_view.tsx +++ b/x-pack/plugins/enterprise_search/public/applications/workplace_search/views/content_sources/sources_view.tsx @@ -8,6 +8,9 @@ import React from 'react'; import { useActions, useValues } from 'kea'; +import { i18n } from '@kbn/i18n'; +import { FormattedMessage } from '@kbn/i18n/react'; + import { EuiButton, EuiLink, @@ -27,6 +30,12 @@ import { SourceIcon } from '../../components/shared/source_icon'; import { EXTERNAL_IDENTITIES_DOCS_URL, DOCUMENT_PERMISSIONS_DOCS_URL } from '../../routes'; +import { + EXTERNAL_IDENTITIES_LINK, + DOCUMENT_PERMISSIONS_LINK, + UNDERSTAND_BUTTON, +} from './constants'; + import { SourcesLogic } from './sources_logic'; interface SourcesViewProps { @@ -59,35 +68,53 @@ export const SourcesView: React.FC = ({ children }) => { - {addedSourceName} requires additional configuration + + {i18n.translate( + 'xpack.enterpriseSearch.workplaceSearch.sourcesView.modal.heading', + { + defaultMessage: '{addedSourceName} requires additional configuration', + values: { addedSourceName }, + } + )} +

- {addedSourceName} has been successfully connected and initial content synchronization - is already underway. Since you have elected to synchronize document-level permission - information, you must now provide user and group mappings using the  - - External Identities API - - . + + {EXTERNAL_IDENTITIES_LINK} + + ), + }} + />

- Documents will not be searchable from Workplace Search until user and group mappings - have been configured.  - - Learn more about document-level permission configuration - - . + + {DOCUMENT_PERMISSIONS_LINK} + + ), + }} + />

- I understand + {UNDERSTAND_BUTTON} diff --git a/x-pack/plugins/grokdebugger/tsconfig.json b/x-pack/plugins/grokdebugger/tsconfig.json new file mode 100644 index 00000000000000..34cf8d74c0024b --- /dev/null +++ b/x-pack/plugins/grokdebugger/tsconfig.json @@ -0,0 +1,24 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "composite": true, + "outDir": "./target/types", + "emitDeclarationOnly": true, + "declaration": true, + "declarationMap": true + }, + "include": [ + "common/**/*", + "public/**/*", + "server/**/*", + "../../typings/**/*", + ], + "references": [ + { "path": "../../../src/core/tsconfig.json" }, + { "path": "../../../src/plugins/dev_tools/tsconfig.json"}, + { "path": "../../../src/plugins/home/tsconfig.json"}, + { "path": "../../../src/plugins/kibana_react/tsconfig.json" }, + { "path": "../../../src/plugins/es_ui_shared/tsconfig.json" }, + { "path": "../licensing/tsconfig.json" } + ] +} diff --git a/x-pack/plugins/infra/common/graphql/types.ts b/x-pack/plugins/infra/common/graphql/types.ts index 4a18c3d5ff3340..ee536feb1ce653 100644 --- a/x-pack/plugins/infra/common/graphql/types.ts +++ b/x-pack/plugins/infra/common/graphql/types.ts @@ -28,12 +28,6 @@ export interface InfraSource { configuration: InfraSourceConfiguration; /** The status of the source */ status: InfraSourceStatus; - /** A consecutive span of log entries surrounding a point in time */ - logEntriesAround: InfraLogEntryInterval; - /** A consecutive span of log entries within an interval */ - logEntriesBetween: InfraLogEntryInterval; - /** Sequences of log entries matching sets of highlighting queries within an interval */ - logEntryHighlights: InfraLogEntryInterval[]; /** A snapshot of nodes */ snapshot?: InfraSnapshotResponse | null; @@ -129,80 +123,6 @@ export interface InfraIndexField { /** Whether the field should be displayed based on event.module and a ECS allowed list */ displayable: boolean; } -/** A consecutive sequence of log entries */ -export interface InfraLogEntryInterval { - /** The key corresponding to the start of the interval covered by the entries */ - start?: InfraTimeKey | null; - /** The key corresponding to the end of the interval covered by the entries */ - end?: InfraTimeKey | null; - /** Whether there are more log entries available before the start */ - hasMoreBefore: boolean; - /** Whether there are more log entries available after the end */ - hasMoreAfter: boolean; - /** The query the log entries were filtered by */ - filterQuery?: string | null; - /** The query the log entries were highlighted with */ - highlightQuery?: string | null; - /** A list of the log entries */ - entries: InfraLogEntry[]; -} -/** A representation of the log entry's position in the event stream */ -export interface InfraTimeKey { - /** The timestamp of the event that the log entry corresponds to */ - time: number; - /** The tiebreaker that disambiguates events with the same timestamp */ - tiebreaker: number; -} -/** A log entry */ -export interface InfraLogEntry { - /** A unique representation of the log entry's position in the event stream */ - key: InfraTimeKey; - /** The log entry's id */ - gid: string; - /** The source id */ - source: string; - /** The columns used for rendering the log entry */ - columns: InfraLogEntryColumn[]; -} -/** A special built-in column that contains the log entry's timestamp */ -export interface InfraLogEntryTimestampColumn { - /** The id of the corresponding column configuration */ - columnId: string; - /** The timestamp */ - timestamp: number; -} -/** A special built-in column that contains the log entry's constructed message */ -export interface InfraLogEntryMessageColumn { - /** The id of the corresponding column configuration */ - columnId: string; - /** A list of the formatted log entry segments */ - message: InfraLogMessageSegment[]; -} -/** A segment of the log entry message that was derived from a field */ -export interface InfraLogMessageFieldSegment { - /** The field the segment was derived from */ - field: string; - /** The segment's message */ - value: string; - /** A list of highlighted substrings of the value */ - highlights: string[]; -} -/** A segment of the log entry message that was derived from a string literal */ -export interface InfraLogMessageConstantSegment { - /** The segment's message */ - constant: string; -} -/** A column that contains the value of a field of the log entry */ -export interface InfraLogEntryFieldColumn { - /** The id of the corresponding column configuration */ - columnId: string; - /** The field name of the column */ - field: string; - /** The value of the field in the log entry */ - value: string; - /** A list of highlighted substrings of the value */ - highlights: string[]; -} export interface InfraSnapshotResponse { /** Nodes of type host, container or pod grouped by 0, 1 or 2 terms */ @@ -276,21 +196,6 @@ export interface DeleteSourceResult { // InputTypes // ==================================================== -export interface InfraTimeKeyInput { - time: number; - - tiebreaker: number; -} -/** A highlighting definition */ -export interface InfraLogEntryHighlightInput { - /** The query to highlight by */ - query: string; - /** The number of highlighted documents to include beyond the beginning of the interval */ - countBefore: number; - /** The number of highlighted documents to include beyond the end of the interval */ - countAfter: number; -} - export interface InfraTimerangeInput { /** The interval string to use for last bucket. The format is '{value}{unit}'. For example '5m' would return the metrics for the last 5 minutes of the timespan. */ interval: string; @@ -381,34 +286,6 @@ export interface SourceQueryArgs { /** The id of the source */ id: string; } -export interface LogEntriesAroundInfraSourceArgs { - /** The sort key that corresponds to the point in time */ - key: InfraTimeKeyInput; - /** The maximum number of preceding to return */ - countBefore?: number | null; - /** The maximum number of following to return */ - countAfter?: number | null; - /** The query to filter the log entries by */ - filterQuery?: string | null; -} -export interface LogEntriesBetweenInfraSourceArgs { - /** The sort key that corresponds to the start of the interval */ - startKey: InfraTimeKeyInput; - /** The sort key that corresponds to the end of the interval */ - endKey: InfraTimeKeyInput; - /** The query to filter the log entries by */ - filterQuery?: string | null; -} -export interface LogEntryHighlightsInfraSourceArgs { - /** The sort key that corresponds to the start of the interval */ - startKey: InfraTimeKeyInput; - /** The sort key that corresponds to the end of the interval */ - endKey: InfraTimeKeyInput; - /** The query to filter the log entries by */ - filterQuery?: string | null; - /** The highlighting to apply to the log entries */ - highlights: InfraLogEntryHighlightInput[]; -} export interface SnapshotInfraSourceArgs { timerange: InfraTimerangeInput; @@ -565,15 +442,6 @@ export type InfraSourceLogColumn = | InfraSourceMessageLogColumn | InfraSourceFieldLogColumn; -/** A column of a log entry */ -export type InfraLogEntryColumn = - | InfraLogEntryTimestampColumn - | InfraLogEntryMessageColumn - | InfraLogEntryFieldColumn; - -/** A segment of the log entry message */ -export type InfraLogMessageSegment = InfraLogMessageFieldSegment | InfraLogMessageConstantSegment; - // ==================================================== // END: Typescript template // ==================================================== @@ -582,46 +450,6 @@ export type InfraLogMessageSegment = InfraLogMessageFieldSegment | InfraLogMessa // Documents // ==================================================== -export namespace LogEntryHighlightsQuery { - export type Variables = { - sourceId?: string | null; - startKey: InfraTimeKeyInput; - endKey: InfraTimeKeyInput; - filterQuery?: string | null; - highlights: InfraLogEntryHighlightInput[]; - }; - - export type Query = { - __typename?: 'Query'; - - source: Source; - }; - - export type Source = { - __typename?: 'InfraSource'; - - id: string; - - logEntryHighlights: LogEntryHighlights[]; - }; - - export type LogEntryHighlights = { - __typename?: 'InfraLogEntryInterval'; - - start?: Start | null; - - end?: End | null; - - entries: Entries[]; - }; - - export type Start = InfraTimeKeyFields.Fragment; - - export type End = InfraTimeKeyFields.Fragment; - - export type Entries = InfraLogEntryHighlightFields.Fragment; -} - export namespace MetricsQuery { export type Variables = { sourceId: string; @@ -820,50 +648,6 @@ export namespace WaffleNodesQuery { }; } -export namespace LogEntries { - export type Variables = { - sourceId?: string | null; - timeKey: InfraTimeKeyInput; - countBefore?: number | null; - countAfter?: number | null; - filterQuery?: string | null; - }; - - export type Query = { - __typename?: 'Query'; - - source: Source; - }; - - export type Source = { - __typename?: 'InfraSource'; - - id: string; - - logEntriesAround: LogEntriesAround; - }; - - export type LogEntriesAround = { - __typename?: 'InfraLogEntryInterval'; - - start?: Start | null; - - end?: End | null; - - hasMoreBefore: boolean; - - hasMoreAfter: boolean; - - entries: Entries[]; - }; - - export type Start = InfraTimeKeyFields.Fragment; - - export type End = InfraTimeKeyFields.Fragment; - - export type Entries = InfraLogEntryFields.Fragment; -} - export namespace SourceConfigurationFields { export type Fragment = { __typename?: 'InfraSourceConfiguration'; @@ -994,124 +778,3 @@ export namespace InfraSourceFields { origin: string; }; } - -export namespace InfraLogEntryFields { - export type Fragment = { - __typename?: 'InfraLogEntry'; - - gid: string; - - key: Key; - - columns: Columns[]; - }; - - export type Key = { - __typename?: 'InfraTimeKey'; - - time: number; - - tiebreaker: number; - }; - - export type Columns = - | InfraLogEntryTimestampColumnInlineFragment - | InfraLogEntryMessageColumnInlineFragment - | InfraLogEntryFieldColumnInlineFragment; - - export type InfraLogEntryTimestampColumnInlineFragment = { - __typename?: 'InfraLogEntryTimestampColumn'; - - columnId: string; - - timestamp: number; - }; - - export type InfraLogEntryMessageColumnInlineFragment = { - __typename?: 'InfraLogEntryMessageColumn'; - - columnId: string; - - message: Message[]; - }; - - export type Message = - | InfraLogMessageFieldSegmentInlineFragment - | InfraLogMessageConstantSegmentInlineFragment; - - export type InfraLogMessageFieldSegmentInlineFragment = { - __typename?: 'InfraLogMessageFieldSegment'; - - field: string; - - value: string; - }; - - export type InfraLogMessageConstantSegmentInlineFragment = { - __typename?: 'InfraLogMessageConstantSegment'; - - constant: string; - }; - - export type InfraLogEntryFieldColumnInlineFragment = { - __typename?: 'InfraLogEntryFieldColumn'; - - columnId: string; - - field: string; - - value: string; - }; -} - -export namespace InfraLogEntryHighlightFields { - export type Fragment = { - __typename?: 'InfraLogEntry'; - - gid: string; - - key: Key; - - columns: Columns[]; - }; - - export type Key = { - __typename?: 'InfraTimeKey'; - - time: number; - - tiebreaker: number; - }; - - export type Columns = - | InfraLogEntryMessageColumnInlineFragment - | InfraLogEntryFieldColumnInlineFragment; - - export type InfraLogEntryMessageColumnInlineFragment = { - __typename?: 'InfraLogEntryMessageColumn'; - - columnId: string; - - message: Message[]; - }; - - export type Message = InfraLogMessageFieldSegmentInlineFragment; - - export type InfraLogMessageFieldSegmentInlineFragment = { - __typename?: 'InfraLogMessageFieldSegment'; - - field: string; - - highlights: string[]; - }; - - export type InfraLogEntryFieldColumnInlineFragment = { - __typename?: 'InfraLogEntryFieldColumn'; - - columnId: string; - - field: string; - - highlights: string[]; - }; -} diff --git a/x-pack/plugins/infra/common/http_api/log_sources/log_source_configuration.ts b/x-pack/plugins/infra/common/http_api/log_sources/log_source_configuration.ts index 7581e29692356f..df7d80d33f1e60 100644 --- a/x-pack/plugins/infra/common/http_api/log_sources/log_source_configuration.ts +++ b/x-pack/plugins/infra/common/http_api/log_sources/log_source_configuration.ts @@ -53,6 +53,7 @@ export const logSourceColumnConfigurationRT = rt.union([ logSourceMessageColumnConfigurationRT, logSourceFieldColumnConfigurationRT, ]); +export type LogSourceColumnConfiguration = rt.TypeOf; export const logSourceConfigurationPropertiesRT = rt.strict({ name: rt.string, diff --git a/x-pack/plugins/infra/common/log_entry/log_entry.ts b/x-pack/plugins/infra/common/log_entry/log_entry.ts index eec1fb59f3091c..bf3f9ceb0b0887 100644 --- a/x-pack/plugins/infra/common/log_entry/log_entry.ts +++ b/x-pack/plugins/infra/common/log_entry/log_entry.ts @@ -6,87 +6,79 @@ import * as rt from 'io-ts'; import { TimeKey } from '../time'; -import { logEntryCursorRT } from './log_entry_cursor'; import { jsonArrayRT } from '../typed_json'; - -export interface LogEntryOrigin { - id: string; - index: string; - type: string; -} +import { logEntryCursorRT } from './log_entry_cursor'; export type LogEntryTime = TimeKey; -export interface LogEntryFieldsMapping { - message: string; - tiebreaker: string; - time: string; -} - -export function isEqual(time1: LogEntryTime, time2: LogEntryTime) { - return time1.time === time2.time && time1.tiebreaker === time2.tiebreaker; -} - -export function isLess(time1: LogEntryTime, time2: LogEntryTime) { - return ( - time1.time < time2.time || (time1.time === time2.time && time1.tiebreaker < time2.tiebreaker) - ); -} - -export function isLessOrEqual(time1: LogEntryTime, time2: LogEntryTime) { - return ( - time1.time < time2.time || (time1.time === time2.time && time1.tiebreaker <= time2.tiebreaker) - ); -} - -export function isBetween(min: LogEntryTime, max: LogEntryTime, operand: LogEntryTime) { - return isLessOrEqual(min, operand) && isLessOrEqual(operand, max); -} +/** + * message parts + */ export const logMessageConstantPartRT = rt.type({ constant: rt.string, }); +export type LogMessageConstantPart = rt.TypeOf; + export const logMessageFieldPartRT = rt.type({ field: rt.string, value: jsonArrayRT, highlights: rt.array(rt.string), }); +export type LogMessageFieldPart = rt.TypeOf; export const logMessagePartRT = rt.union([logMessageConstantPartRT, logMessageFieldPartRT]); +export type LogMessagePart = rt.TypeOf; + +/** + * columns + */ export const logTimestampColumnRT = rt.type({ columnId: rt.string, timestamp: rt.number }); +export type LogTimestampColumn = rt.TypeOf; + export const logFieldColumnRT = rt.type({ columnId: rt.string, field: rt.string, value: jsonArrayRT, highlights: rt.array(rt.string), }); +export type LogFieldColumn = rt.TypeOf; + export const logMessageColumnRT = rt.type({ columnId: rt.string, message: rt.array(logMessagePartRT), }); +export type LogMessageColumn = rt.TypeOf; export const logColumnRT = rt.union([logTimestampColumnRT, logFieldColumnRT, logMessageColumnRT]); +export type LogColumn = rt.TypeOf; +/** + * fields + */ export const logEntryContextRT = rt.union([ rt.type({}), rt.type({ 'container.id': rt.string }), rt.type({ 'host.name': rt.string, 'log.file.path': rt.string }), ]); +export type LogEntryContext = rt.TypeOf; + +export const logEntryFieldRT = rt.type({ + field: rt.string, + value: jsonArrayRT, +}); +export type LogEntryField = rt.TypeOf; + +/** + * entry + */ export const logEntryRT = rt.type({ id: rt.string, + index: rt.string, cursor: logEntryCursorRT, columns: rt.array(logColumnRT), context: logEntryContextRT, }); - -export type LogMessageConstantPart = rt.TypeOf; -export type LogMessageFieldPart = rt.TypeOf; -export type LogMessagePart = rt.TypeOf; -export type LogEntryContext = rt.TypeOf; export type LogEntry = rt.TypeOf; -export type LogTimestampColumn = rt.TypeOf; -export type LogFieldColumn = rt.TypeOf; -export type LogMessageColumn = rt.TypeOf; -export type LogColumn = rt.TypeOf; diff --git a/x-pack/plugins/infra/common/log_entry/log_entry_cursor.ts b/x-pack/plugins/infra/common/log_entry/log_entry_cursor.ts index 280403dd5438d4..b11a48822e758f 100644 --- a/x-pack/plugins/infra/common/log_entry/log_entry_cursor.ts +++ b/x-pack/plugins/infra/common/log_entry/log_entry_cursor.ts @@ -11,9 +11,23 @@ export const logEntryCursorRT = rt.type({ time: rt.number, tiebreaker: rt.number, }); - export type LogEntryCursor = rt.TypeOf; +export const logEntryBeforeCursorRT = rt.type({ + before: rt.union([logEntryCursorRT, rt.literal('last')]), +}); +export type LogEntryBeforeCursor = rt.TypeOf; + +export const logEntryAfterCursorRT = rt.type({ + after: rt.union([logEntryCursorRT, rt.literal('first')]), +}); +export type LogEntryAfterCursor = rt.TypeOf; + +export const logEntryAroundCursorRT = rt.type({ + center: logEntryCursorRT, +}); +export type LogEntryAroundCursor = rt.TypeOf; + export const getLogEntryCursorFromHit = (hit: { sort: [number, number] }) => decodeOrThrow(logEntryCursorRT)({ time: hit.sort[0], diff --git a/x-pack/plugins/infra/common/search_strategies/log_entries/log_entries.ts b/x-pack/plugins/infra/common/search_strategies/log_entries/log_entries.ts new file mode 100644 index 00000000000000..b2a879c3b72fd3 --- /dev/null +++ b/x-pack/plugins/infra/common/search_strategies/log_entries/log_entries.ts @@ -0,0 +1,74 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import * as rt from 'io-ts'; +import { DslQuery } from '../../../../../../src/plugins/data/common'; +import { logSourceColumnConfigurationRT } from '../../http_api/log_sources'; +import { + logEntryAfterCursorRT, + logEntryBeforeCursorRT, + logEntryCursorRT, + logEntryRT, +} from '../../log_entry'; +import { JsonObject, jsonObjectRT } from '../../typed_json'; +import { searchStrategyErrorRT } from '../common/errors'; + +export const LOG_ENTRIES_SEARCH_STRATEGY = 'infra-log-entries'; + +const logEntriesBaseSearchRequestParamsRT = rt.intersection([ + rt.type({ + sourceId: rt.string, + startTimestamp: rt.number, + endTimestamp: rt.number, + size: rt.number, + }), + rt.partial({ + query: jsonObjectRT, + columns: rt.array(logSourceColumnConfigurationRT), + highlightPhrase: rt.string, + }), +]); + +export const logEntriesBeforeSearchRequestParamsRT = rt.intersection([ + logEntriesBaseSearchRequestParamsRT, + logEntryBeforeCursorRT, +]); + +export const logEntriesAfterSearchRequestParamsRT = rt.intersection([ + logEntriesBaseSearchRequestParamsRT, + logEntryAfterCursorRT, +]); + +export const logEntriesSearchRequestParamsRT = rt.union([ + logEntriesBaseSearchRequestParamsRT, + logEntriesBeforeSearchRequestParamsRT, + logEntriesAfterSearchRequestParamsRT, +]); + +export type LogEntriesSearchRequestParams = rt.TypeOf; + +export type LogEntriesSearchRequestQuery = JsonObject | DslQuery; + +export const logEntriesSearchResponsePayloadRT = rt.intersection([ + rt.type({ + data: rt.intersection([ + rt.type({ + entries: rt.array(logEntryRT), + topCursor: rt.union([logEntryCursorRT, rt.null]), + bottomCursor: rt.union([logEntryCursorRT, rt.null]), + }), + rt.partial({ + hasMoreBefore: rt.boolean, + hasMoreAfter: rt.boolean, + }), + ]), + }), + rt.partial({ + errors: rt.array(searchStrategyErrorRT), + }), +]); + +export type LogEntriesSearchResponsePayload = rt.TypeOf; diff --git a/x-pack/plugins/infra/common/search_strategies/log_entries/log_entry.ts b/x-pack/plugins/infra/common/search_strategies/log_entries/log_entry.ts index af6bd203f980ea..986f6baf044881 100644 --- a/x-pack/plugins/infra/common/search_strategies/log_entries/log_entry.ts +++ b/x-pack/plugins/infra/common/search_strategies/log_entries/log_entry.ts @@ -5,8 +5,7 @@ */ import * as rt from 'io-ts'; -import { logEntryCursorRT } from '../../log_entry'; -import { jsonArrayRT } from '../../typed_json'; +import { logEntryCursorRT, logEntryFieldRT } from '../../log_entry'; import { searchStrategyErrorRT } from '../common/errors'; export const LOG_ENTRY_SEARCH_STRATEGY = 'infra-log-entry'; @@ -18,18 +17,11 @@ export const logEntrySearchRequestParamsRT = rt.type({ export type LogEntrySearchRequestParams = rt.TypeOf; -const logEntryFieldRT = rt.type({ - field: rt.string, - value: jsonArrayRT, -}); - -export type LogEntryField = rt.TypeOf; - export const logEntryRT = rt.type({ id: rt.string, index: rt.string, fields: rt.array(logEntryFieldRT), - key: logEntryCursorRT, + cursor: logEntryCursorRT, }); export type LogEntry = rt.TypeOf; diff --git a/x-pack/plugins/infra/common/typed_json.ts b/x-pack/plugins/infra/common/typed_json.ts index f3e7608910e095..5aec8d3eaf2ccf 100644 --- a/x-pack/plugins/infra/common/typed_json.ts +++ b/x-pack/plugins/infra/common/typed_json.ts @@ -7,6 +7,8 @@ import * as rt from 'io-ts'; import { JsonArray, JsonObject, JsonValue } from '../../../../src/plugins/kibana_utils/common'; +export { JsonArray, JsonObject, JsonValue }; + export const jsonScalarRT = rt.union([rt.null, rt.boolean, rt.number, rt.string]); export const jsonValueRT: rt.Type = rt.recursion('JsonValue', () => diff --git a/x-pack/plugins/infra/public/components/log_stream/log_stream.stories.mdx b/x-pack/plugins/infra/public/components/log_stream/log_stream.stories.mdx index bda52d9323eb60..901a4b6a8383e3 100644 --- a/x-pack/plugins/infra/public/components/log_stream/log_stream.stories.mdx +++ b/x-pack/plugins/infra/public/components/log_stream/log_stream.stories.mdx @@ -1,10 +1,12 @@ import { Meta, Story, Canvas, ArgsTable } from '@storybook/addon-docs/blocks'; -import { Subject } from 'rxjs'; +import { defer, of, Subject } from 'rxjs'; +import { delay } from 'rxjs/operators'; import { I18nProvider } from '@kbn/i18n/react'; import { EuiThemeProvider } from '../../../../../../src/plugins/kibana_react/common'; import { KibanaContextProvider } from '../../../../../../src/plugins/kibana_react/public'; +import { LOG_ENTRIES_SEARCH_STRATEGY } from '../../../common/search_strategies/log_entries/log_entries'; import { DEFAULT_SOURCE_CONFIGURATION } from '../../test_utils/source_configuration'; import { generateFakeEntries, ENTRIES_EMPTY } from '../../test_utils/entries'; @@ -15,30 +17,61 @@ import { LogStream } from './'; export const startTimestamp = 1595145600000; export const endTimestamp = startTimestamp + 15 * 60 * 1000; +export const dataMock = { + search: { + search: ({ params }, options) => { + return defer(() => { + switch (options.strategy) { + case LOG_ENTRIES_SEARCH_STRATEGY: + if (params.after?.time === params.endTimestamp || params.before?.time === params.startTimestamp) { + return of({ + id: 'EMPTY_FAKE_RESPONSE', + total: 1, + loaded: 1, + isRunning: false, + isPartial: false, + rawResponse: ENTRIES_EMPTY, + }); + } else { + const entries = generateFakeEntries( + 200, + params.startTimestamp, + params.endTimestamp, + params.columns || DEFAULT_SOURCE_CONFIGURATION.data.configuration.logColumns + ); + return of({ + id: 'FAKE_RESPONSE', + total: 1, + loaded: 1, + isRunning: false, + isPartial: false, + rawResponse: { + data: { + entries, + topCursor: entries[0].cursor, + bottomCursor: entries[entries.length - 1].cursor, + hasMoreBefore: false, + }, + errors: [], + } + }); + } + default: + return of({ + id: 'FAKE_RESPONSE', + rawResponse: {}, + }); + } + }).pipe(delay(2000)); + }, + }, +}; + + export const fetch = function (url, params) { switch (url) { case '/api/infra/log_source_configurations/default': return DEFAULT_SOURCE_CONFIGURATION; - case '/api/log_entries/entries': - const body = JSON.parse(params.body); - if (body.after?.time === body.endTimestamp || body.before?.time === body.startTimestamp) { - return ENTRIES_EMPTY; - } else { - const entries = generateFakeEntries( - 200, - body.startTimestamp, - body.endTimestamp, - body.columns || DEFAULT_SOURCE_CONFIGURATION.data.configuration.logColumns - ); - return { - data: { - entries, - topCursor: entries[0].cursor, - bottomCursor: entries[entries.length - 1].cursor, - hasMoreBefore: false, - }, - }; - } default: return {}; } @@ -67,7 +100,7 @@ export const Template = (args) => ; (story) => ( - + {story()} diff --git a/x-pack/plugins/infra/public/components/log_stream/log_stream.tsx b/x-pack/plugins/infra/public/components/log_stream/log_stream.tsx index b7410fda6f6fd4..ab9bc0099f1969 100644 --- a/x-pack/plugins/infra/public/components/log_stream/log_stream.tsx +++ b/x-pack/plugins/infra/public/components/log_stream/log_stream.tsx @@ -101,14 +101,14 @@ Read more at https://github.com/elastic/kibana/blob/master/src/plugins/kibana_re // Internal state const { - loadingState, - pageLoadingState, entries, - hasMoreBefore, - hasMoreAfter, fetchEntries, - fetchPreviousEntries, fetchNextEntries, + fetchPreviousEntries, + hasMoreAfter, + hasMoreBefore, + isLoadingMore, + isReloading, } = useLogStream({ sourceId, startTimestamp, @@ -118,12 +118,6 @@ Read more at https://github.com/elastic/kibana/blob/master/src/plugins/kibana_re columns: customColumns, }); - // Derived state - const isReloading = - isLoadingSourceConfiguration || loadingState === 'uninitialized' || loadingState === 'loading'; - - const isLoadingMore = pageLoadingState === 'loading'; - const columnConfigurations = useMemo(() => { return sourceConfiguration ? customColumns ?? sourceConfiguration.configuration.logColumns : []; }, [sourceConfiguration, customColumns]); @@ -177,7 +171,7 @@ Read more at https://github.com/elastic/kibana/blob/master/src/plugins/kibana_re items={streamItems} scale="medium" wrap={true} - isReloading={isReloading} + isReloading={isLoadingSourceConfiguration || isReloading} isLoadingMore={isLoadingMore} hasMoreBeforeStart={hasMoreBefore} hasMoreAfterEnd={hasMoreAfter} diff --git a/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_actions_menu.test.tsx b/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_actions_menu.test.tsx index f578292d6d6fcb..447e6afbbf1fdb 100644 --- a/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_actions_menu.test.tsx +++ b/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_actions_menu.test.tsx @@ -32,7 +32,7 @@ describe('LogEntryActionsMenu component', () => { fields: [{ field: 'host.ip', value: ['HOST_IP'] }], id: 'ITEM_ID', index: 'INDEX', - key: { + cursor: { time: 0, tiebreaker: 0, }, @@ -62,7 +62,7 @@ describe('LogEntryActionsMenu component', () => { fields: [{ field: 'container.id', value: ['CONTAINER_ID'] }], id: 'ITEM_ID', index: 'INDEX', - key: { + cursor: { time: 0, tiebreaker: 0, }, @@ -92,7 +92,7 @@ describe('LogEntryActionsMenu component', () => { fields: [{ field: 'kubernetes.pod.uid', value: ['POD_UID'] }], id: 'ITEM_ID', index: 'INDEX', - key: { + cursor: { time: 0, tiebreaker: 0, }, @@ -126,7 +126,7 @@ describe('LogEntryActionsMenu component', () => { ], id: 'ITEM_ID', index: 'INDEX', - key: { + cursor: { time: 0, tiebreaker: 0, }, @@ -158,7 +158,7 @@ describe('LogEntryActionsMenu component', () => { fields: [], id: 'ITEM_ID', index: 'INDEX', - key: { + cursor: { time: 0, tiebreaker: 0, }, @@ -192,7 +192,7 @@ describe('LogEntryActionsMenu component', () => { fields: [{ field: 'trace.id', value: ['1234567'] }], id: 'ITEM_ID', index: 'INDEX', - key: { + cursor: { time: 0, tiebreaker: 0, }, @@ -226,7 +226,7 @@ describe('LogEntryActionsMenu component', () => { ], id: 'ITEM_ID', index: 'INDEX', - key: { + cursor: { time: 0, tiebreaker: 0, }, @@ -256,7 +256,7 @@ describe('LogEntryActionsMenu component', () => { fields: [], id: 'ITEM_ID', index: 'INDEX', - key: { + cursor: { time: 0, tiebreaker: 0, }, diff --git a/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_fields_table.tsx b/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_fields_table.tsx index 44e9902e0413f1..b3c80a3a4924a2 100644 --- a/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_fields_table.tsx +++ b/x-pack/plugins/infra/public/components/logging/log_entry_flyout/log_entry_fields_table.tsx @@ -7,10 +7,8 @@ import { EuiBasicTableColumn, EuiInMemoryTable } from '@elastic/eui'; import { i18n } from '@kbn/i18n'; import React, { useMemo } from 'react'; -import { - LogEntry, - LogEntryField, -} from '../../../../common/search_strategies/log_entries/log_entry'; +import { LogEntryField } from '../../../../common/log_entry'; +import { LogEntry } from '../../../../common/search_strategies/log_entries/log_entry'; import { TimeKey } from '../../../../common/time'; import { FieldValue } from '../log_text_stream/field_value'; @@ -22,7 +20,7 @@ export const LogEntryFieldsTable: React.FC<{ () => onSetFieldFilter ? (field: LogEntryField) => () => { - onSetFieldFilter?.(`${field.field}:"${field.value}"`, logEntry.id, logEntry.key); + onSetFieldFilter?.(`${field.field}:"${field.value}"`, logEntry.id, logEntry.cursor); } : undefined, [logEntry, onSetFieldFilter] diff --git a/x-pack/plugins/infra/public/components/logging/log_text_stream/item.ts b/x-pack/plugins/infra/public/components/logging/log_text_stream/item.ts index b0ff36574bedef..13d5b7b889465a 100644 --- a/x-pack/plugins/infra/public/components/logging/log_text_stream/item.ts +++ b/x-pack/plugins/infra/public/components/logging/log_text_stream/item.ts @@ -5,7 +5,6 @@ */ import { bisector } from 'd3-array'; - import { compareToTimeKey, TimeKey } from '../../../../common/time'; import { LogEntry } from '../../../../common/log_entry'; diff --git a/x-pack/plugins/infra/public/components/logging/log_text_stream/log_entry_row.tsx b/x-pack/plugins/infra/public/components/logging/log_text_stream/log_entry_row.tsx index 1a472df2b5c906..036818317011c2 100644 --- a/x-pack/plugins/infra/public/components/logging/log_text_stream/log_entry_row.tsx +++ b/x-pack/plugins/infra/public/components/logging/log_text_stream/log_entry_row.tsx @@ -7,7 +7,6 @@ import React, { memo, useState, useCallback, useMemo } from 'react'; import { i18n } from '@kbn/i18n'; import { isEmpty } from 'lodash'; - import { euiStyled } from '../../../../../../../src/plugins/kibana_react/common'; import { useUiTracker } from '../../../../../observability/public'; import { isTimestampColumn } from '../../../utils/log_entry'; diff --git a/x-pack/plugins/infra/public/containers/logs/log_entry.ts b/x-pack/plugins/infra/public/containers/logs/log_entry.ts index af8618b8be5657..60000e0b8baba1 100644 --- a/x-pack/plugins/infra/public/containers/logs/log_entry.ts +++ b/x-pack/plugins/infra/public/containers/logs/log_entry.ts @@ -11,7 +11,11 @@ import { logEntrySearchResponsePayloadRT, LOG_ENTRY_SEARCH_STRATEGY, } from '../../../common/search_strategies/log_entries/log_entry'; -import { useDataSearch, useLatestPartialDataSearchResponse } from '../../utils/data_search'; +import { + normalizeDataSearchResponses, + useDataSearch, + useLatestPartialDataSearchResponse, +} from '../../utils/data_search'; export const useLogEntry = ({ sourceId, @@ -31,6 +35,7 @@ export const useLogEntry = ({ } : null; }, [sourceId, logEntryId]), + parseResponses: parseLogEntrySearchResponses, }); const { @@ -41,11 +46,7 @@ export const useLogEntry = ({ latestResponseErrors, loaded, total, - } = useLatestPartialDataSearchResponse( - logEntrySearchRequests$, - null, - decodeLogEntrySearchResponse - ); + } = useLatestPartialDataSearchResponse(logEntrySearchRequests$); return { cancelRequest, @@ -59,4 +60,7 @@ export const useLogEntry = ({ }; }; -const decodeLogEntrySearchResponse = decodeOrThrow(logEntrySearchResponsePayloadRT); +const parseLogEntrySearchResponses = normalizeDataSearchResponses( + null, + decodeOrThrow(logEntrySearchResponsePayloadRT) +); diff --git a/x-pack/plugins/infra/public/containers/logs/log_highlights/log_entry_highlights.gql_query.ts b/x-pack/plugins/infra/public/containers/logs/log_highlights/log_entry_highlights.gql_query.ts deleted file mode 100644 index 9d9fab58754275..00000000000000 --- a/x-pack/plugins/infra/public/containers/logs/log_highlights/log_entry_highlights.gql_query.ts +++ /dev/null @@ -1,42 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License; - * you may not use this file except in compliance with the Elastic License. - */ - -import gql from 'graphql-tag'; - -import { sharedFragments } from '../../../../common/graphql/shared'; - -export const logEntryHighlightsQuery = gql` - query LogEntryHighlightsQuery( - $sourceId: ID = "default" - $startKey: InfraTimeKeyInput! - $endKey: InfraTimeKeyInput! - $filterQuery: String - $highlights: [InfraLogEntryHighlightInput!]! - ) { - source(id: $sourceId) { - id - logEntryHighlights( - startKey: $startKey - endKey: $endKey - filterQuery: $filterQuery - highlights: $highlights - ) { - start { - ...InfraTimeKeyFields - } - end { - ...InfraTimeKeyFields - } - entries { - ...InfraLogEntryHighlightFields - } - } - } - } - - ${sharedFragments.InfraTimeKey} - ${sharedFragments.InfraLogEntryHighlightFields} -`; diff --git a/x-pack/plugins/infra/public/containers/logs/log_highlights/log_entry_highlights.tsx b/x-pack/plugins/infra/public/containers/logs/log_highlights/log_entry_highlights.tsx index fb72874df54099..caac28a0756a16 100644 --- a/x-pack/plugins/infra/public/containers/logs/log_highlights/log_entry_highlights.tsx +++ b/x-pack/plugins/infra/public/containers/logs/log_highlights/log_entry_highlights.tsx @@ -5,13 +5,12 @@ */ import { useEffect, useMemo, useState } from 'react'; - -import { TimeKey } from '../../../../common/time'; -import { useTrackedPromise } from '../../../utils/use_tracked_promise'; -import { fetchLogEntriesHighlights } from './api/fetch_log_entries_highlights'; import { LogEntriesHighlightsResponse } from '../../../../common/http_api'; import { LogEntry } from '../../../../common/log_entry'; +import { TimeKey } from '../../../../common/time'; import { useKibanaContextForPlugin } from '../../../hooks/use_kibana'; +import { useTrackedPromise } from '../../../utils/use_tracked_promise'; +import { fetchLogEntriesHighlights } from './api/fetch_log_entries_highlights'; export const useLogEntryHighlights = ( sourceId: string, diff --git a/x-pack/plugins/infra/public/containers/logs/log_stream/index.ts b/x-pack/plugins/infra/public/containers/logs/log_stream/index.ts index 1d9a7a1b1d7779..8343525c828628 100644 --- a/x-pack/plugins/infra/public/containers/logs/log_stream/index.ts +++ b/x-pack/plugins/infra/public/containers/logs/log_stream/index.ts @@ -4,15 +4,16 @@ * you may not use this file except in compliance with the Elastic License. */ -import { useMemo, useEffect } from 'react'; -import useSetState from 'react-use/lib/useSetState'; +import { useCallback, useEffect, useMemo } from 'react'; import usePrevious from 'react-use/lib/usePrevious'; +import useSetState from 'react-use/lib/useSetState'; import { esKuery, esQuery, Query } from '../../../../../../../src/plugins/data/public'; -import { fetchLogEntries } from '../log_entries/api/fetch_log_entries'; -import { useTrackedPromise } from '../../../utils/use_tracked_promise'; -import { LogEntryCursor, LogEntry } from '../../../../common/log_entry'; -import { useKibanaContextForPlugin } from '../../../hooks/use_kibana'; +import { LogEntry, LogEntryCursor } from '../../../../common/log_entry'; +import { useSubscription } from '../../../utils/use_observable'; import { LogSourceConfigurationProperties } from '../log_source'; +import { useFetchLogEntriesAfter } from './use_fetch_log_entries_after'; +import { useFetchLogEntriesAround } from './use_fetch_log_entries_around'; +import { useFetchLogEntriesBefore } from './use_fetch_log_entries_before'; interface LogStreamProps { sourceId: string; @@ -31,16 +32,6 @@ interface LogStreamState { hasMoreAfter: boolean; } -type LoadingState = 'uninitialized' | 'loading' | 'success' | 'error'; - -interface LogStreamReturn extends LogStreamState { - fetchEntries: () => void; - fetchPreviousEntries: () => void; - fetchNextEntries: () => void; - loadingState: LoadingState; - pageLoadingState: LoadingState; -} - const INITIAL_STATE: LogStreamState = { entries: [], topCursor: null, @@ -50,11 +41,7 @@ const INITIAL_STATE: LogStreamState = { hasMoreAfter: true, }; -const EMPTY_DATA = { - entries: [], - topCursor: null, - bottomCursor: null, -}; +const LOG_ENTRIES_CHUNK_SIZE = 200; export function useLogStream({ sourceId, @@ -63,8 +50,7 @@ export function useLogStream({ query, center, columns, -}: LogStreamProps): LogStreamReturn { - const { services } = useKibanaContextForPlugin(); +}: LogStreamProps) { const [state, setState] = useSetState(INITIAL_STATE); // Ensure the pagination keeps working when the timerange gets extended @@ -85,175 +71,151 @@ export function useLogStream({ const parsedQuery = useMemo(() => { if (!query) { - return null; - } - - let q; - - if (typeof query === 'string') { - q = esKuery.toElasticsearchQuery(esKuery.fromKueryExpression(query)); + return undefined; + } else if (typeof query === 'string') { + return esKuery.toElasticsearchQuery(esKuery.fromKueryExpression(query)); } else if (query.language === 'kuery') { - q = esKuery.toElasticsearchQuery(esKuery.fromKueryExpression(query.query as string)); + return esKuery.toElasticsearchQuery(esKuery.fromKueryExpression(query.query as string)); } else if (query.language === 'lucene') { - q = esQuery.luceneStringToDsl(query.query as string); + return esQuery.luceneStringToDsl(query.query as string); + } else { + return undefined; } - - return JSON.stringify(q); }, [query]); - // Callbacks - const [entriesPromise, fetchEntries] = useTrackedPromise( - { - cancelPreviousOn: 'creation', - createPromise: () => { - setState(INITIAL_STATE); - const fetchPosition = center ? { center } : { before: 'last' }; + const commonFetchArguments = useMemo( + () => ({ + sourceId, + startTimestamp, + endTimestamp, + query: parsedQuery, + columnOverrides: columns, + }), + [columns, endTimestamp, parsedQuery, sourceId, startTimestamp] + ); - return fetchLogEntries( - { - sourceId, - startTimestamp, - endTimestamp, - query: parsedQuery, - columns, - ...fetchPosition, - }, - services.http.fetch - ); - }, - onResolve: ({ data }) => { + const { + fetchLogEntriesAround, + isRequestRunning: isLogEntriesAroundRequestRunning, + logEntriesAroundSearchResponses$, + } = useFetchLogEntriesAround(commonFetchArguments); + + useSubscription(logEntriesAroundSearchResponses$, { + next: ({ before, after, combined }) => { + if ((before.response.data != null || after?.response.data != null) && !combined.isPartial) { setState((prevState) => ({ - ...data, - hasMoreBefore: data.hasMoreBefore ?? prevState.hasMoreBefore, - hasMoreAfter: data.hasMoreAfter ?? prevState.hasMoreAfter, + ...prevState, + entries: combined.entries, + hasMoreAfter: combined.hasMoreAfter ?? prevState.hasMoreAfter, + hasMoreBefore: combined.hasMoreAfter ?? prevState.hasMoreAfter, + bottomCursor: combined.bottomCursor, + topCursor: combined.topCursor, })); - }, + } }, - [sourceId, startTimestamp, endTimestamp, query] - ); + }); - const [previousEntriesPromise, fetchPreviousEntries] = useTrackedPromise( - { - cancelPreviousOn: 'creation', - createPromise: () => { - if (state.topCursor === null) { - throw new Error( - 'useLogState: Cannot fetch previous entries. No cursor is set.\nEnsure you have called `fetchEntries` at least once.' - ); - } + const { + fetchLogEntriesBefore, + isRequestRunning: isLogEntriesBeforeRequestRunning, + logEntriesBeforeSearchResponse$, + } = useFetchLogEntriesBefore(commonFetchArguments); - if (!state.hasMoreBefore) { - return Promise.resolve({ data: EMPTY_DATA }); - } - - return fetchLogEntries( - { - sourceId, - startTimestamp, - endTimestamp, - query: parsedQuery, - before: state.topCursor, - }, - services.http.fetch - ); - }, - onResolve: ({ data }) => { - if (!data.entries.length) { - return; - } + useSubscription(logEntriesBeforeSearchResponse$, { + next: ({ response: { data, isPartial } }) => { + if (data != null && !isPartial) { setState((prevState) => ({ + ...prevState, entries: [...data.entries, ...prevState.entries], hasMoreBefore: data.hasMoreBefore ?? prevState.hasMoreBefore, topCursor: data.topCursor ?? prevState.topCursor, + bottomCursor: prevState.bottomCursor ?? data.bottomCursor, })); - }, + } }, - [sourceId, startTimestamp, endTimestamp, query, state.topCursor] - ); + }); - const [nextEntriesPromise, fetchNextEntries] = useTrackedPromise( - { - cancelPreviousOn: 'creation', - createPromise: () => { - if (state.bottomCursor === null) { - throw new Error( - 'useLogState: Cannot fetch next entries. No cursor is set.\nEnsure you have called `fetchEntries` at least once.' - ); - } + const fetchPreviousEntries = useCallback(() => { + if (state.topCursor === null) { + throw new Error( + 'useLogState: Cannot fetch previous entries. No cursor is set.\nEnsure you have called `fetchEntries` at least once.' + ); + } + + if (!state.hasMoreBefore) { + return; + } - if (!state.hasMoreAfter) { - return Promise.resolve({ data: EMPTY_DATA }); - } + fetchLogEntriesBefore(state.topCursor, LOG_ENTRIES_CHUNK_SIZE); + }, [fetchLogEntriesBefore, state.topCursor, state.hasMoreBefore]); - return fetchLogEntries( - { - sourceId, - startTimestamp, - endTimestamp, - query: parsedQuery, - after: state.bottomCursor, - }, - services.http.fetch - ); - }, - onResolve: ({ data }) => { - if (!data.entries.length) { - return; - } + const { + fetchLogEntriesAfter, + isRequestRunning: isLogEntriesAfterRequestRunning, + logEntriesAfterSearchResponse$, + } = useFetchLogEntriesAfter(commonFetchArguments); + + useSubscription(logEntriesAfterSearchResponse$, { + next: ({ response: { data, isPartial } }) => { + if (data != null && !isPartial) { setState((prevState) => ({ + ...prevState, entries: [...prevState.entries, ...data.entries], hasMoreAfter: data.hasMoreAfter ?? prevState.hasMoreAfter, + topCursor: prevState.topCursor ?? data.topCursor, bottomCursor: data.bottomCursor ?? prevState.bottomCursor, })); - }, + } }, - [sourceId, startTimestamp, endTimestamp, query, state.bottomCursor] - ); - - const loadingState = useMemo( - () => convertPromiseStateToLoadingState(entriesPromise.state), - [entriesPromise.state] - ); + }); - const pageLoadingState = useMemo(() => { - const states = [previousEntriesPromise.state, nextEntriesPromise.state]; - - if (states.includes('pending')) { - return 'loading'; + const fetchNextEntries = useCallback(() => { + if (state.bottomCursor === null) { + throw new Error( + 'useLogState: Cannot fetch next entries. No cursor is set.\nEnsure you have called `fetchEntries` at least once.' + ); } - if (states.includes('rejected')) { - return 'error'; + if (!state.hasMoreAfter) { + return; } - if (states.includes('resolved')) { - return 'success'; + fetchLogEntriesAfter(state.bottomCursor, LOG_ENTRIES_CHUNK_SIZE); + }, [fetchLogEntriesAfter, state.bottomCursor, state.hasMoreAfter]); + + const fetchEntries = useCallback(() => { + setState(INITIAL_STATE); + + if (center) { + fetchLogEntriesAround(center, LOG_ENTRIES_CHUNK_SIZE); + } else { + fetchLogEntriesBefore('last', LOG_ENTRIES_CHUNK_SIZE); } + }, [center, fetchLogEntriesAround, fetchLogEntriesBefore, setState]); + + const isReloading = useMemo( + () => + isLogEntriesAroundRequestRunning || + (state.bottomCursor == null && state.topCursor == null && isLogEntriesBeforeRequestRunning), + [ + isLogEntriesAroundRequestRunning, + isLogEntriesBeforeRequestRunning, + state.bottomCursor, + state.topCursor, + ] + ); - return 'uninitialized'; - }, [previousEntriesPromise.state, nextEntriesPromise.state]); + const isLoadingMore = useMemo( + () => isLogEntriesBeforeRequestRunning || isLogEntriesAfterRequestRunning, + [isLogEntriesAfterRequestRunning, isLogEntriesBeforeRequestRunning] + ); return { ...state, fetchEntries, - fetchPreviousEntries, fetchNextEntries, - loadingState, - pageLoadingState, + fetchPreviousEntries, + isLoadingMore, + isReloading, }; } - -function convertPromiseStateToLoadingState( - state: 'uninitialized' | 'pending' | 'resolved' | 'rejected' -): LoadingState { - switch (state) { - case 'uninitialized': - return 'uninitialized'; - case 'pending': - return 'loading'; - case 'resolved': - return 'success'; - case 'rejected': - return 'error'; - } -} diff --git a/x-pack/plugins/infra/public/containers/logs/log_stream/use_fetch_log_entries_after.ts b/x-pack/plugins/infra/public/containers/logs/log_stream/use_fetch_log_entries_after.ts new file mode 100644 index 00000000000000..c7076ec51db6a3 --- /dev/null +++ b/x-pack/plugins/infra/public/containers/logs/log_stream/use_fetch_log_entries_after.ts @@ -0,0 +1,157 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { useCallback } from 'react'; +import { Observable } from 'rxjs'; +import { exhaustMap } from 'rxjs/operators'; +import { IKibanaSearchRequest } from '../../../../../../../src/plugins/data/public'; +import { LogSourceColumnConfiguration } from '../../../../common/http_api/log_sources'; +import { LogEntryAfterCursor } from '../../../../common/log_entry'; +import { decodeOrThrow } from '../../../../common/runtime_types'; +import { + logEntriesSearchRequestParamsRT, + LogEntriesSearchRequestQuery, + LogEntriesSearchResponsePayload, + logEntriesSearchResponsePayloadRT, + LOG_ENTRIES_SEARCH_STRATEGY, +} from '../../../../common/search_strategies/log_entries/log_entries'; +import { + flattenDataSearchResponseDescriptor, + normalizeDataSearchResponses, + ParsedDataSearchRequestDescriptor, + useDataSearch, + useDataSearchResponseState, +} from '../../../utils/data_search'; +import { useOperator } from '../../../utils/use_observable'; + +export const useLogEntriesAfterRequest = ({ + columnOverrides, + endTimestamp, + highlightPhrase, + query, + sourceId, + startTimestamp, +}: { + columnOverrides?: LogSourceColumnConfiguration[]; + endTimestamp: number; + highlightPhrase?: string; + query?: LogEntriesSearchRequestQuery; + sourceId: string; + startTimestamp: number; +}) => { + const { search: fetchLogEntriesAfter, requests$: logEntriesAfterSearchRequests$ } = useDataSearch( + { + getRequest: useCallback( + (cursor: LogEntryAfterCursor['after'], size: number) => { + return !!sourceId + ? { + request: { + params: logEntriesSearchRequestParamsRT.encode({ + after: cursor, + columns: columnOverrides, + endTimestamp, + highlightPhrase, + query, + size, + sourceId, + startTimestamp, + }), + }, + options: { strategy: LOG_ENTRIES_SEARCH_STRATEGY }, + } + : null; + }, + [columnOverrides, endTimestamp, highlightPhrase, query, sourceId, startTimestamp] + ), + parseResponses: parseLogEntriesAfterSearchResponses, + } + ); + + return { + fetchLogEntriesAfter, + logEntriesAfterSearchRequests$, + }; +}; + +export const useLogEntriesAfterResponse = ( + logEntriesAfterSearchRequests$: Observable< + ParsedDataSearchRequestDescriptor + > +) => { + const logEntriesAfterSearchResponse$ = useOperator( + logEntriesAfterSearchRequests$, + flattenLogEntriesAfterSearchResponse + ); + + const { + cancelRequest, + isRequestRunning, + isResponsePartial, + loaded, + total, + } = useDataSearchResponseState(logEntriesAfterSearchResponse$); + + return { + cancelRequest, + isRequestRunning, + isResponsePartial, + loaded, + logEntriesAfterSearchRequests$, + logEntriesAfterSearchResponse$, + total, + }; +}; + +export const useFetchLogEntriesAfter = ({ + columnOverrides, + endTimestamp, + highlightPhrase, + query, + sourceId, + startTimestamp, +}: { + columnOverrides?: LogSourceColumnConfiguration[]; + endTimestamp: number; + highlightPhrase?: string; + query?: LogEntriesSearchRequestQuery; + sourceId: string; + startTimestamp: number; +}) => { + const { fetchLogEntriesAfter, logEntriesAfterSearchRequests$ } = useLogEntriesAfterRequest({ + columnOverrides, + endTimestamp, + highlightPhrase, + query, + sourceId, + startTimestamp, + }); + + const { + cancelRequest, + isRequestRunning, + isResponsePartial, + loaded, + logEntriesAfterSearchResponse$, + total, + } = useLogEntriesAfterResponse(logEntriesAfterSearchRequests$); + + return { + cancelRequest, + fetchLogEntriesAfter, + isRequestRunning, + isResponsePartial, + loaded, + logEntriesAfterSearchResponse$, + total, + }; +}; + +export const parseLogEntriesAfterSearchResponses = normalizeDataSearchResponses( + null, + decodeOrThrow(logEntriesSearchResponsePayloadRT) +); + +const flattenLogEntriesAfterSearchResponse = exhaustMap(flattenDataSearchResponseDescriptor); diff --git a/x-pack/plugins/infra/public/containers/logs/log_stream/use_fetch_log_entries_around.ts b/x-pack/plugins/infra/public/containers/logs/log_stream/use_fetch_log_entries_around.ts new file mode 100644 index 00000000000000..01f6336e0d5c8b --- /dev/null +++ b/x-pack/plugins/infra/public/containers/logs/log_stream/use_fetch_log_entries_around.ts @@ -0,0 +1,184 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { useCallback } from 'react'; +import { combineLatest, Observable, Subject } from 'rxjs'; +import { last, map, startWith, switchMap } from 'rxjs/operators'; +import { LogSourceColumnConfiguration } from '../../../../common/http_api/log_sources'; +import { LogEntryCursor } from '../../../../common/log_entry'; +import { LogEntriesSearchRequestQuery } from '../../../../common/search_strategies/log_entries/log_entries'; +import { flattenDataSearchResponseDescriptor } from '../../../utils/data_search'; +import { useObservable, useObservableState } from '../../../utils/use_observable'; +import { useLogEntriesAfterRequest } from './use_fetch_log_entries_after'; +import { useLogEntriesBeforeRequest } from './use_fetch_log_entries_before'; + +export const useFetchLogEntriesAround = ({ + columnOverrides, + endTimestamp, + highlightPhrase, + query, + sourceId, + startTimestamp, +}: { + columnOverrides?: LogSourceColumnConfiguration[]; + endTimestamp: number; + highlightPhrase?: string; + query?: LogEntriesSearchRequestQuery; + sourceId: string; + startTimestamp: number; +}) => { + const { fetchLogEntriesBefore } = useLogEntriesBeforeRequest({ + columnOverrides, + endTimestamp, + highlightPhrase, + query, + sourceId, + startTimestamp, + }); + + const { fetchLogEntriesAfter } = useLogEntriesAfterRequest({ + columnOverrides, + endTimestamp, + highlightPhrase, + query, + sourceId, + startTimestamp, + }); + + type LogEntriesBeforeRequest = NonNullable>; + type LogEntriesAfterRequest = NonNullable>; + + const logEntriesAroundSearchRequests$ = useObservable( + () => new Subject<[LogEntriesBeforeRequest, Observable]>(), + [] + ); + + const fetchLogEntriesAround = useCallback( + (cursor: LogEntryCursor, size: number) => { + const logEntriesBeforeSearchRequest = fetchLogEntriesBefore(cursor, Math.floor(size / 2)); + + if (logEntriesBeforeSearchRequest == null) { + return; + } + + const logEntriesAfterSearchRequest$ = flattenDataSearchResponseDescriptor( + logEntriesBeforeSearchRequest + ).pipe( + last(), // in the future we could start earlier if we receive partial results already + map((lastBeforeSearchResponse) => { + const cursorAfter = lastBeforeSearchResponse.response.data?.bottomCursor ?? { + time: cursor.time - 1, + tiebreaker: 0, + }; + + const logEntriesAfterSearchRequest = fetchLogEntriesAfter( + cursorAfter, + Math.ceil(size / 2) + ); + + if (logEntriesAfterSearchRequest == null) { + throw new Error('Failed to create request: no request args given'); + } + + return logEntriesAfterSearchRequest; + }) + ); + + logEntriesAroundSearchRequests$.next([ + logEntriesBeforeSearchRequest, + logEntriesAfterSearchRequest$, + ]); + }, + [fetchLogEntriesAfter, fetchLogEntriesBefore, logEntriesAroundSearchRequests$] + ); + + const logEntriesAroundSearchResponses$ = useObservable( + (inputs$) => + inputs$.pipe( + switchMap(([currentSearchRequests$]) => + currentSearchRequests$.pipe( + switchMap(([beforeRequest, afterRequest$]) => { + const beforeResponse$ = flattenDataSearchResponseDescriptor(beforeRequest); + const afterResponse$ = afterRequest$.pipe( + switchMap(flattenDataSearchResponseDescriptor), + startWith(undefined) // emit "before" response even if "after" hasn't started yet + ); + return combineLatest([beforeResponse$, afterResponse$]); + }), + map(([beforeResponse, afterResponse]) => { + const loadedBefore = beforeResponse.response.loaded; + const loadedAfter = afterResponse?.response.loaded; + const totalBefore = beforeResponse.response.total; + const totalAfter = afterResponse?.response.total; + + return { + before: beforeResponse, + after: afterResponse, + combined: { + isRunning: + (beforeResponse.response.isRunning || afterResponse?.response.isRunning) ?? + false, + isPartial: + (beforeResponse.response.isPartial || afterResponse?.response.isPartial) ?? + false, + loaded: + loadedBefore != null || loadedAfter != null + ? (loadedBefore ?? 0) + (loadedAfter ?? 0) + : undefined, + total: + totalBefore != null || totalAfter != null + ? (totalBefore ?? 0) + (totalAfter ?? 0) + : undefined, + entries: [ + ...(beforeResponse.response.data?.entries ?? []), + ...(afterResponse?.response.data?.entries ?? []), + ], + errors: [ + ...(beforeResponse.response.errors ?? []), + ...(afterResponse?.response.errors ?? []), + ], + hasMoreBefore: beforeResponse.response.data?.hasMoreBefore, + hasMoreAfter: afterResponse?.response.data?.hasMoreAfter, + topCursor: beforeResponse.response.data?.topCursor, + bottomCursor: afterResponse?.response.data?.bottomCursor, + }, + }; + }) + ) + ) + ), + [logEntriesAroundSearchRequests$] + ); + + const { + latestValue: { + before: latestBeforeResponse, + after: latestAfterResponse, + combined: latestCombinedResponse, + }, + } = useObservableState(logEntriesAroundSearchResponses$, initialCombinedResponse); + + const cancelRequest = useCallback(() => { + latestBeforeResponse?.abortController.abort(); + latestAfterResponse?.abortController.abort(); + }, [latestBeforeResponse, latestAfterResponse]); + + return { + cancelRequest, + fetchLogEntriesAround, + isRequestRunning: latestCombinedResponse?.isRunning ?? false, + isResponsePartial: latestCombinedResponse?.isPartial ?? false, + loaded: latestCombinedResponse?.loaded, + logEntriesAroundSearchResponses$, + total: latestCombinedResponse?.total, + }; +}; + +const initialCombinedResponse = { + before: undefined, + after: undefined, + combined: undefined, +} as const; diff --git a/x-pack/plugins/infra/public/containers/logs/log_stream/use_fetch_log_entries_before.ts b/x-pack/plugins/infra/public/containers/logs/log_stream/use_fetch_log_entries_before.ts new file mode 100644 index 00000000000000..5553be11b9fef3 --- /dev/null +++ b/x-pack/plugins/infra/public/containers/logs/log_stream/use_fetch_log_entries_before.ts @@ -0,0 +1,158 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { useCallback } from 'react'; +import { Observable } from 'rxjs'; +import { exhaustMap } from 'rxjs/operators'; +import { IKibanaSearchRequest } from '../../../../../../../src/plugins/data/public'; +import { LogSourceColumnConfiguration } from '../../../../common/http_api/log_sources'; +import { LogEntryBeforeCursor } from '../../../../common/log_entry'; +import { decodeOrThrow } from '../../../../common/runtime_types'; +import { + logEntriesSearchRequestParamsRT, + LogEntriesSearchRequestQuery, + LogEntriesSearchResponsePayload, + logEntriesSearchResponsePayloadRT, + LOG_ENTRIES_SEARCH_STRATEGY, +} from '../../../../common/search_strategies/log_entries/log_entries'; +import { + flattenDataSearchResponseDescriptor, + normalizeDataSearchResponses, + ParsedDataSearchRequestDescriptor, + useDataSearch, + useDataSearchResponseState, +} from '../../../utils/data_search'; +import { useOperator } from '../../../utils/use_observable'; + +export const useLogEntriesBeforeRequest = ({ + columnOverrides, + endTimestamp, + highlightPhrase, + query, + sourceId, + startTimestamp, +}: { + columnOverrides?: LogSourceColumnConfiguration[]; + endTimestamp: number; + highlightPhrase?: string; + query?: LogEntriesSearchRequestQuery; + sourceId: string; + startTimestamp: number; +}) => { + const { + search: fetchLogEntriesBefore, + requests$: logEntriesBeforeSearchRequests$, + } = useDataSearch({ + getRequest: useCallback( + (cursor: LogEntryBeforeCursor['before'], size: number) => { + return !!sourceId + ? { + request: { + params: logEntriesSearchRequestParamsRT.encode({ + before: cursor, + columns: columnOverrides, + endTimestamp, + highlightPhrase, + query, + size, + sourceId, + startTimestamp, + }), + }, + options: { strategy: LOG_ENTRIES_SEARCH_STRATEGY }, + } + : null; + }, + [columnOverrides, endTimestamp, highlightPhrase, query, sourceId, startTimestamp] + ), + parseResponses: parseLogEntriesBeforeSearchResponses, + }); + + return { + fetchLogEntriesBefore, + logEntriesBeforeSearchRequests$, + }; +}; + +export const useLogEntriesBeforeResponse = ( + logEntriesBeforeSearchRequests$: Observable< + ParsedDataSearchRequestDescriptor + > +) => { + const logEntriesBeforeSearchResponse$ = useOperator( + logEntriesBeforeSearchRequests$, + flattenLogEntriesBeforeSearchResponse + ); + + const { + cancelRequest, + isRequestRunning, + isResponsePartial, + loaded, + total, + } = useDataSearchResponseState(logEntriesBeforeSearchResponse$); + + return { + cancelRequest, + isRequestRunning, + isResponsePartial, + loaded, + logEntriesBeforeSearchRequests$, + logEntriesBeforeSearchResponse$, + total, + }; +}; + +export const useFetchLogEntriesBefore = ({ + columnOverrides, + endTimestamp, + highlightPhrase, + query, + sourceId, + startTimestamp, +}: { + columnOverrides?: LogSourceColumnConfiguration[]; + endTimestamp: number; + highlightPhrase?: string; + query?: LogEntriesSearchRequestQuery; + sourceId: string; + startTimestamp: number; +}) => { + const { fetchLogEntriesBefore, logEntriesBeforeSearchRequests$ } = useLogEntriesBeforeRequest({ + columnOverrides, + endTimestamp, + highlightPhrase, + query, + sourceId, + startTimestamp, + }); + + const { + cancelRequest, + isRequestRunning, + isResponsePartial, + loaded, + logEntriesBeforeSearchResponse$, + total, + } = useLogEntriesBeforeResponse(logEntriesBeforeSearchRequests$); + + return { + cancelRequest, + fetchLogEntriesBefore, + isRequestRunning, + isResponsePartial, + loaded, + logEntriesBeforeSearchResponse$, + total, + }; +}; + +export const parseLogEntriesBeforeSearchResponses = normalizeDataSearchResponses( + null, + decodeOrThrow(logEntriesSearchResponsePayloadRT) +); + +const flattenLogEntriesBeforeSearchResponse = exhaustMap(flattenDataSearchResponseDescriptor); diff --git a/x-pack/plugins/infra/public/graphql/introspection.json b/x-pack/plugins/infra/public/graphql/introspection.json index 5d351f3259ac5f..efdca72c1383aa 100644 --- a/x-pack/plugins/infra/public/graphql/introspection.json +++ b/x-pack/plugins/infra/public/graphql/introspection.json @@ -137,155 +137,6 @@ "isDeprecated": false, "deprecationReason": null }, - { - "name": "logEntriesAround", - "description": "A consecutive span of log entries surrounding a point in time", - "args": [ - { - "name": "key", - "description": "The sort key that corresponds to the point in time", - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "INPUT_OBJECT", "name": "InfraTimeKeyInput", "ofType": null } - }, - "defaultValue": null - }, - { - "name": "countBefore", - "description": "The maximum number of preceding to return", - "type": { "kind": "SCALAR", "name": "Int", "ofType": null }, - "defaultValue": "0" - }, - { - "name": "countAfter", - "description": "The maximum number of following to return", - "type": { "kind": "SCALAR", "name": "Int", "ofType": null }, - "defaultValue": "0" - }, - { - "name": "filterQuery", - "description": "The query to filter the log entries by", - "type": { "kind": "SCALAR", "name": "String", "ofType": null }, - "defaultValue": null - } - ], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "OBJECT", "name": "InfraLogEntryInterval", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "logEntriesBetween", - "description": "A consecutive span of log entries within an interval", - "args": [ - { - "name": "startKey", - "description": "The sort key that corresponds to the start of the interval", - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "INPUT_OBJECT", "name": "InfraTimeKeyInput", "ofType": null } - }, - "defaultValue": null - }, - { - "name": "endKey", - "description": "The sort key that corresponds to the end of the interval", - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "INPUT_OBJECT", "name": "InfraTimeKeyInput", "ofType": null } - }, - "defaultValue": null - }, - { - "name": "filterQuery", - "description": "The query to filter the log entries by", - "type": { "kind": "SCALAR", "name": "String", "ofType": null }, - "defaultValue": null - } - ], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "OBJECT", "name": "InfraLogEntryInterval", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "logEntryHighlights", - "description": "Sequences of log entries matching sets of highlighting queries within an interval", - "args": [ - { - "name": "startKey", - "description": "The sort key that corresponds to the start of the interval", - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "INPUT_OBJECT", "name": "InfraTimeKeyInput", "ofType": null } - }, - "defaultValue": null - }, - { - "name": "endKey", - "description": "The sort key that corresponds to the end of the interval", - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "INPUT_OBJECT", "name": "InfraTimeKeyInput", "ofType": null } - }, - "defaultValue": null - }, - { - "name": "filterQuery", - "description": "The query to filter the log entries by", - "type": { "kind": "SCALAR", "name": "String", "ofType": null }, - "defaultValue": null - }, - { - "name": "highlights", - "description": "The highlighting to apply to the log entries", - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "LIST", - "name": null, - "ofType": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "INPUT_OBJECT", - "name": "InfraLogEntryHighlightInput", - "ofType": null - } - } - } - }, - "defaultValue": null - } - ], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "LIST", - "name": null, - "ofType": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "OBJECT", "name": "InfraLogEntryInterval", "ofType": null } - } - } - }, - "isDeprecated": false, - "deprecationReason": null - }, { "name": "snapshot", "description": "A snapshot of nodes", @@ -993,37 +844,6 @@ "enumValues": null, "possibleTypes": null }, - { - "kind": "INPUT_OBJECT", - "name": "InfraTimeKeyInput", - "description": "", - "fields": null, - "inputFields": [ - { - "name": "time", - "description": "", - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "Float", "ofType": null } - }, - "defaultValue": null - }, - { - "name": "tiebreaker", - "description": "", - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "Float", "ofType": null } - }, - "defaultValue": null - } - ], - "interfaces": null, - "enumValues": null, - "possibleTypes": null - }, { "kind": "SCALAR", "name": "Int", @@ -1034,486 +854,6 @@ "enumValues": null, "possibleTypes": null }, - { - "kind": "OBJECT", - "name": "InfraLogEntryInterval", - "description": "A consecutive sequence of log entries", - "fields": [ - { - "name": "start", - "description": "The key corresponding to the start of the interval covered by the entries", - "args": [], - "type": { "kind": "OBJECT", "name": "InfraTimeKey", "ofType": null }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "end", - "description": "The key corresponding to the end of the interval covered by the entries", - "args": [], - "type": { "kind": "OBJECT", "name": "InfraTimeKey", "ofType": null }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "hasMoreBefore", - "description": "Whether there are more log entries available before the start", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "Boolean", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "hasMoreAfter", - "description": "Whether there are more log entries available after the end", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "Boolean", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "filterQuery", - "description": "The query the log entries were filtered by", - "args": [], - "type": { "kind": "SCALAR", "name": "String", "ofType": null }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "highlightQuery", - "description": "The query the log entries were highlighted with", - "args": [], - "type": { "kind": "SCALAR", "name": "String", "ofType": null }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "entries", - "description": "A list of the log entries", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "LIST", - "name": null, - "ofType": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "OBJECT", "name": "InfraLogEntry", "ofType": null } - } - } - }, - "isDeprecated": false, - "deprecationReason": null - } - ], - "inputFields": null, - "interfaces": [], - "enumValues": null, - "possibleTypes": null - }, - { - "kind": "OBJECT", - "name": "InfraTimeKey", - "description": "A representation of the log entry's position in the event stream", - "fields": [ - { - "name": "time", - "description": "The timestamp of the event that the log entry corresponds to", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "Float", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "tiebreaker", - "description": "The tiebreaker that disambiguates events with the same timestamp", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "Float", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - } - ], - "inputFields": null, - "interfaces": [], - "enumValues": null, - "possibleTypes": null - }, - { - "kind": "OBJECT", - "name": "InfraLogEntry", - "description": "A log entry", - "fields": [ - { - "name": "key", - "description": "A unique representation of the log entry's position in the event stream", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "OBJECT", "name": "InfraTimeKey", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "gid", - "description": "The log entry's id", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "String", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "source", - "description": "The source id", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "String", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "columns", - "description": "The columns used for rendering the log entry", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "LIST", - "name": null, - "ofType": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "UNION", "name": "InfraLogEntryColumn", "ofType": null } - } - } - }, - "isDeprecated": false, - "deprecationReason": null - } - ], - "inputFields": null, - "interfaces": [], - "enumValues": null, - "possibleTypes": null - }, - { - "kind": "UNION", - "name": "InfraLogEntryColumn", - "description": "A column of a log entry", - "fields": null, - "inputFields": null, - "interfaces": null, - "enumValues": null, - "possibleTypes": [ - { "kind": "OBJECT", "name": "InfraLogEntryTimestampColumn", "ofType": null }, - { "kind": "OBJECT", "name": "InfraLogEntryMessageColumn", "ofType": null }, - { "kind": "OBJECT", "name": "InfraLogEntryFieldColumn", "ofType": null } - ] - }, - { - "kind": "OBJECT", - "name": "InfraLogEntryTimestampColumn", - "description": "A special built-in column that contains the log entry's timestamp", - "fields": [ - { - "name": "columnId", - "description": "The id of the corresponding column configuration", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "ID", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "timestamp", - "description": "The timestamp", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "Float", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - } - ], - "inputFields": null, - "interfaces": [], - "enumValues": null, - "possibleTypes": null - }, - { - "kind": "OBJECT", - "name": "InfraLogEntryMessageColumn", - "description": "A special built-in column that contains the log entry's constructed message", - "fields": [ - { - "name": "columnId", - "description": "The id of the corresponding column configuration", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "ID", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "message", - "description": "A list of the formatted log entry segments", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "LIST", - "name": null, - "ofType": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "UNION", "name": "InfraLogMessageSegment", "ofType": null } - } - } - }, - "isDeprecated": false, - "deprecationReason": null - } - ], - "inputFields": null, - "interfaces": [], - "enumValues": null, - "possibleTypes": null - }, - { - "kind": "UNION", - "name": "InfraLogMessageSegment", - "description": "A segment of the log entry message", - "fields": null, - "inputFields": null, - "interfaces": null, - "enumValues": null, - "possibleTypes": [ - { "kind": "OBJECT", "name": "InfraLogMessageFieldSegment", "ofType": null }, - { "kind": "OBJECT", "name": "InfraLogMessageConstantSegment", "ofType": null } - ] - }, - { - "kind": "OBJECT", - "name": "InfraLogMessageFieldSegment", - "description": "A segment of the log entry message that was derived from a field", - "fields": [ - { - "name": "field", - "description": "The field the segment was derived from", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "String", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "value", - "description": "The segment's message", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "String", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "highlights", - "description": "A list of highlighted substrings of the value", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "LIST", - "name": null, - "ofType": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "String", "ofType": null } - } - } - }, - "isDeprecated": false, - "deprecationReason": null - } - ], - "inputFields": null, - "interfaces": [], - "enumValues": null, - "possibleTypes": null - }, - { - "kind": "OBJECT", - "name": "InfraLogMessageConstantSegment", - "description": "A segment of the log entry message that was derived from a string literal", - "fields": [ - { - "name": "constant", - "description": "The segment's message", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "String", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - } - ], - "inputFields": null, - "interfaces": [], - "enumValues": null, - "possibleTypes": null - }, - { - "kind": "OBJECT", - "name": "InfraLogEntryFieldColumn", - "description": "A column that contains the value of a field of the log entry", - "fields": [ - { - "name": "columnId", - "description": "The id of the corresponding column configuration", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "ID", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "field", - "description": "The field name of the column", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "String", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "value", - "description": "The value of the field in the log entry", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "String", "ofType": null } - }, - "isDeprecated": false, - "deprecationReason": null - }, - { - "name": "highlights", - "description": "A list of highlighted substrings of the value", - "args": [], - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { - "kind": "LIST", - "name": null, - "ofType": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "String", "ofType": null } - } - } - }, - "isDeprecated": false, - "deprecationReason": null - } - ], - "inputFields": null, - "interfaces": [], - "enumValues": null, - "possibleTypes": null - }, - { - "kind": "INPUT_OBJECT", - "name": "InfraLogEntryHighlightInput", - "description": "A highlighting definition", - "fields": null, - "inputFields": [ - { - "name": "query", - "description": "The query to highlight by", - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "String", "ofType": null } - }, - "defaultValue": null - }, - { - "name": "countBefore", - "description": "The number of highlighted documents to include beyond the beginning of the interval", - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "Int", "ofType": null } - }, - "defaultValue": null - }, - { - "name": "countAfter", - "description": "The number of highlighted documents to include beyond the end of the interval", - "type": { - "kind": "NON_NULL", - "name": null, - "ofType": { "kind": "SCALAR", "name": "Int", "ofType": null } - }, - "defaultValue": null - } - ], - "interfaces": null, - "enumValues": null, - "possibleTypes": null - }, { "kind": "INPUT_OBJECT", "name": "InfraTimerangeInput", diff --git a/x-pack/plugins/infra/public/graphql/types.ts b/x-pack/plugins/infra/public/graphql/types.ts index f0f74c34a19e6c..eb025ee4efd79b 100644 --- a/x-pack/plugins/infra/public/graphql/types.ts +++ b/x-pack/plugins/infra/public/graphql/types.ts @@ -30,12 +30,6 @@ export interface InfraSource { configuration: InfraSourceConfiguration; /** The status of the source */ status: InfraSourceStatus; - /** A consecutive span of log entries surrounding a point in time */ - logEntriesAround: InfraLogEntryInterval; - /** A consecutive span of log entries within an interval */ - logEntriesBetween: InfraLogEntryInterval; - /** Sequences of log entries matching sets of highlighting queries within an interval */ - logEntryHighlights: InfraLogEntryInterval[]; /** A snapshot of nodes */ snapshot?: InfraSnapshotResponse | null; @@ -135,80 +129,6 @@ export interface InfraIndexField { /** Whether the field should be displayed based on event.module and a ECS allowed list */ displayable: boolean; } -/** A consecutive sequence of log entries */ -export interface InfraLogEntryInterval { - /** The key corresponding to the start of the interval covered by the entries */ - start?: InfraTimeKey | null; - /** The key corresponding to the end of the interval covered by the entries */ - end?: InfraTimeKey | null; - /** Whether there are more log entries available before the start */ - hasMoreBefore: boolean; - /** Whether there are more log entries available after the end */ - hasMoreAfter: boolean; - /** The query the log entries were filtered by */ - filterQuery?: string | null; - /** The query the log entries were highlighted with */ - highlightQuery?: string | null; - /** A list of the log entries */ - entries: InfraLogEntry[]; -} -/** A representation of the log entry's position in the event stream */ -export interface InfraTimeKey { - /** The timestamp of the event that the log entry corresponds to */ - time: number; - /** The tiebreaker that disambiguates events with the same timestamp */ - tiebreaker: number; -} -/** A log entry */ -export interface InfraLogEntry { - /** A unique representation of the log entry's position in the event stream */ - key: InfraTimeKey; - /** The log entry's id */ - gid: string; - /** The source id */ - source: string; - /** The columns used for rendering the log entry */ - columns: InfraLogEntryColumn[]; -} -/** A special built-in column that contains the log entry's timestamp */ -export interface InfraLogEntryTimestampColumn { - /** The id of the corresponding column configuration */ - columnId: string; - /** The timestamp */ - timestamp: number; -} -/** A special built-in column that contains the log entry's constructed message */ -export interface InfraLogEntryMessageColumn { - /** The id of the corresponding column configuration */ - columnId: string; - /** A list of the formatted log entry segments */ - message: InfraLogMessageSegment[]; -} -/** A segment of the log entry message that was derived from a field */ -export interface InfraLogMessageFieldSegment { - /** The field the segment was derived from */ - field: string; - /** The segment's message */ - value: string; - /** A list of highlighted substrings of the value */ - highlights: string[]; -} -/** A segment of the log entry message that was derived from a string literal */ -export interface InfraLogMessageConstantSegment { - /** The segment's message */ - constant: string; -} -/** A column that contains the value of a field of the log entry */ -export interface InfraLogEntryFieldColumn { - /** The id of the corresponding column configuration */ - columnId: string; - /** The field name of the column */ - field: string; - /** The value of the field in the log entry */ - value: string; - /** A list of highlighted substrings of the value */ - highlights: string[]; -} export interface InfraSnapshotResponse { /** Nodes of type host, container or pod grouped by 0, 1 or 2 terms */ @@ -282,21 +202,6 @@ export interface DeleteSourceResult { // InputTypes // ==================================================== -export interface InfraTimeKeyInput { - time: number; - - tiebreaker: number; -} -/** A highlighting definition */ -export interface InfraLogEntryHighlightInput { - /** The query to highlight by */ - query: string; - /** The number of highlighted documents to include beyond the beginning of the interval */ - countBefore: number; - /** The number of highlighted documents to include beyond the end of the interval */ - countAfter: number; -} - export interface InfraTimerangeInput { /** The interval string to use for last bucket. The format is '{value}{unit}'. For example '5m' would return the metrics for the last 5 minutes of the timespan. */ interval: string; @@ -387,34 +292,6 @@ export interface SourceQueryArgs { /** The id of the source */ id: string; } -export interface LogEntriesAroundInfraSourceArgs { - /** The sort key that corresponds to the point in time */ - key: InfraTimeKeyInput; - /** The maximum number of preceding to return */ - countBefore?: number | null; - /** The maximum number of following to return */ - countAfter?: number | null; - /** The query to filter the log entries by */ - filterQuery?: string | null; -} -export interface LogEntriesBetweenInfraSourceArgs { - /** The sort key that corresponds to the start of the interval */ - startKey: InfraTimeKeyInput; - /** The sort key that corresponds to the end of the interval */ - endKey: InfraTimeKeyInput; - /** The query to filter the log entries by */ - filterQuery?: string | null; -} -export interface LogEntryHighlightsInfraSourceArgs { - /** The sort key that corresponds to the start of the interval */ - startKey: InfraTimeKeyInput; - /** The sort key that corresponds to the end of the interval */ - endKey: InfraTimeKeyInput; - /** The query to filter the log entries by */ - filterQuery?: string | null; - /** The highlighting to apply to the log entries */ - highlights: InfraLogEntryHighlightInput[]; -} export interface SnapshotInfraSourceArgs { timerange: InfraTimerangeInput; @@ -571,15 +448,6 @@ export type InfraSourceLogColumn = | InfraSourceMessageLogColumn | InfraSourceFieldLogColumn; -/** A column of a log entry */ -export type InfraLogEntryColumn = - | InfraLogEntryTimestampColumn - | InfraLogEntryMessageColumn - | InfraLogEntryFieldColumn; - -/** A segment of the log entry message */ -export type InfraLogMessageSegment = InfraLogMessageFieldSegment | InfraLogMessageConstantSegment; - // ==================================================== // END: Typescript template // ==================================================== @@ -588,46 +456,6 @@ export type InfraLogMessageSegment = InfraLogMessageFieldSegment | InfraLogMessa // Documents // ==================================================== -export namespace LogEntryHighlightsQuery { - export type Variables = { - sourceId?: string | null; - startKey: InfraTimeKeyInput; - endKey: InfraTimeKeyInput; - filterQuery?: string | null; - highlights: InfraLogEntryHighlightInput[]; - }; - - export type Query = { - __typename?: 'Query'; - - source: Source; - }; - - export type Source = { - __typename?: 'InfraSource'; - - id: string; - - logEntryHighlights: LogEntryHighlights[]; - }; - - export type LogEntryHighlights = { - __typename?: 'InfraLogEntryInterval'; - - start?: Start | null; - - end?: End | null; - - entries: Entries[]; - }; - - export type Start = InfraTimeKeyFields.Fragment; - - export type End = InfraTimeKeyFields.Fragment; - - export type Entries = InfraLogEntryHighlightFields.Fragment; -} - export namespace MetricsQuery { export type Variables = { sourceId: string; @@ -826,50 +654,6 @@ export namespace WaffleNodesQuery { }; } -export namespace LogEntries { - export type Variables = { - sourceId?: string | null; - timeKey: InfraTimeKeyInput; - countBefore?: number | null; - countAfter?: number | null; - filterQuery?: string | null; - }; - - export type Query = { - __typename?: 'Query'; - - source: Source; - }; - - export type Source = { - __typename?: 'InfraSource'; - - id: string; - - logEntriesAround: LogEntriesAround; - }; - - export type LogEntriesAround = { - __typename?: 'InfraLogEntryInterval'; - - start?: Start | null; - - end?: End | null; - - hasMoreBefore: boolean; - - hasMoreAfter: boolean; - - entries: Entries[]; - }; - - export type Start = InfraTimeKeyFields.Fragment; - - export type End = InfraTimeKeyFields.Fragment; - - export type Entries = InfraLogEntryFields.Fragment; -} - export namespace SourceConfigurationFields { export type Fragment = { __typename?: 'InfraSourceConfiguration'; @@ -1000,124 +784,3 @@ export namespace InfraSourceFields { origin: string; }; } - -export namespace InfraLogEntryFields { - export type Fragment = { - __typename?: 'InfraLogEntry'; - - gid: string; - - key: Key; - - columns: Columns[]; - }; - - export type Key = { - __typename?: 'InfraTimeKey'; - - time: number; - - tiebreaker: number; - }; - - export type Columns = - | InfraLogEntryTimestampColumnInlineFragment - | InfraLogEntryMessageColumnInlineFragment - | InfraLogEntryFieldColumnInlineFragment; - - export type InfraLogEntryTimestampColumnInlineFragment = { - __typename?: 'InfraLogEntryTimestampColumn'; - - columnId: string; - - timestamp: number; - }; - - export type InfraLogEntryMessageColumnInlineFragment = { - __typename?: 'InfraLogEntryMessageColumn'; - - columnId: string; - - message: Message[]; - }; - - export type Message = - | InfraLogMessageFieldSegmentInlineFragment - | InfraLogMessageConstantSegmentInlineFragment; - - export type InfraLogMessageFieldSegmentInlineFragment = { - __typename?: 'InfraLogMessageFieldSegment'; - - field: string; - - value: string; - }; - - export type InfraLogMessageConstantSegmentInlineFragment = { - __typename?: 'InfraLogMessageConstantSegment'; - - constant: string; - }; - - export type InfraLogEntryFieldColumnInlineFragment = { - __typename?: 'InfraLogEntryFieldColumn'; - - columnId: string; - - field: string; - - value: string; - }; -} - -export namespace InfraLogEntryHighlightFields { - export type Fragment = { - __typename?: 'InfraLogEntry'; - - gid: string; - - key: Key; - - columns: Columns[]; - }; - - export type Key = { - __typename?: 'InfraTimeKey'; - - time: number; - - tiebreaker: number; - }; - - export type Columns = - | InfraLogEntryMessageColumnInlineFragment - | InfraLogEntryFieldColumnInlineFragment; - - export type InfraLogEntryMessageColumnInlineFragment = { - __typename?: 'InfraLogEntryMessageColumn'; - - columnId: string; - - message: Message[]; - }; - - export type Message = InfraLogMessageFieldSegmentInlineFragment; - - export type InfraLogMessageFieldSegmentInlineFragment = { - __typename?: 'InfraLogMessageFieldSegment'; - - field: string; - - highlights: string[]; - }; - - export type InfraLogEntryFieldColumnInlineFragment = { - __typename?: 'InfraLogEntryFieldColumn'; - - columnId: string; - - field: string; - - highlights: string[]; - }; -} diff --git a/x-pack/plugins/infra/public/pages/logs/log_entry_categories/sections/top_categories/category_example_message.tsx b/x-pack/plugins/infra/public/pages/logs/log_entry_categories/sections/top_categories/category_example_message.tsx index e24fdd06bc6d9b..83659ace3ce548 100644 --- a/x-pack/plugins/infra/public/pages/logs/log_entry_categories/sections/top_categories/category_example_message.tsx +++ b/x-pack/plugins/infra/public/pages/logs/log_entry_categories/sections/top_categories/category_example_message.tsx @@ -127,6 +127,7 @@ export const CategoryExampleMessage: React.FunctionComponent<{ onClick: () => { const logEntry: LogEntry = { id, + index: '', // TODO: use real index when loading via async search context, cursor: { time: timestamp, tiebreaker }, columns: [], diff --git a/x-pack/plugins/infra/public/test_utils/entries.ts b/x-pack/plugins/infra/public/test_utils/entries.ts index 96737fb1753650..1633b9d8dc0760 100644 --- a/x-pack/plugins/infra/public/test_utils/entries.ts +++ b/x-pack/plugins/infra/public/test_utils/entries.ts @@ -28,6 +28,7 @@ export function generateFakeEntries( const timestamp = i === count - 1 ? endTimestamp : startTimestamp + timestampStep * i; entries.push({ id: `entry-${i}`, + index: 'logs-fake', context: {}, cursor: { time: timestamp, tiebreaker: i }, columns: columns.map((column) => { diff --git a/x-pack/plugins/infra/public/utils/data_search/data_search.stories.mdx b/x-pack/plugins/infra/public/utils/data_search/data_search.stories.mdx index a698b806b4cd70..a8854692caa36c 100644 --- a/x-pack/plugins/infra/public/utils/data_search/data_search.stories.mdx +++ b/x-pack/plugins/infra/public/utils/data_search/data_search.stories.mdx @@ -43,15 +43,35 @@ be issued by calling the returned `search()` function. For each new request the hook emits an object describing the request and its state in the `requests$` `Observable`. +Since the specific response shape depends on the data strategy used, the hook +takes a projection function, that is responsible for decoding the response in +an appropriate way. Because most response projections follow a similar pattern +there's a helper `normalizeDataSearchResponses(initialResponse, +parseRawResponse)`, which generates an RxJS operator, that... + +- emits an initial response containing the given `initialResponse` value +- applies `parseRawResponse` to the `rawResponse` property of each emitted response +- transforms transport layer errors as well as parsing errors into + `SearchStrategyError`s + ```typescript +const parseMyCustomSearchResponse = normalizeDataSearchResponses( + 'initial value', + decodeOrThrow(myCustomSearchResponsePayloadRT) +); + const { search, requests$ } = useDataSearch({ getRequest: useCallback((searchTerm: string) => ({ request: { params: { searchTerm - } - } - }), []); + }, + options: { + strategy: 'my-custom-search-strategy', + }, + }, + }), []), + parseResponses: parseMyCustomSearchResponse, }); ``` @@ -68,10 +88,6 @@ observables are unsubscribed from for proper cancellation if a new request has been created. This uses RxJS's `switchMap()` operator under the hood. The hook also makes sure that all observables are unsubscribed from on unmount. -Since the specific response shape depends on the data strategy used, the hook -takes a projection function, that is responsible for decoding the response in -an appropriate way. - A request can fail due to various reasons that include servers-side errors, Elasticsearch shard failures and network failures. The intention is to map all of them to a common `SearchStrategyError` interface. While the @@ -94,11 +110,7 @@ const { latestResponseErrors, loaded, total, -} = useLatestPartialDataSearchResponse( - requests$, - 'initialValue', - useMemo(() => decodeOrThrow(mySearchStrategyResponsePayloadRT), []), -); +} = useLatestPartialDataSearchResponse(requests$); ``` ## Representing the request state to the user diff --git a/x-pack/plugins/infra/public/utils/data_search/flatten_data_search_response.ts b/x-pack/plugins/infra/public/utils/data_search/flatten_data_search_response.ts new file mode 100644 index 00000000000000..98df6d441bd806 --- /dev/null +++ b/x-pack/plugins/infra/public/utils/data_search/flatten_data_search_response.ts @@ -0,0 +1,29 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { map } from 'rxjs/operators'; +import { IKibanaSearchRequest } from '../../../../../../src/plugins/data/public'; +import { ParsedDataSearchRequestDescriptor } from './types'; + +export const flattenDataSearchResponseDescriptor = < + Request extends IKibanaSearchRequest, + Response +>({ + abortController, + options, + request, + response$, +}: ParsedDataSearchRequestDescriptor) => + response$.pipe( + map((response) => { + return { + abortController, + options, + request, + response, + }; + }) + ); diff --git a/x-pack/plugins/infra/public/utils/data_search/index.ts b/x-pack/plugins/infra/public/utils/data_search/index.ts index c08ab0727fd904..10beba4aa4fdcd 100644 --- a/x-pack/plugins/infra/public/utils/data_search/index.ts +++ b/x-pack/plugins/infra/public/utils/data_search/index.ts @@ -4,6 +4,9 @@ * you may not use this file except in compliance with the Elastic License. */ +export * from './flatten_data_search_response'; +export * from './normalize_data_search_responses'; export * from './types'; export * from './use_data_search_request'; +export * from './use_data_search_response_state'; export * from './use_latest_partial_data_search_response'; diff --git a/x-pack/plugins/infra/public/utils/data_search/normalize_data_search_responses.ts b/x-pack/plugins/infra/public/utils/data_search/normalize_data_search_responses.ts new file mode 100644 index 00000000000000..5046cc128a835a --- /dev/null +++ b/x-pack/plugins/infra/public/utils/data_search/normalize_data_search_responses.ts @@ -0,0 +1,77 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { Observable, of } from 'rxjs'; +import { catchError, map, startWith } from 'rxjs/operators'; +import { IKibanaSearchResponse } from '../../../../../../src/plugins/data/public'; +import { AbortError } from '../../../../../../src/plugins/kibana_utils/public'; +import { SearchStrategyError } from '../../../common/search_strategies/common/errors'; +import { ParsedKibanaSearchResponse } from './types'; + +export type RawResponseParser = ( + rawResponse: RawResponse +) => { data: Response; errors?: SearchStrategyError[] }; + +/** + * An operator factory that normalizes each {@link IKibanaSearchResponse} by + * parsing it into a {@link ParsedKibanaSearchResponse} and adding initial + * responses and error handling. + * + * @param initialResponse - The initial value to emit when a new request is + * handled. + * @param projectResponse - The projection function to apply to each response + * payload. It should validate that the response payload is of the type {@link + * RawResponse} and decode it to a {@link Response}. + * + * @return An operator that adds parsing and error handling transformations to + * each response payload using the arguments given above. + */ +export const normalizeDataSearchResponses = ( + initialResponse: InitialResponse, + parseRawResponse: RawResponseParser +) => ( + response$: Observable> +): Observable> => + response$.pipe( + map((response) => { + const { data, errors = [] } = parseRawResponse(response.rawResponse); + return { + data, + errors, + isPartial: response.isPartial ?? false, + isRunning: response.isRunning ?? false, + loaded: response.loaded, + total: response.total, + }; + }), + startWith({ + data: initialResponse, + errors: [], + isPartial: true, + isRunning: true, + loaded: 0, + total: undefined, + }), + catchError((error) => + of({ + data: initialResponse, + errors: [ + error instanceof AbortError + ? { + type: 'aborted' as const, + } + : { + type: 'generic' as const, + message: `${error.message ?? error}`, + }, + ], + isPartial: true, + isRunning: false, + loaded: 0, + total: undefined, + }) + ) + ); diff --git a/x-pack/plugins/infra/public/utils/data_search/types.ts b/x-pack/plugins/infra/public/utils/data_search/types.ts index ba0a4c639dae4a..4fcb5898ea5bd8 100644 --- a/x-pack/plugins/infra/public/utils/data_search/types.ts +++ b/x-pack/plugins/infra/public/utils/data_search/types.ts @@ -19,7 +19,17 @@ export interface DataSearchRequestDescriptor { +export interface ParsedDataSearchRequestDescriptor< + Request extends IKibanaSearchRequest, + ResponseData +> { + request: Request; + options: ISearchOptions; + response$: Observable>; + abortController: AbortController; +} + +export interface ParsedKibanaSearchResponse { total?: number; loaded?: number; isRunning: boolean; @@ -28,9 +38,12 @@ export interface NormalizedKibanaSearchResponse { errors: SearchStrategyError[]; } -export interface DataSearchResponseDescriptor { +export interface ParsedDataSearchResponseDescriptor< + Request extends IKibanaSearchRequest, + Response +> { request: Request; options: ISearchOptions; - response: NormalizedKibanaSearchResponse; + response: ParsedKibanaSearchResponse; abortController: AbortController; } diff --git a/x-pack/plugins/infra/public/utils/data_search/use_data_search_request.test.tsx b/x-pack/plugins/infra/public/utils/data_search/use_data_search_request.test.tsx index 87c091f12ad90a..780476abb7b1bf 100644 --- a/x-pack/plugins/infra/public/utils/data_search/use_data_search_request.test.tsx +++ b/x-pack/plugins/infra/public/utils/data_search/use_data_search_request.test.tsx @@ -17,6 +17,7 @@ import { import { dataPluginMock } from '../../../../../../src/plugins/data/public/mocks'; import { createKibanaReactContext } from '../../../../../../src/plugins/kibana_react/public'; import { PluginKibanaContextValue } from '../../hooks/use_kibana'; +import { normalizeDataSearchResponses } from './normalize_data_search_responses'; import { useDataSearch } from './use_data_search_request'; describe('useDataSearch hook', () => { @@ -34,6 +35,7 @@ describe('useDataSearch hook', () => { () => useDataSearch({ getRequest, + parseResponses: noopParseResponse, }), { wrapper: ({ children }) => {children}, @@ -48,7 +50,7 @@ describe('useDataSearch hook', () => { expect(dataMock.search.search).not.toHaveBeenCalled(); }); - it('creates search requests with the given params and options', async () => { + it('creates search requests with the given params and options and parses the responses', async () => { const dataMock = createDataPluginMock(); const searchResponseMock$ = of({ rawResponse: { @@ -78,6 +80,7 @@ describe('useDataSearch hook', () => { () => useDataSearch({ getRequest, + parseResponses: noopParseResponse, }), { wrapper: ({ children }) => {children}, @@ -112,10 +115,11 @@ describe('useDataSearch hook', () => { }); expect(firstRequest).toHaveProperty('options.strategy', 'test-search-strategy'); expect(firstRequest).toHaveProperty('response$', expect.any(Observable)); - await expect(firstRequest.response$.toPromise()).resolves.toEqual({ - rawResponse: { - firstKey: 'firstValue', + await expect(firstRequest.response$.toPromise()).resolves.toMatchObject({ + data: { + firstKey: 'firstValue', // because this specific response parser just copies the raw response }, + errors: [], }); }); @@ -145,6 +149,7 @@ describe('useDataSearch hook', () => { () => useDataSearch({ getRequest, + parseResponses: noopParseResponse, }), { wrapper: ({ children }) => {children}, @@ -186,3 +191,8 @@ const createDataPluginMock = () => { }; return dataMock; }; + +const noopParseResponse = normalizeDataSearchResponses( + null, + (response: Response) => ({ data: response }) +); diff --git a/x-pack/plugins/infra/public/utils/data_search/use_data_search_request.ts b/x-pack/plugins/infra/public/utils/data_search/use_data_search_request.ts index a23f06adc0353c..0f1686a93be829 100644 --- a/x-pack/plugins/infra/public/utils/data_search/use_data_search_request.ts +++ b/x-pack/plugins/infra/public/utils/data_search/use_data_search_request.ts @@ -5,8 +5,8 @@ */ import { useCallback } from 'react'; -import { Subject } from 'rxjs'; -import { map, share, switchMap, tap } from 'rxjs/operators'; +import { OperatorFunction, Subject } from 'rxjs'; +import { share, tap } from 'rxjs/operators'; import { IKibanaSearchRequest, IKibanaSearchResponse, @@ -14,6 +14,7 @@ import { } from '../../../../../../src/plugins/data/public'; import { useKibanaContextForPlugin } from '../../hooks/use_kibana'; import { tapUnsubscribe, useObservable } from '../use_observable'; +import { ParsedDataSearchRequestDescriptor, ParsedKibanaSearchResponse } from './types'; export type DataSearchRequestFactory = ( ...args: Args @@ -25,69 +26,74 @@ export type DataSearchRequestFactory = OperatorFunction< + IKibanaSearchResponse, + ParsedKibanaSearchResponse +>; + export const useDataSearch = < RequestFactoryArgs extends any[], - Request extends IKibanaSearchRequest, - RawResponse + RequestParams, + Request extends IKibanaSearchRequest, + RawResponse, + Response >({ getRequest, + parseResponses, }: { getRequest: DataSearchRequestFactory; + parseResponses: ParseResponsesOperator; }) => { const { services } = useKibanaContextForPlugin(); - const request$ = useObservable( - () => new Subject<{ request: Request; options: ISearchOptions }>(), - [] - ); const requests$ = useObservable( - (inputs$) => - inputs$.pipe( - switchMap(([currentRequest$]) => currentRequest$), - map(({ request, options }) => { - const abortController = new AbortController(); - let isAbortable = true; - - return { - abortController, - request, - options, - response$: services.data.search - .search>(request, { - abortSignal: abortController.signal, - ...options, - }) - .pipe( - // avoid aborting failed or completed requests - tap({ - error: () => { - isAbortable = false; - }, - complete: () => { - isAbortable = false; - }, - }), - tapUnsubscribe(() => { - if (isAbortable) { - abortController.abort(); - } - }), - share() - ), - }; - }) - ), - [request$] + () => new Subject>(), + [] ); const search = useCallback( (...args: RequestFactoryArgs) => { - const request = getRequest(...args); + const requestArgs = getRequest(...args); - if (request) { - request$.next(request); + if (requestArgs == null) { + return; } + + const abortController = new AbortController(); + let isAbortable = true; + + const newRequestDescriptor = { + ...requestArgs, + abortController, + response$: services.data.search + .search>(requestArgs.request, { + abortSignal: abortController.signal, + ...requestArgs.options, + }) + .pipe( + // avoid aborting failed or completed requests + tap({ + error: () => { + isAbortable = false; + }, + complete: () => { + isAbortable = false; + }, + }), + tapUnsubscribe(() => { + if (isAbortable) { + abortController.abort(); + } + }), + parseResponses, + share() + ), + }; + + requests$.next(newRequestDescriptor); + + return newRequestDescriptor; }, - [getRequest, request$] + [getRequest, services.data.search, parseResponses, requests$] ); return { diff --git a/x-pack/plugins/infra/public/utils/data_search/use_data_search_response_state.ts b/x-pack/plugins/infra/public/utils/data_search/use_data_search_response_state.ts new file mode 100644 index 00000000000000..3b37b80f26cdc2 --- /dev/null +++ b/x-pack/plugins/infra/public/utils/data_search/use_data_search_response_state.ts @@ -0,0 +1,35 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { useCallback } from 'react'; +import { Observable } from 'rxjs'; +import { IKibanaSearchRequest } from '../../../../../../src/plugins/data/public'; +import { useObservableState } from '../use_observable'; +import { ParsedDataSearchResponseDescriptor } from './types'; + +export const useDataSearchResponseState = < + Request extends IKibanaSearchRequest, + Response, + InitialResponse +>( + response$: Observable> +) => { + const { latestValue } = useObservableState(response$, undefined); + + const cancelRequest = useCallback(() => { + latestValue?.abortController.abort(); + }, [latestValue]); + + return { + cancelRequest, + isRequestRunning: latestValue?.response.isRunning ?? false, + isResponsePartial: latestValue?.response.isPartial ?? false, + latestResponseData: latestValue?.response.data, + latestResponseErrors: latestValue?.response.errors, + loaded: latestValue?.response.loaded, + total: latestValue?.response.total, + }; +}; diff --git a/x-pack/plugins/infra/public/utils/data_search/use_latest_partial_data_search_response.test.tsx b/x-pack/plugins/infra/public/utils/data_search/use_latest_partial_data_search_response.test.tsx index 4c336aa1107a22..864d92f43bc17e 100644 --- a/x-pack/plugins/infra/public/utils/data_search/use_latest_partial_data_search_response.test.tsx +++ b/x-pack/plugins/infra/public/utils/data_search/use_latest_partial_data_search_response.test.tsx @@ -5,12 +5,9 @@ */ import { act, renderHook } from '@testing-library/react-hooks'; -import { Observable, of, Subject } from 'rxjs'; -import { - IKibanaSearchRequest, - IKibanaSearchResponse, -} from '../../../../../../src/plugins/data/public'; -import { DataSearchRequestDescriptor } from './types'; +import { BehaviorSubject, Observable, of, Subject } from 'rxjs'; +import { IKibanaSearchRequest } from '../../../../../../src/plugins/data/public'; +import { ParsedDataSearchRequestDescriptor, ParsedKibanaSearchResponse } from './types'; import { useLatestPartialDataSearchResponse } from './use_latest_partial_data_search_response'; describe('useLatestPartialDataSearchResponse hook', () => { @@ -19,25 +16,31 @@ describe('useLatestPartialDataSearchResponse hook', () => { abortController: new AbortController(), options: {}, request: { params: 'firstRequestParam' }, - response$: new Subject>(), + response$: new BehaviorSubject>({ + data: 'initial', + isRunning: true, + isPartial: true, + errors: [], + }), }; const secondRequest = { abortController: new AbortController(), options: {}, request: { params: 'secondRequestParam' }, - response$: new Subject>(), + response$: new BehaviorSubject>({ + data: 'initial', + isRunning: true, + isPartial: true, + errors: [], + }), }; const requests$ = new Subject< - DataSearchRequestDescriptor, string> + ParsedDataSearchRequestDescriptor, string> >(); - const { result } = renderHook(() => - useLatestPartialDataSearchResponse(requests$, 'initial', (response) => ({ - data: `projection of ${response}`, - })) - ); + const { result } = renderHook(() => useLatestPartialDataSearchResponse(requests$)); expect(result).toHaveProperty('current.isRequestRunning', false); expect(result).toHaveProperty('current.latestResponseData', undefined); @@ -52,37 +55,43 @@ describe('useLatestPartialDataSearchResponse hook', () => { // first response of the first request arrives act(() => { - firstRequest.response$.next({ rawResponse: 'request-1-response-1', isRunning: true }); + firstRequest.response$.next({ + data: 'request-1-response-1', + isRunning: true, + isPartial: true, + errors: [], + }); }); expect(result).toHaveProperty('current.isRequestRunning', true); - expect(result).toHaveProperty( - 'current.latestResponseData', - 'projection of request-1-response-1' - ); + expect(result).toHaveProperty('current.latestResponseData', 'request-1-response-1'); // second request is started before the second response of the first request arrives act(() => { requests$.next(secondRequest); - secondRequest.response$.next({ rawResponse: 'request-2-response-1', isRunning: true }); + secondRequest.response$.next({ + data: 'request-2-response-1', + isRunning: true, + isPartial: true, + errors: [], + }); }); expect(result).toHaveProperty('current.isRequestRunning', true); - expect(result).toHaveProperty( - 'current.latestResponseData', - 'projection of request-2-response-1' - ); + expect(result).toHaveProperty('current.latestResponseData', 'request-2-response-1'); // second response of the second request arrives act(() => { - secondRequest.response$.next({ rawResponse: 'request-2-response-2', isRunning: false }); + secondRequest.response$.next({ + data: 'request-2-response-2', + isRunning: false, + isPartial: false, + errors: [], + }); }); expect(result).toHaveProperty('current.isRequestRunning', false); - expect(result).toHaveProperty( - 'current.latestResponseData', - 'projection of request-2-response-2' - ); + expect(result).toHaveProperty('current.latestResponseData', 'request-2-response-2'); }); it("unsubscribes from the latest request's response observable on unmount", () => { @@ -92,20 +101,16 @@ describe('useLatestPartialDataSearchResponse hook', () => { abortController: new AbortController(), options: {}, request: { params: 'firstRequestParam' }, - response$: new Observable>(() => { + response$: new Observable>(() => { return onUnsubscribe; }), }; - const requests$ = of, string>>( + const requests$ = of, string>>( firstRequest ); - const { unmount } = renderHook(() => - useLatestPartialDataSearchResponse(requests$, 'initial', (response) => ({ - data: `projection of ${response}`, - })) - ); + const { unmount } = renderHook(() => useLatestPartialDataSearchResponse(requests$)); expect(onUnsubscribe).not.toHaveBeenCalled(); diff --git a/x-pack/plugins/infra/public/utils/data_search/use_latest_partial_data_search_response.ts b/x-pack/plugins/infra/public/utils/data_search/use_latest_partial_data_search_response.ts index 71fd96283d0ef9..9366df8adbaf71 100644 --- a/x-pack/plugins/infra/public/utils/data_search/use_latest_partial_data_search_response.ts +++ b/x-pack/plugins/infra/public/utils/data_search/use_latest_partial_data_search_response.ts @@ -4,111 +4,40 @@ * you may not use this file except in compliance with the Elastic License. */ -import { useCallback } from 'react'; -import { Observable, of } from 'rxjs'; -import { catchError, map, startWith, switchMap } from 'rxjs/operators'; +import { Observable } from 'rxjs'; +import { switchMap } from 'rxjs/operators'; import { IKibanaSearchRequest } from '../../../../../../src/plugins/data/public'; -import { AbortError } from '../../../../../../src/plugins/kibana_utils/public'; -import { SearchStrategyError } from '../../../common/search_strategies/common/errors'; -import { useLatest, useObservable, useObservableState } from '../use_observable'; -import { DataSearchRequestDescriptor, DataSearchResponseDescriptor } from './types'; +import { useOperator } from '../use_observable'; +import { flattenDataSearchResponseDescriptor } from './flatten_data_search_response'; +import { ParsedDataSearchRequestDescriptor, ParsedDataSearchResponseDescriptor } from './types'; +import { useDataSearchResponseState } from './use_data_search_response_state'; -export const useLatestPartialDataSearchResponse = < - Request extends IKibanaSearchRequest, - RawResponse, - Response, - InitialResponse ->( - requests$: Observable>, - initialResponse: InitialResponse, - projectResponse: (rawResponse: RawResponse) => { data: Response; errors?: SearchStrategyError[] } +export const useLatestPartialDataSearchResponse = ( + requests$: Observable> ) => { - const latestInitialResponse = useLatest(initialResponse); - const latestProjectResponse = useLatest(projectResponse); - const latestResponse$: Observable< - DataSearchResponseDescriptor - > = useObservable( - (inputs$) => - inputs$.pipe( - switchMap(([currentRequests$]) => - currentRequests$.pipe( - switchMap(({ abortController, options, request, response$ }) => - response$.pipe( - map((response) => { - const { data, errors = [] } = latestProjectResponse.current(response.rawResponse); - return { - abortController, - options, - request, - response: { - data, - errors, - isPartial: response.isPartial ?? false, - isRunning: response.isRunning ?? false, - loaded: response.loaded, - total: response.total, - }, - }; - }), - startWith({ - abortController, - options, - request, - response: { - data: latestInitialResponse.current, - errors: [], - isPartial: true, - isRunning: true, - loaded: 0, - total: undefined, - }, - }), - catchError((error) => - of({ - abortController, - options, - request, - response: { - data: latestInitialResponse.current, - errors: [ - error instanceof AbortError - ? { - type: 'aborted' as const, - } - : { - type: 'generic' as const, - message: `${error.message ?? error}`, - }, - ], - isPartial: true, - isRunning: false, - loaded: 0, - total: undefined, - }, - }) - ) - ) - ) - ) - ) - ), - [requests$] as const - ); - - const { latestValue } = useObservableState(latestResponse$, undefined); + ParsedDataSearchResponseDescriptor + > = useOperator(requests$, flattenLatestDataSearchResponse); - const cancelRequest = useCallback(() => { - latestValue?.abortController.abort(); - }, [latestValue]); + const { + cancelRequest, + isRequestRunning, + isResponsePartial, + latestResponseData, + latestResponseErrors, + loaded, + total, + } = useDataSearchResponseState(latestResponse$); return { cancelRequest, - isRequestRunning: latestValue?.response.isRunning ?? false, - isResponsePartial: latestValue?.response.isPartial ?? false, - latestResponseData: latestValue?.response.data, - latestResponseErrors: latestValue?.response.errors, - loaded: latestValue?.response.loaded, - total: latestValue?.response.total, + isRequestRunning, + isResponsePartial, + latestResponseData, + latestResponseErrors, + loaded, + total, }; }; + +const flattenLatestDataSearchResponse = switchMap(flattenDataSearchResponseDescriptor); diff --git a/x-pack/plugins/infra/public/utils/log_entry/log_entry.ts b/x-pack/plugins/infra/public/utils/log_entry/log_entry.ts index c69104ad6177e7..60034aea6be639 100644 --- a/x-pack/plugins/infra/public/utils/log_entry/log_entry.ts +++ b/x-pack/plugins/infra/public/utils/log_entry/log_entry.ts @@ -5,9 +5,7 @@ */ import { bisector } from 'd3-array'; - import { compareToTimeKey, getIndexAtTimeKey, TimeKey, UniqueTimeKey } from '../../../common/time'; -import { InfraLogEntryFields } from '../../graphql/types'; import { LogEntry, LogColumn, @@ -19,10 +17,6 @@ import { LogMessageConstantPart, } from '../../../common/log_entry'; -export type LogEntryMessageSegment = InfraLogEntryFields.Message; -export type LogEntryConstantMessageSegment = InfraLogEntryFields.InfraLogMessageConstantSegmentInlineFragment; -export type LogEntryFieldMessageSegment = InfraLogEntryFields.InfraLogMessageFieldSegmentInlineFragment; - export const getLogEntryKey = (entry: { cursor: TimeKey }) => entry.cursor; export const getUniqueLogEntryKey = (entry: { id: string; cursor: TimeKey }): UniqueTimeKey => ({ diff --git a/x-pack/plugins/infra/public/utils/log_entry/log_entry_highlight.ts b/x-pack/plugins/infra/public/utils/log_entry/log_entry_highlight.ts index 208316c693d4db..e14d938c426f90 100644 --- a/x-pack/plugins/infra/public/utils/log_entry/log_entry_highlight.ts +++ b/x-pack/plugins/infra/public/utils/log_entry/log_entry_highlight.ts @@ -4,7 +4,6 @@ * you may not use this file except in compliance with the Elastic License. */ -import { InfraLogEntryHighlightFields } from '../../graphql/types'; import { LogEntry, LogColumn, @@ -14,13 +13,6 @@ import { LogMessageFieldPart, } from '../../../common/log_entry'; -export type LogEntryHighlightColumn = InfraLogEntryHighlightFields.Columns; -export type LogEntryHighlightMessageColumn = InfraLogEntryHighlightFields.InfraLogEntryMessageColumnInlineFragment; -export type LogEntryHighlightFieldColumn = InfraLogEntryHighlightFields.InfraLogEntryFieldColumnInlineFragment; - -export type LogEntryHighlightMessageSegment = InfraLogEntryHighlightFields.Message | {}; -export type LogEntryHighlightFieldMessageSegment = InfraLogEntryHighlightFields.InfraLogMessageFieldSegmentInlineFragment; - export interface LogEntryHighlightsMap { [entryId: string]: LogEntry[]; } diff --git a/x-pack/plugins/infra/public/utils/use_observable.ts b/x-pack/plugins/infra/public/utils/use_observable.ts index 342aa5aa797b13..508684f8d72686 100644 --- a/x-pack/plugins/infra/public/utils/use_observable.ts +++ b/x-pack/plugins/infra/public/utils/use_observable.ts @@ -5,7 +5,8 @@ */ import { useEffect, useRef, useState } from 'react'; -import { BehaviorSubject, Observable, PartialObserver, Subscription } from 'rxjs'; +import { BehaviorSubject, Observable, OperatorFunction, PartialObserver, Subscription } from 'rxjs'; +import { switchMap } from 'rxjs/operators'; export const useLatest = (value: Value) => { const valueRef = useRef(value); @@ -62,7 +63,9 @@ export const useSubscription = ( const fixedUnsubscribe = latestUnsubscribe.current; const subscription = input$.subscribe({ - next: (value) => latestNext.current?.(value), + next: (value) => { + return latestNext.current?.(value); + }, error: (value) => latestError.current?.(value), complete: () => latestComplete.current?.(), }); @@ -78,6 +81,19 @@ export const useSubscription = ( return latestSubscription.current; }; +export const useOperator = ( + input$: Observable, + operator: OperatorFunction +) => { + const latestOperator = useLatest(operator); + + return useObservable( + (inputs$) => + inputs$.pipe(switchMap(([currentInput$]) => latestOperator.current(currentInput$))), + [input$] as const + ); +}; + export const tapUnsubscribe = (onUnsubscribe: () => void) => (source$: Observable) => { return new Observable((subscriber) => { const subscription = source$.subscribe({ diff --git a/x-pack/plugins/infra/server/graphql/types.ts b/x-pack/plugins/infra/server/graphql/types.ts index 02dcd76e8b34c8..712438ce2bfe02 100644 --- a/x-pack/plugins/infra/server/graphql/types.ts +++ b/x-pack/plugins/infra/server/graphql/types.ts @@ -56,12 +56,6 @@ export interface InfraSource { configuration: InfraSourceConfiguration; /** The status of the source */ status: InfraSourceStatus; - /** A consecutive span of log entries surrounding a point in time */ - logEntriesAround: InfraLogEntryInterval; - /** A consecutive span of log entries within an interval */ - logEntriesBetween: InfraLogEntryInterval; - /** Sequences of log entries matching sets of highlighting queries within an interval */ - logEntryHighlights: InfraLogEntryInterval[]; /** A snapshot of nodes */ snapshot?: InfraSnapshotResponse | null; @@ -157,80 +151,6 @@ export interface InfraIndexField { /** Whether the field should be displayed based on event.module and a ECS allowed list */ displayable: boolean; } -/** A consecutive sequence of log entries */ -export interface InfraLogEntryInterval { - /** The key corresponding to the start of the interval covered by the entries */ - start?: InfraTimeKey | null; - /** The key corresponding to the end of the interval covered by the entries */ - end?: InfraTimeKey | null; - /** Whether there are more log entries available before the start */ - hasMoreBefore: boolean; - /** Whether there are more log entries available after the end */ - hasMoreAfter: boolean; - /** The query the log entries were filtered by */ - filterQuery?: string | null; - /** The query the log entries were highlighted with */ - highlightQuery?: string | null; - /** A list of the log entries */ - entries: InfraLogEntry[]; -} -/** A representation of the log entry's position in the event stream */ -export interface InfraTimeKey { - /** The timestamp of the event that the log entry corresponds to */ - time: number; - /** The tiebreaker that disambiguates events with the same timestamp */ - tiebreaker: number; -} -/** A log entry */ -export interface InfraLogEntry { - /** A unique representation of the log entry's position in the event stream */ - key: InfraTimeKey; - /** The log entry's id */ - gid: string; - /** The source id */ - source: string; - /** The columns used for rendering the log entry */ - columns: InfraLogEntryColumn[]; -} -/** A special built-in column that contains the log entry's timestamp */ -export interface InfraLogEntryTimestampColumn { - /** The id of the corresponding column configuration */ - columnId: string; - /** The timestamp */ - timestamp: number; -} -/** A special built-in column that contains the log entry's constructed message */ -export interface InfraLogEntryMessageColumn { - /** The id of the corresponding column configuration */ - columnId: string; - /** A list of the formatted log entry segments */ - message: InfraLogMessageSegment[]; -} -/** A segment of the log entry message that was derived from a field */ -export interface InfraLogMessageFieldSegment { - /** The field the segment was derived from */ - field: string; - /** The segment's message */ - value: string; - /** A list of highlighted substrings of the value */ - highlights: string[]; -} -/** A segment of the log entry message that was derived from a string literal */ -export interface InfraLogMessageConstantSegment { - /** The segment's message */ - constant: string; -} -/** A column that contains the value of a field of the log entry */ -export interface InfraLogEntryFieldColumn { - /** The id of the corresponding column configuration */ - columnId: string; - /** The field name of the column */ - field: string; - /** The value of the field in the log entry */ - value: string; - /** A list of highlighted substrings of the value */ - highlights: string[]; -} export interface InfraSnapshotResponse { /** Nodes of type host, container or pod grouped by 0, 1 or 2 terms */ @@ -304,21 +224,6 @@ export interface DeleteSourceResult { // InputTypes // ==================================================== -export interface InfraTimeKeyInput { - time: number; - - tiebreaker: number; -} -/** A highlighting definition */ -export interface InfraLogEntryHighlightInput { - /** The query to highlight by */ - query: string; - /** The number of highlighted documents to include beyond the beginning of the interval */ - countBefore: number; - /** The number of highlighted documents to include beyond the end of the interval */ - countAfter: number; -} - export interface InfraTimerangeInput { /** The interval string to use for last bucket. The format is '{value}{unit}'. For example '5m' would return the metrics for the last 5 minutes of the timespan. */ interval: string; @@ -409,34 +314,6 @@ export interface SourceQueryArgs { /** The id of the source */ id: string; } -export interface LogEntriesAroundInfraSourceArgs { - /** The sort key that corresponds to the point in time */ - key: InfraTimeKeyInput; - /** The maximum number of preceding to return */ - countBefore?: number | null; - /** The maximum number of following to return */ - countAfter?: number | null; - /** The query to filter the log entries by */ - filterQuery?: string | null; -} -export interface LogEntriesBetweenInfraSourceArgs { - /** The sort key that corresponds to the start of the interval */ - startKey: InfraTimeKeyInput; - /** The sort key that corresponds to the end of the interval */ - endKey: InfraTimeKeyInput; - /** The query to filter the log entries by */ - filterQuery?: string | null; -} -export interface LogEntryHighlightsInfraSourceArgs { - /** The sort key that corresponds to the start of the interval */ - startKey: InfraTimeKeyInput; - /** The sort key that corresponds to the end of the interval */ - endKey: InfraTimeKeyInput; - /** The query to filter the log entries by */ - filterQuery?: string | null; - /** The highlighting to apply to the log entries */ - highlights: InfraLogEntryHighlightInput[]; -} export interface SnapshotInfraSourceArgs { timerange: InfraTimerangeInput; @@ -593,15 +470,6 @@ export type InfraSourceLogColumn = | InfraSourceMessageLogColumn | InfraSourceFieldLogColumn; -/** A column of a log entry */ -export type InfraLogEntryColumn = - | InfraLogEntryTimestampColumn - | InfraLogEntryMessageColumn - | InfraLogEntryFieldColumn; - -/** A segment of the log entry message */ -export type InfraLogMessageSegment = InfraLogMessageFieldSegment | InfraLogMessageConstantSegment; - // ==================================================== // END: Typescript template // ==================================================== @@ -650,12 +518,6 @@ export namespace InfraSourceResolvers { configuration?: ConfigurationResolver; /** The status of the source */ status?: StatusResolver; - /** A consecutive span of log entries surrounding a point in time */ - logEntriesAround?: LogEntriesAroundResolver; - /** A consecutive span of log entries within an interval */ - logEntriesBetween?: LogEntriesBetweenResolver; - /** Sequences of log entries matching sets of highlighting queries within an interval */ - logEntryHighlights?: LogEntryHighlightsResolver; /** A snapshot of nodes */ snapshot?: SnapshotResolver; @@ -693,51 +555,6 @@ export namespace InfraSourceResolvers { Parent = InfraSource, Context = InfraContext > = Resolver; - export type LogEntriesAroundResolver< - R = InfraLogEntryInterval, - Parent = InfraSource, - Context = InfraContext - > = Resolver; - export interface LogEntriesAroundArgs { - /** The sort key that corresponds to the point in time */ - key: InfraTimeKeyInput; - /** The maximum number of preceding to return */ - countBefore?: number | null; - /** The maximum number of following to return */ - countAfter?: number | null; - /** The query to filter the log entries by */ - filterQuery?: string | null; - } - - export type LogEntriesBetweenResolver< - R = InfraLogEntryInterval, - Parent = InfraSource, - Context = InfraContext - > = Resolver; - export interface LogEntriesBetweenArgs { - /** The sort key that corresponds to the start of the interval */ - startKey: InfraTimeKeyInput; - /** The sort key that corresponds to the end of the interval */ - endKey: InfraTimeKeyInput; - /** The query to filter the log entries by */ - filterQuery?: string | null; - } - - export type LogEntryHighlightsResolver< - R = InfraLogEntryInterval[], - Parent = InfraSource, - Context = InfraContext - > = Resolver; - export interface LogEntryHighlightsArgs { - /** The sort key that corresponds to the start of the interval */ - startKey: InfraTimeKeyInput; - /** The sort key that corresponds to the end of the interval */ - endKey: InfraTimeKeyInput; - /** The query to filter the log entries by */ - filterQuery?: string | null; - /** The highlighting to apply to the log entries */ - highlights: InfraLogEntryHighlightInput[]; - } export type SnapshotResolver< R = InfraSnapshotResponse | null, @@ -1059,229 +876,6 @@ export namespace InfraIndexFieldResolvers { Context = InfraContext > = Resolver; } -/** A consecutive sequence of log entries */ -export namespace InfraLogEntryIntervalResolvers { - export interface Resolvers { - /** The key corresponding to the start of the interval covered by the entries */ - start?: StartResolver; - /** The key corresponding to the end of the interval covered by the entries */ - end?: EndResolver; - /** Whether there are more log entries available before the start */ - hasMoreBefore?: HasMoreBeforeResolver; - /** Whether there are more log entries available after the end */ - hasMoreAfter?: HasMoreAfterResolver; - /** The query the log entries were filtered by */ - filterQuery?: FilterQueryResolver; - /** The query the log entries were highlighted with */ - highlightQuery?: HighlightQueryResolver; - /** A list of the log entries */ - entries?: EntriesResolver; - } - - export type StartResolver< - R = InfraTimeKey | null, - Parent = InfraLogEntryInterval, - Context = InfraContext - > = Resolver; - export type EndResolver< - R = InfraTimeKey | null, - Parent = InfraLogEntryInterval, - Context = InfraContext - > = Resolver; - export type HasMoreBeforeResolver< - R = boolean, - Parent = InfraLogEntryInterval, - Context = InfraContext - > = Resolver; - export type HasMoreAfterResolver< - R = boolean, - Parent = InfraLogEntryInterval, - Context = InfraContext - > = Resolver; - export type FilterQueryResolver< - R = string | null, - Parent = InfraLogEntryInterval, - Context = InfraContext - > = Resolver; - export type HighlightQueryResolver< - R = string | null, - Parent = InfraLogEntryInterval, - Context = InfraContext - > = Resolver; - export type EntriesResolver< - R = InfraLogEntry[], - Parent = InfraLogEntryInterval, - Context = InfraContext - > = Resolver; -} -/** A representation of the log entry's position in the event stream */ -export namespace InfraTimeKeyResolvers { - export interface Resolvers { - /** The timestamp of the event that the log entry corresponds to */ - time?: TimeResolver; - /** The tiebreaker that disambiguates events with the same timestamp */ - tiebreaker?: TiebreakerResolver; - } - - export type TimeResolver = Resolver< - R, - Parent, - Context - >; - export type TiebreakerResolver< - R = number, - Parent = InfraTimeKey, - Context = InfraContext - > = Resolver; -} -/** A log entry */ -export namespace InfraLogEntryResolvers { - export interface Resolvers { - /** A unique representation of the log entry's position in the event stream */ - key?: KeyResolver; - /** The log entry's id */ - gid?: GidResolver; - /** The source id */ - source?: SourceResolver; - /** The columns used for rendering the log entry */ - columns?: ColumnsResolver; - } - - export type KeyResolver< - R = InfraTimeKey, - Parent = InfraLogEntry, - Context = InfraContext - > = Resolver; - export type GidResolver = Resolver< - R, - Parent, - Context - >; - export type SourceResolver = Resolver< - R, - Parent, - Context - >; - export type ColumnsResolver< - R = InfraLogEntryColumn[], - Parent = InfraLogEntry, - Context = InfraContext - > = Resolver; -} -/** A special built-in column that contains the log entry's timestamp */ -export namespace InfraLogEntryTimestampColumnResolvers { - export interface Resolvers { - /** The id of the corresponding column configuration */ - columnId?: ColumnIdResolver; - /** The timestamp */ - timestamp?: TimestampResolver; - } - - export type ColumnIdResolver< - R = string, - Parent = InfraLogEntryTimestampColumn, - Context = InfraContext - > = Resolver; - export type TimestampResolver< - R = number, - Parent = InfraLogEntryTimestampColumn, - Context = InfraContext - > = Resolver; -} -/** A special built-in column that contains the log entry's constructed message */ -export namespace InfraLogEntryMessageColumnResolvers { - export interface Resolvers { - /** The id of the corresponding column configuration */ - columnId?: ColumnIdResolver; - /** A list of the formatted log entry segments */ - message?: MessageResolver; - } - - export type ColumnIdResolver< - R = string, - Parent = InfraLogEntryMessageColumn, - Context = InfraContext - > = Resolver; - export type MessageResolver< - R = InfraLogMessageSegment[], - Parent = InfraLogEntryMessageColumn, - Context = InfraContext - > = Resolver; -} -/** A segment of the log entry message that was derived from a field */ -export namespace InfraLogMessageFieldSegmentResolvers { - export interface Resolvers { - /** The field the segment was derived from */ - field?: FieldResolver; - /** The segment's message */ - value?: ValueResolver; - /** A list of highlighted substrings of the value */ - highlights?: HighlightsResolver; - } - - export type FieldResolver< - R = string, - Parent = InfraLogMessageFieldSegment, - Context = InfraContext - > = Resolver; - export type ValueResolver< - R = string, - Parent = InfraLogMessageFieldSegment, - Context = InfraContext - > = Resolver; - export type HighlightsResolver< - R = string[], - Parent = InfraLogMessageFieldSegment, - Context = InfraContext - > = Resolver; -} -/** A segment of the log entry message that was derived from a string literal */ -export namespace InfraLogMessageConstantSegmentResolvers { - export interface Resolvers { - /** The segment's message */ - constant?: ConstantResolver; - } - - export type ConstantResolver< - R = string, - Parent = InfraLogMessageConstantSegment, - Context = InfraContext - > = Resolver; -} -/** A column that contains the value of a field of the log entry */ -export namespace InfraLogEntryFieldColumnResolvers { - export interface Resolvers { - /** The id of the corresponding column configuration */ - columnId?: ColumnIdResolver; - /** The field name of the column */ - field?: FieldResolver; - /** The value of the field in the log entry */ - value?: ValueResolver; - /** A list of highlighted substrings of the value */ - highlights?: HighlightsResolver; - } - - export type ColumnIdResolver< - R = string, - Parent = InfraLogEntryFieldColumn, - Context = InfraContext - > = Resolver; - export type FieldResolver< - R = string, - Parent = InfraLogEntryFieldColumn, - Context = InfraContext - > = Resolver; - export type ValueResolver< - R = string, - Parent = InfraLogEntryFieldColumn, - Context = InfraContext - > = Resolver; - export type HighlightsResolver< - R = string[], - Parent = InfraLogEntryFieldColumn, - Context = InfraContext - > = Resolver; -} export namespace InfraSnapshotResponseResolvers { export interface Resolvers { diff --git a/x-pack/plugins/infra/server/lib/adapters/log_entries/kibana_log_entries_adapter.ts b/x-pack/plugins/infra/server/lib/adapters/log_entries/kibana_log_entries_adapter.ts index ffbc750af14f82..6702a43cb2316e 100644 --- a/x-pack/plugins/infra/server/lib/adapters/log_entries/kibana_log_entries_adapter.ts +++ b/x-pack/plugins/infra/server/lib/adapters/log_entries/kibana_log_entries_adapter.ts @@ -215,6 +215,7 @@ function mapHitsToLogEntryDocuments(hits: SortedSearchHit[], fields: string[]): return { id: hit._id, + index: hit._index, cursor: { time: hit.sort[0], tiebreaker: hit.sort[1] }, fields: logFields, highlights: hit.highlight || {}, diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/log_entries_domain.ts b/x-pack/plugins/infra/server/lib/domains/log_entries_domain/log_entries_domain.ts index 4c5debe58ed260..e318075045522f 100644 --- a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/log_entries_domain.ts +++ b/x-pack/plugins/infra/server/lib/domains/log_entries_domain/log_entries_domain.ts @@ -12,19 +12,19 @@ import { LogEntriesSummaryHighlightsBucket, LogEntriesRequest, } from '../../../../common/http_api'; -import { LogEntry, LogColumn } from '../../../../common/log_entry'; +import { LogColumn, LogEntryCursor, LogEntry } from '../../../../common/log_entry'; import { InfraSourceConfiguration, InfraSources, SavedSourceConfigurationFieldColumnRuntimeType, } from '../../sources'; -import { getBuiltinRules } from './builtin_rules'; +import { getBuiltinRules } from '../../../services/log_entries/message/builtin_rules'; import { CompiledLogMessageFormattingRule, Fields, Highlights, compileFormattingRules, -} from './message'; +} from '../../../services/log_entries/message/message'; import { KibanaFramework } from '../../adapters/framework/kibana_framework_adapter'; import { decodeOrThrow } from '../../../../common/runtime_types'; import { @@ -33,7 +33,6 @@ import { CompositeDatasetKey, createLogEntryDatasetsQuery, } from './queries/log_entry_datasets'; -import { LogEntryCursor } from '../../../../common/log_entry'; export interface LogEntriesParams { startTimestamp: number; @@ -156,6 +155,7 @@ export class InfraLogEntriesDomain { const entries = documents.map((doc) => { return { id: doc.id, + index: doc.index, cursor: doc.cursor, columns: columnDefinitions.map( (column): LogColumn => { @@ -317,6 +317,7 @@ export type LogEntryQuery = JsonObject; export interface LogEntryDocument { id: string; + index: string; fields: Fields; highlights: Highlights; cursor: LogEntryCursor; diff --git a/x-pack/plugins/infra/server/lib/log_analysis/log_entry_categories_analysis.ts b/x-pack/plugins/infra/server/lib/log_analysis/log_entry_categories_analysis.ts index 071a8a94e009be..5d4846598d2045 100644 --- a/x-pack/plugins/infra/server/lib/log_analysis/log_entry_categories_analysis.ts +++ b/x-pack/plugins/infra/server/lib/log_analysis/log_entry_categories_analysis.ts @@ -5,7 +5,6 @@ */ import type { ILegacyScopedClusterClient } from 'src/core/server'; -import { LogEntryContext } from '../../../common/log_entry'; import { compareDatasetsByMaximumAnomalyScore, getJobId, @@ -13,6 +12,7 @@ import { logEntryCategoriesJobTypes, CategoriesSort, } from '../../../common/log_analysis'; +import { LogEntryContext } from '../../../common/log_entry'; import { startTracingSpan } from '../../../common/performance_tracing'; import { decodeOrThrow } from '../../../common/runtime_types'; import type { MlAnomalyDetectors, MlSystem } from '../../types'; diff --git a/x-pack/plugins/infra/server/services/log_entries/log_entries_search_strategy.test.ts b/x-pack/plugins/infra/server/services/log_entries/log_entries_search_strategy.test.ts new file mode 100644 index 00000000000000..f07ee0508fa6c4 --- /dev/null +++ b/x-pack/plugins/infra/server/services/log_entries/log_entries_search_strategy.test.ts @@ -0,0 +1,318 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { ResponseError } from '@elastic/elasticsearch/lib/errors'; +import { of, throwError } from 'rxjs'; +import { + elasticsearchServiceMock, + savedObjectsClientMock, + uiSettingsServiceMock, +} from 'src/core/server/mocks'; +import { + IEsSearchRequest, + IEsSearchResponse, + ISearchStrategy, + SearchStrategyDependencies, +} from 'src/plugins/data/server'; +import { InfraSource } from '../../lib/sources'; +import { createInfraSourcesMock } from '../../lib/sources/mocks'; +import { + logEntriesSearchRequestStateRT, + logEntriesSearchStrategyProvider, +} from './log_entries_search_strategy'; + +describe('LogEntries search strategy', () => { + it('handles initial search requests', async () => { + const esSearchStrategyMock = createEsSearchStrategyMock({ + id: 'ASYNC_REQUEST_ID', + isRunning: true, + rawResponse: { + took: 0, + _shards: { total: 1, failed: 0, skipped: 0, successful: 0 }, + timed_out: false, + hits: { total: 0, max_score: 0, hits: [] }, + }, + }); + + const dataMock = createDataPluginMock(esSearchStrategyMock); + const sourcesMock = createInfraSourcesMock(); + sourcesMock.getSourceConfiguration.mockResolvedValue(createSourceConfigurationMock()); + const mockDependencies = createSearchStrategyDependenciesMock(); + + const logEntriesSearchStrategy = logEntriesSearchStrategyProvider({ + data: dataMock, + sources: sourcesMock, + }); + + const response = await logEntriesSearchStrategy + .search( + { + params: { + sourceId: 'SOURCE_ID', + startTimestamp: 100, + endTimestamp: 200, + size: 3, + }, + }, + {}, + mockDependencies + ) + .toPromise(); + + expect(sourcesMock.getSourceConfiguration).toHaveBeenCalled(); + expect(esSearchStrategyMock.search).toHaveBeenCalledWith( + expect.objectContaining({ + params: expect.objectContaining({ + index: 'log-indices-*', + body: expect.objectContaining({ + fields: expect.arrayContaining(['event.dataset', 'message']), + }), + }), + }), + expect.anything(), + expect.anything() + ); + expect(response.id).toEqual(expect.any(String)); + expect(response.isRunning).toBe(true); + }); + + it('handles subsequent polling requests', async () => { + const esSearchStrategyMock = createEsSearchStrategyMock({ + id: 'ASYNC_REQUEST_ID', + isRunning: false, + rawResponse: { + took: 1, + _shards: { total: 1, failed: 0, skipped: 0, successful: 1 }, + timed_out: false, + hits: { + total: 0, + max_score: 0, + hits: [ + { + _id: 'HIT_ID', + _index: 'HIT_INDEX', + _type: '_doc', + _score: 0, + _source: null, + fields: { + '@timestamp': [1605116827143], + 'event.dataset': ['HIT_DATASET'], + MESSAGE_FIELD: ['HIT_MESSAGE'], + 'container.id': ['HIT_CONTAINER_ID'], + }, + sort: [1605116827143 as any, 1 as any], // incorrectly typed as string upstream + }, + ], + }, + }, + }); + const dataMock = createDataPluginMock(esSearchStrategyMock); + const sourcesMock = createInfraSourcesMock(); + sourcesMock.getSourceConfiguration.mockResolvedValue(createSourceConfigurationMock()); + const mockDependencies = createSearchStrategyDependenciesMock(); + + const logEntriesSearchStrategy = logEntriesSearchStrategyProvider({ + data: dataMock, + sources: sourcesMock, + }); + const requestId = logEntriesSearchRequestStateRT.encode({ + esRequestId: 'ASYNC_REQUEST_ID', + }); + + const response = await logEntriesSearchStrategy + .search( + { + id: requestId, + params: { + sourceId: 'SOURCE_ID', + startTimestamp: 100, + endTimestamp: 200, + size: 3, + }, + }, + {}, + mockDependencies + ) + .toPromise(); + + expect(sourcesMock.getSourceConfiguration).toHaveBeenCalled(); + expect(esSearchStrategyMock.search).toHaveBeenCalled(); + expect(response.id).toEqual(requestId); + expect(response.isRunning).toBe(false); + expect(response.rawResponse.data.entries).toEqual([ + { + id: 'HIT_ID', + index: 'HIT_INDEX', + cursor: { + time: 1605116827143, + tiebreaker: 1, + }, + columns: [ + { + columnId: 'TIMESTAMP_COLUMN_ID', + timestamp: 1605116827143, + }, + { + columnId: 'DATASET_COLUMN_ID', + field: 'event.dataset', + value: ['HIT_DATASET'], + highlights: [], + }, + { + columnId: 'MESSAGE_COLUMN_ID', + message: [ + { + field: 'MESSAGE_FIELD', + value: ['HIT_MESSAGE'], + highlights: [], + }, + ], + }, + ], + context: { + 'container.id': 'HIT_CONTAINER_ID', + }, + }, + ]); + }); + + it('forwards errors from the underlying search strategy', async () => { + const esSearchStrategyMock = createEsSearchStrategyMock({ + id: 'ASYNC_REQUEST_ID', + isRunning: false, + rawResponse: { + took: 1, + _shards: { total: 1, failed: 0, skipped: 0, successful: 1 }, + timed_out: false, + hits: { total: 0, max_score: 0, hits: [] }, + }, + }); + const dataMock = createDataPluginMock(esSearchStrategyMock); + const sourcesMock = createInfraSourcesMock(); + sourcesMock.getSourceConfiguration.mockResolvedValue(createSourceConfigurationMock()); + const mockDependencies = createSearchStrategyDependenciesMock(); + + const logEntriesSearchStrategy = logEntriesSearchStrategyProvider({ + data: dataMock, + sources: sourcesMock, + }); + + const response = logEntriesSearchStrategy.search( + { + id: logEntriesSearchRequestStateRT.encode({ esRequestId: 'UNKNOWN_ID' }), + params: { + sourceId: 'SOURCE_ID', + startTimestamp: 100, + endTimestamp: 200, + size: 3, + }, + }, + {}, + mockDependencies + ); + + await expect(response.toPromise()).rejects.toThrowError(ResponseError); + }); + + it('forwards cancellation to the underlying search strategy', async () => { + const esSearchStrategyMock = createEsSearchStrategyMock({ + id: 'ASYNC_REQUEST_ID', + isRunning: false, + rawResponse: { + took: 1, + _shards: { total: 1, failed: 0, skipped: 0, successful: 1 }, + timed_out: false, + hits: { total: 0, max_score: 0, hits: [] }, + }, + }); + const dataMock = createDataPluginMock(esSearchStrategyMock); + const sourcesMock = createInfraSourcesMock(); + sourcesMock.getSourceConfiguration.mockResolvedValue(createSourceConfigurationMock()); + const mockDependencies = createSearchStrategyDependenciesMock(); + + const logEntriesSearchStrategy = logEntriesSearchStrategyProvider({ + data: dataMock, + sources: sourcesMock, + }); + const requestId = logEntriesSearchRequestStateRT.encode({ + esRequestId: 'ASYNC_REQUEST_ID', + }); + + await logEntriesSearchStrategy.cancel?.(requestId, {}, mockDependencies); + + expect(esSearchStrategyMock.cancel).toHaveBeenCalled(); + }); +}); + +const createSourceConfigurationMock = (): InfraSource => ({ + id: 'SOURCE_ID', + origin: 'stored' as const, + configuration: { + name: 'SOURCE_NAME', + description: 'SOURCE_DESCRIPTION', + logAlias: 'log-indices-*', + metricAlias: 'metric-indices-*', + inventoryDefaultView: 'DEFAULT_VIEW', + metricsExplorerDefaultView: 'DEFAULT_VIEW', + logColumns: [ + { timestampColumn: { id: 'TIMESTAMP_COLUMN_ID' } }, + { + fieldColumn: { + id: 'DATASET_COLUMN_ID', + field: 'event.dataset', + }, + }, + { + messageColumn: { id: 'MESSAGE_COLUMN_ID' }, + }, + ], + fields: { + pod: 'POD_FIELD', + host: 'HOST_FIELD', + container: 'CONTAINER_FIELD', + message: ['MESSAGE_FIELD'], + timestamp: 'TIMESTAMP_FIELD', + tiebreaker: 'TIEBREAKER_FIELD', + }, + }, +}); + +const createEsSearchStrategyMock = (esSearchResponse: IEsSearchResponse) => ({ + search: jest.fn((esSearchRequest: IEsSearchRequest) => { + if (typeof esSearchRequest.id === 'string') { + if (esSearchRequest.id === esSearchResponse.id) { + return of(esSearchResponse); + } else { + return throwError( + new ResponseError({ + body: {}, + headers: {}, + meta: {} as any, + statusCode: 404, + warnings: [], + }) + ); + } + } else { + return of(esSearchResponse); + } + }), + cancel: jest.fn().mockResolvedValue(undefined), +}); + +const createSearchStrategyDependenciesMock = (): SearchStrategyDependencies => ({ + uiSettingsClient: uiSettingsServiceMock.createClient(), + esClient: elasticsearchServiceMock.createScopedClusterClient(), + savedObjectsClient: savedObjectsClientMock.create(), +}); + +// using the official data mock from within x-pack doesn't type-check successfully, +// because the `licensing` plugin modifies the `RequestHandlerContext` core type. +const createDataPluginMock = (esSearchStrategyMock: ISearchStrategy): any => ({ + search: { + getSearchStrategy: jest.fn().mockReturnValue(esSearchStrategyMock), + }, +}); diff --git a/x-pack/plugins/infra/server/services/log_entries/log_entries_search_strategy.ts b/x-pack/plugins/infra/server/services/log_entries/log_entries_search_strategy.ts new file mode 100644 index 00000000000000..6ce3d4410a2dd2 --- /dev/null +++ b/x-pack/plugins/infra/server/services/log_entries/log_entries_search_strategy.ts @@ -0,0 +1,245 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { pick } from '@kbn/std'; +import * as rt from 'io-ts'; +import { combineLatest, concat, defer, forkJoin, of } from 'rxjs'; +import { concatMap, filter, map, shareReplay, take } from 'rxjs/operators'; +import type { + IEsSearchRequest, + IKibanaSearchRequest, + IKibanaSearchResponse, +} from '../../../../../../src/plugins/data/common'; +import type { + ISearchStrategy, + PluginStart as DataPluginStart, +} from '../../../../../../src/plugins/data/server'; +import { LogSourceColumnConfiguration } from '../../../common/http_api/log_sources'; +import { + getLogEntryCursorFromHit, + LogColumn, + LogEntry, + LogEntryAfterCursor, + logEntryAfterCursorRT, + LogEntryBeforeCursor, + logEntryBeforeCursorRT, + LogEntryContext, +} from '../../../common/log_entry'; +import { decodeOrThrow } from '../../../common/runtime_types'; +import { + LogEntriesSearchRequestParams, + logEntriesSearchRequestParamsRT, + LogEntriesSearchResponsePayload, + logEntriesSearchResponsePayloadRT, +} from '../../../common/search_strategies/log_entries/log_entries'; +import type { IInfraSources } from '../../lib/sources'; +import { + createAsyncRequestRTs, + createErrorFromShardFailure, + jsonFromBase64StringRT, +} from '../../utils/typed_search_strategy'; +import { + CompiledLogMessageFormattingRule, + compileFormattingRules, + getBuiltinRules, +} from './message'; +import { + createGetLogEntriesQuery, + getLogEntriesResponseRT, + getSortDirection, + LogEntryHit, +} from './queries/log_entries'; + +type LogEntriesSearchRequest = IKibanaSearchRequest; +type LogEntriesSearchResponse = IKibanaSearchResponse; + +export const logEntriesSearchStrategyProvider = ({ + data, + sources, +}: { + data: DataPluginStart; + sources: IInfraSources; +}): ISearchStrategy => { + const esSearchStrategy = data.search.getSearchStrategy('ese'); + + return { + search: (rawRequest, options, dependencies) => + defer(() => { + const request = decodeOrThrow(asyncRequestRT)(rawRequest); + + const sourceConfiguration$ = defer(() => + sources.getSourceConfiguration(dependencies.savedObjectsClient, request.params.sourceId) + ).pipe(take(1), shareReplay(1)); + + const messageFormattingRules$ = defer(() => + sourceConfiguration$.pipe( + map(({ configuration }) => + compileFormattingRules(getBuiltinRules(configuration.fields.message)) + ) + ) + ).pipe(take(1), shareReplay(1)); + + const recoveredRequest$ = of(request).pipe( + filter(asyncRecoveredRequestRT.is), + map(({ id: { esRequestId } }) => ({ id: esRequestId })) + ); + + const initialRequest$ = of(request).pipe( + filter(asyncInitialRequestRT.is), + concatMap(({ params }) => + forkJoin([sourceConfiguration$, messageFormattingRules$]).pipe( + map( + ([{ configuration }, messageFormattingRules]): IEsSearchRequest => { + return { + params: createGetLogEntriesQuery( + configuration.logAlias, + params.startTimestamp, + params.endTimestamp, + pickRequestCursor(params), + params.size + 1, + configuration.fields.timestamp, + configuration.fields.tiebreaker, + messageFormattingRules.requiredFields, + params.query, + params.highlightPhrase + ), + }; + } + ) + ) + ) + ); + + const searchResponse$ = concat(recoveredRequest$, initialRequest$).pipe( + take(1), + concatMap((esRequest) => esSearchStrategy.search(esRequest, options, dependencies)) + ); + + return combineLatest([searchResponse$, sourceConfiguration$, messageFormattingRules$]).pipe( + map(([esResponse, { configuration }, messageFormattingRules]) => { + const rawResponse = decodeOrThrow(getLogEntriesResponseRT)(esResponse.rawResponse); + + const entries = rawResponse.hits.hits + .slice(0, request.params.size) + .map(getLogEntryFromHit(configuration.logColumns, messageFormattingRules)); + + const sortDirection = getSortDirection(pickRequestCursor(request.params)); + + if (sortDirection === 'desc') { + entries.reverse(); + } + + const hasMore = rawResponse.hits.hits.length > entries.length; + const hasMoreBefore = sortDirection === 'desc' ? hasMore : undefined; + const hasMoreAfter = sortDirection === 'asc' ? hasMore : undefined; + + const { topCursor, bottomCursor } = getResponseCursors(entries); + + const errors = (rawResponse._shards.failures ?? []).map(createErrorFromShardFailure); + + return { + ...esResponse, + ...(esResponse.id + ? { id: logEntriesSearchRequestStateRT.encode({ esRequestId: esResponse.id }) } + : {}), + rawResponse: logEntriesSearchResponsePayloadRT.encode({ + data: { entries, topCursor, bottomCursor, hasMoreBefore, hasMoreAfter }, + errors, + }), + }; + }) + ); + }), + cancel: async (id, options, dependencies) => { + const { esRequestId } = decodeOrThrow(logEntriesSearchRequestStateRT)(id); + return await esSearchStrategy.cancel?.(esRequestId, options, dependencies); + }, + }; +}; + +// exported for tests +export const logEntriesSearchRequestStateRT = rt.string.pipe(jsonFromBase64StringRT).pipe( + rt.type({ + esRequestId: rt.string, + }) +); + +const { asyncInitialRequestRT, asyncRecoveredRequestRT, asyncRequestRT } = createAsyncRequestRTs( + logEntriesSearchRequestStateRT, + logEntriesSearchRequestParamsRT +); + +const getLogEntryFromHit = ( + columnDefinitions: LogSourceColumnConfiguration[], + messageFormattingRules: CompiledLogMessageFormattingRule +) => (hit: LogEntryHit): LogEntry => { + const cursor = getLogEntryCursorFromHit(hit); + return { + id: hit._id, + index: hit._index, + cursor, + columns: columnDefinitions.map( + (column): LogColumn => { + if ('timestampColumn' in column) { + return { + columnId: column.timestampColumn.id, + timestamp: cursor.time, + }; + } else if ('messageColumn' in column) { + return { + columnId: column.messageColumn.id, + message: messageFormattingRules.format(hit.fields, hit.highlight || {}), + }; + } else { + return { + columnId: column.fieldColumn.id, + field: column.fieldColumn.field, + value: hit.fields[column.fieldColumn.field] ?? [], + highlights: hit.highlight?.[column.fieldColumn.field] ?? [], + }; + } + } + ), + context: getContextFromHit(hit), + }; +}; + +const pickRequestCursor = ( + params: LogEntriesSearchRequestParams +): LogEntryAfterCursor | LogEntryBeforeCursor | null => { + if (logEntryAfterCursorRT.is(params)) { + return pick(params, ['after']); + } else if (logEntryBeforeCursorRT.is(params)) { + return pick(params, ['before']); + } + + return null; +}; + +const getContextFromHit = (hit: LogEntryHit): LogEntryContext => { + // Get all context fields, then test for the presence and type of the ones that go together + const containerId = hit.fields['container.id']?.[0]; + const hostName = hit.fields['host.name']?.[0]; + const logFilePath = hit.fields['log.file.path']?.[0]; + + if (typeof containerId === 'string') { + return { 'container.id': containerId }; + } + + if (typeof hostName === 'string' && typeof logFilePath === 'string') { + return { 'host.name': hostName, 'log.file.path': logFilePath }; + } + + return {}; +}; + +function getResponseCursors(entries: LogEntry[]) { + const hasEntries = entries.length > 0; + const topCursor = hasEntries ? entries[0].cursor : null; + const bottomCursor = hasEntries ? entries[entries.length - 1].cursor : null; + + return { topCursor, bottomCursor }; +} diff --git a/x-pack/plugins/infra/server/services/log_entries/log_entries_service.ts b/x-pack/plugins/infra/server/services/log_entries/log_entries_service.ts index edd53be9db8411..9aba69428f2575 100644 --- a/x-pack/plugins/infra/server/services/log_entries/log_entries_service.ts +++ b/x-pack/plugins/infra/server/services/log_entries/log_entries_service.ts @@ -6,12 +6,18 @@ import { CoreSetup } from 'src/core/server'; import { LOG_ENTRY_SEARCH_STRATEGY } from '../../../common/search_strategies/log_entries/log_entry'; +import { LOG_ENTRIES_SEARCH_STRATEGY } from '../../../common/search_strategies/log_entries/log_entries'; +import { logEntriesSearchStrategyProvider } from './log_entries_search_strategy'; import { logEntrySearchStrategyProvider } from './log_entry_search_strategy'; import { LogEntriesServiceSetupDeps, LogEntriesServiceStartDeps } from './types'; export class LogEntriesService { public setup(core: CoreSetup, setupDeps: LogEntriesServiceSetupDeps) { core.getStartServices().then(([, startDeps]) => { + setupDeps.data.search.registerSearchStrategy( + LOG_ENTRIES_SEARCH_STRATEGY, + logEntriesSearchStrategyProvider({ ...setupDeps, ...startDeps }) + ); setupDeps.data.search.registerSearchStrategy( LOG_ENTRY_SEARCH_STRATEGY, logEntrySearchStrategyProvider({ ...setupDeps, ...startDeps }) diff --git a/x-pack/plugins/infra/server/services/log_entries/log_entry_search_strategy.test.ts b/x-pack/plugins/infra/server/services/log_entries/log_entry_search_strategy.test.ts index 38626675f5ae7b..b3e1a31f73b7ab 100644 --- a/x-pack/plugins/infra/server/services/log_entries/log_entry_search_strategy.test.ts +++ b/x-pack/plugins/infra/server/services/log_entries/log_entry_search_strategy.test.ts @@ -121,7 +121,7 @@ describe('LogEntry search strategy', () => { expect(response.rawResponse.data).toEqual({ id: 'HIT_ID', index: 'HIT_INDEX', - key: { + cursor: { time: 1605116827143, tiebreaker: 1, }, diff --git a/x-pack/plugins/infra/server/services/log_entries/log_entry_search_strategy.ts b/x-pack/plugins/infra/server/services/log_entries/log_entry_search_strategy.ts index a0dfe3d7176fd4..ab2b72055e4a41 100644 --- a/x-pack/plugins/infra/server/services/log_entries/log_entry_search_strategy.ts +++ b/x-pack/plugins/infra/server/services/log_entries/log_entry_search_strategy.ts @@ -119,6 +119,6 @@ const { asyncInitialRequestRT, asyncRecoveredRequestRT, asyncRequestRT } = creat const createLogEntryFromHit = (hit: LogEntryHit) => ({ id: hit._id, index: hit._index, - key: getLogEntryCursorFromHit(hit), + cursor: getLogEntryCursorFromHit(hit), fields: Object.entries(hit.fields).map(([field, value]) => ({ field, value })), }); diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_apache2.test.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_apache2.test.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_apache2.test.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_apache2.test.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_apache2.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_apache2.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_apache2.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_apache2.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_auditd.test.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_auditd.test.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_auditd.test.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_auditd.test.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_auditd.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_auditd.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_auditd.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_auditd.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_haproxy.test.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_haproxy.test.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_haproxy.test.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_haproxy.test.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_haproxy.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_haproxy.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_haproxy.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_haproxy.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_icinga.test.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_icinga.test.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_icinga.test.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_icinga.test.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_icinga.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_icinga.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_icinga.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_icinga.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_iis.test.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_iis.test.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_iis.test.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_iis.test.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_iis.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_iis.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_iis.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_iis.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_kafka.test.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_kafka.test.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_kafka.test.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_kafka.test.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_logstash.test.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_logstash.test.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_logstash.test.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_logstash.test.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_logstash.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_logstash.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_logstash.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_logstash.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_mongodb.test.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_mongodb.test.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_mongodb.test.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_mongodb.test.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_mongodb.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_mongodb.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_mongodb.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_mongodb.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_mysql.test.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_mysql.test.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_mysql.test.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_mysql.test.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_mysql.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_mysql.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_mysql.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_mysql.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_nginx.test.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_nginx.test.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_nginx.test.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_nginx.test.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_nginx.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_nginx.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_nginx.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_nginx.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_osquery.test.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_osquery.test.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_osquery.test.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_osquery.test.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_osquery.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_osquery.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_osquery.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_osquery.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_redis.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_redis.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_redis.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_redis.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_system.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_system.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_system.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_system.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_traefik.test.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_traefik.test.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_traefik.test.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_traefik.test.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_traefik.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_traefik.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/filebeat_traefik.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/filebeat_traefik.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/generic.test.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/generic.test.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/generic.test.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/generic.test.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/generic.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/generic.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/generic.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/generic.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/generic_webserver.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/generic_webserver.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/generic_webserver.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/generic_webserver.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/helpers.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/helpers.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/helpers.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/helpers.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/index.ts b/x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/index.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/builtin_rules/index.ts rename to x-pack/plugins/infra/server/services/log_entries/message/builtin_rules/index.ts diff --git a/x-pack/plugins/infra/server/services/log_entries/message/index.ts b/x-pack/plugins/infra/server/services/log_entries/message/index.ts new file mode 100644 index 00000000000000..05126eea075afe --- /dev/null +++ b/x-pack/plugins/infra/server/services/log_entries/message/index.ts @@ -0,0 +1,8 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +export * from './message'; +export { getBuiltinRules } from './builtin_rules'; diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/message.ts b/x-pack/plugins/infra/server/services/log_entries/message/message.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/message.ts rename to x-pack/plugins/infra/server/services/log_entries/message/message.ts diff --git a/x-pack/plugins/infra/server/lib/domains/log_entries_domain/rule_types.ts b/x-pack/plugins/infra/server/services/log_entries/message/rule_types.ts similarity index 100% rename from x-pack/plugins/infra/server/lib/domains/log_entries_domain/rule_types.ts rename to x-pack/plugins/infra/server/services/log_entries/message/rule_types.ts diff --git a/x-pack/plugins/infra/server/services/log_entries/queries/common.ts b/x-pack/plugins/infra/server/services/log_entries/queries/common.ts new file mode 100644 index 00000000000000..f170fa337a8b9e --- /dev/null +++ b/x-pack/plugins/infra/server/services/log_entries/queries/common.ts @@ -0,0 +1,32 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +export const createSortClause = ( + sortDirection: 'asc' | 'desc', + timestampField: string, + tiebreakerField: string +) => ({ + sort: { + [timestampField]: sortDirection, + [tiebreakerField]: sortDirection, + }, +}); + +export const createTimeRangeFilterClauses = ( + startTimestamp: number, + endTimestamp: number, + timestampField: string +) => [ + { + range: { + [timestampField]: { + gte: startTimestamp, + lte: endTimestamp, + format: 'epoch_millis', + }, + }, + }, +]; diff --git a/x-pack/plugins/infra/server/services/log_entries/queries/log_entries.ts b/x-pack/plugins/infra/server/services/log_entries/queries/log_entries.ts new file mode 100644 index 00000000000000..81476fa2b286e2 --- /dev/null +++ b/x-pack/plugins/infra/server/services/log_entries/queries/log_entries.ts @@ -0,0 +1,141 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import type { RequestParams } from '@elastic/elasticsearch'; +import * as rt from 'io-ts'; +import { + LogEntryAfterCursor, + logEntryAfterCursorRT, + LogEntryBeforeCursor, + logEntryBeforeCursorRT, +} from '../../../../common/log_entry'; +import { jsonArrayRT, JsonObject } from '../../../../common/typed_json'; +import { + commonHitFieldsRT, + commonSearchSuccessResponseFieldsRT, +} from '../../../utils/elasticsearch_runtime_types'; +import { createSortClause, createTimeRangeFilterClauses } from './common'; + +export const createGetLogEntriesQuery = ( + logEntriesIndex: string, + startTimestamp: number, + endTimestamp: number, + cursor: LogEntryBeforeCursor | LogEntryAfterCursor | null | undefined, + size: number, + timestampField: string, + tiebreakerField: string, + fields: string[], + query?: JsonObject, + highlightTerm?: string +): RequestParams.AsyncSearchSubmit> => { + const sortDirection = getSortDirection(cursor); + const highlightQuery = createHighlightQuery(highlightTerm, fields); + + return { + index: logEntriesIndex, + allow_no_indices: true, + track_scores: false, + track_total_hits: false, + body: { + size, + query: { + bool: { + filter: [ + ...(query ? [query] : []), + ...(highlightQuery ? [highlightQuery] : []), + ...createTimeRangeFilterClauses(startTimestamp, endTimestamp, timestampField), + ], + }, + }, + fields, + _source: false, + ...createSortClause(sortDirection, timestampField, tiebreakerField), + ...createSearchAfterClause(cursor), + ...createHighlightClause(highlightQuery, fields), + }, + }; +}; + +export const getSortDirection = ( + cursor: LogEntryBeforeCursor | LogEntryAfterCursor | null | undefined +): 'asc' | 'desc' => (logEntryBeforeCursorRT.is(cursor) ? 'desc' : 'asc'); + +const createSearchAfterClause = ( + cursor: LogEntryBeforeCursor | LogEntryAfterCursor | null | undefined +): { search_after?: [number, number] } => { + if (logEntryBeforeCursorRT.is(cursor) && cursor.before !== 'last') { + return { + search_after: [cursor.before.time, cursor.before.tiebreaker], + }; + } else if (logEntryAfterCursorRT.is(cursor) && cursor.after !== 'first') { + return { + search_after: [cursor.after.time, cursor.after.tiebreaker], + }; + } + + return {}; +}; + +const createHighlightClause = (highlightQuery: JsonObject | undefined, fields: string[]) => + highlightQuery + ? { + highlight: { + boundary_scanner: 'word', + fields: fields.reduce( + (highlightFieldConfigs, fieldName) => ({ + ...highlightFieldConfigs, + [fieldName]: {}, + }), + {} + ), + fragment_size: 1, + number_of_fragments: 100, + post_tags: [''], + pre_tags: [''], + highlight_query: highlightQuery, + }, + } + : {}; + +const createHighlightQuery = ( + highlightTerm: string | undefined, + fields: string[] +): JsonObject | undefined => { + if (highlightTerm) { + return { + multi_match: { + fields, + lenient: true, + query: highlightTerm, + type: 'phrase', + }, + }; + } +}; + +export const logEntryHitRT = rt.intersection([ + commonHitFieldsRT, + rt.type({ + fields: rt.record(rt.string, jsonArrayRT), + sort: rt.tuple([rt.number, rt.number]), + }), + rt.partial({ + highlight: rt.record(rt.string, rt.array(rt.string)), + }), +]); + +export type LogEntryHit = rt.TypeOf; + +export const getLogEntriesResponseRT = rt.intersection([ + commonSearchSuccessResponseFieldsRT, + rt.type({ + hits: rt.type({ + hits: rt.array(logEntryHitRT), + }), + }), +]); + +export type GetLogEntriesResponse = rt.TypeOf; diff --git a/x-pack/plugins/security_solution/public/cases/components/configure_cases/mapping.test.tsx b/x-pack/plugins/security_solution/public/cases/components/configure_cases/mapping.test.tsx index c7336d998c4525..bc96aa65b82a7a 100644 --- a/x-pack/plugins/security_solution/public/cases/components/configure_cases/mapping.test.tsx +++ b/x-pack/plugins/security_solution/public/cases/components/configure_cases/mapping.test.tsx @@ -40,7 +40,7 @@ describe('Mapping', () => { wrappingComponent: TestProviders, }); expect(wrapper.find('[data-test-subj="field-mapping-desc"]').first().text()).toBe( - 'Field mappings require an established connection to ServiceNow. Please check your connection credentials.' + 'Field mappings require an established connection to ServiceNow ITSM. Please check your connection credentials.' ); }); }); diff --git a/x-pack/plugins/security_solution/public/cases/components/connectors/config.ts b/x-pack/plugins/security_solution/public/cases/components/connectors/config.ts index 3aca1863788204..a29531d89b405c 100644 --- a/x-pack/plugins/security_solution/public/cases/components/connectors/config.ts +++ b/x-pack/plugins/security_solution/public/cases/components/connectors/config.ts @@ -7,14 +7,14 @@ /* eslint-disable @kbn/eslint/no-restricted-paths */ import { - ServiceNowConnectorConfiguration, + ServiceNowITSMConnectorConfiguration, JiraConnectorConfiguration, ResilientConnectorConfiguration, } from '../../../../../triggers_actions_ui/public/common'; import { ConnectorConfiguration } from './types'; export const connectorsConfiguration: Record = { - '.servicenow': ServiceNowConnectorConfiguration as ConnectorConfiguration, + '.servicenow': ServiceNowITSMConnectorConfiguration as ConnectorConfiguration, '.jira': JiraConnectorConfiguration as ConnectorConfiguration, '.resilient': ResilientConnectorConfiguration as ConnectorConfiguration, }; diff --git a/x-pack/plugins/security_solution/public/cases/components/connectors/types.ts b/x-pack/plugins/security_solution/public/cases/components/connectors/types.ts index 5d83c226bfeca5..00bc01b2ec0a71 100644 --- a/x-pack/plugins/security_solution/public/cases/components/connectors/types.ts +++ b/x-pack/plugins/security_solution/public/cases/components/connectors/types.ts @@ -22,5 +22,4 @@ export interface ThirdPartyField { export interface ConnectorConfiguration extends ActionType { logo: string; - fields: Record; } diff --git a/x-pack/plugins/security_solution/public/detections/containers/detection_engine/alerts/use_privilege_user.tsx b/x-pack/plugins/security_solution/public/detections/containers/detection_engine/alerts/use_privilege_user.tsx index 191c3955caa9ba..c444702312ffb3 100644 --- a/x-pack/plugins/security_solution/public/detections/containers/detection_engine/alerts/use_privilege_user.tsx +++ b/x-pack/plugins/security_solution/public/detections/containers/detection_engine/alerts/use_privilege_user.tsx @@ -62,7 +62,7 @@ export const usePrivilegeUser = (): ReturnPrivilegeUser => { setPrivilegeUser({ isAuthenticated: privilege.is_authenticated, hasEncryptionKey: privilege.has_encryption_key, - hasIndexManage: privilege.index[indexName].manage, + hasIndexManage: privilege.index[indexName].manage && privilege.cluster.manage, hasIndexMaintenance: privilege.index[indexName].maintenance, hasIndexWrite: privilege.index[indexName].create || diff --git a/x-pack/plugins/security_solution/public/timelines/components/notes/add_note/index.tsx b/x-pack/plugins/security_solution/public/timelines/components/notes/add_note/index.tsx index 92e0ac757cc397..10e8ca42f35aee 100644 --- a/x-pack/plugins/security_solution/public/timelines/components/notes/add_note/index.tsx +++ b/x-pack/plugins/security_solution/public/timelines/components/notes/add_note/index.tsx @@ -9,7 +9,6 @@ import { EuiButtonEmpty, EuiFlexGroup, EuiFlexItem, - EuiFocusTrap, EuiScreenReaderOnly, } from '@elastic/eui'; import React, { useCallback } from 'react'; @@ -83,31 +82,29 @@ export const AddNote = React.memo<{ return ( - -
- -

{i18n.YOU_ARE_EDITING_A_NOTE}

- - - - {onCancelAddNote != null ? ( - - - - ) : null} +
+ +

{i18n.YOU_ARE_EDITING_A_NOTE}

+ + + + {onCancelAddNote != null ? ( - - {i18n.ADD_NOTE} - + - -
- + ) : null} + + + {i18n.ADD_NOTE} + + + +
); }); diff --git a/x-pack/plugins/security_solution/scripts/endpoint/resolver_generator_script.ts b/x-pack/plugins/security_solution/scripts/endpoint/resolver_generator_script.ts index 9ad094086b6320..de0cec3c06033e 100644 --- a/x-pack/plugins/security_solution/scripts/endpoint/resolver_generator_script.ts +++ b/x-pack/plugins/security_solution/scripts/endpoint/resolver_generator_script.ts @@ -5,9 +5,10 @@ */ /* eslint-disable no-console */ import yargs from 'yargs'; +import fs from 'fs'; import { Client, ClientOptions } from '@elastic/elasticsearch'; import { ResponseError } from '@elastic/elasticsearch/lib/errors'; -import { KbnClient, ToolingLog } from '@kbn/dev-utils'; +import { KbnClient, ToolingLog, CA_CERT_PATH } from '@kbn/dev-utils'; import { AxiosResponse } from 'axios'; import { indexHostsAndAlerts } from '../../common/endpoint/index_data'; import { ANCESTRY_LIMIT, EndpointDocGenerator } from '../../common/endpoint/generate_data'; @@ -202,15 +203,41 @@ async function main() { type: 'boolean', default: false, }, + ssl: { + alias: 'ssl', + describe: 'Use https for elasticsearch and kbn clients', + type: 'boolean', + default: false, + }, }).argv; + let ca: Buffer; + let kbnClient: KbnClientWithApiKeySupport; + let clientOptions: ClientOptions; - const kbnClient = new KbnClientWithApiKeySupport({ - log: new ToolingLog({ - level: 'info', - writeTo: process.stdout, - }), - url: argv.kibana, - }); + if (argv.ssl) { + ca = fs.readFileSync(CA_CERT_PATH); + const url = argv.kibana.replace('http:', 'https:'); + const node = argv.node.replace('http:', 'https:'); + kbnClient = new KbnClientWithApiKeySupport({ + log: new ToolingLog({ + level: 'info', + writeTo: process.stdout, + }), + url, + certificateAuthorities: [ca], + }); + clientOptions = { node, ssl: { ca: [ca] } }; + } else { + kbnClient = new KbnClientWithApiKeySupport({ + log: new ToolingLog({ + level: 'info', + writeTo: process.stdout, + }), + url: argv.kibana, + }); + clientOptions = { node: argv.node }; + } + const client = new Client(clientOptions); try { await doIngestSetup(kbnClient); @@ -219,9 +246,6 @@ async function main() { process.exit(1); } - const clientOptions: ClientOptions = { node: argv.node }; - const client = new Client(clientOptions); - if (argv.delete) { await deleteIndices( [argv.eventIndex, argv.metadataIndex, argv.policyIndex, argv.alertIndex], diff --git a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/search_after_bulk_create.ts b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/search_after_bulk_create.ts index 3030bd8c52c705..2aa8981cc618bf 100644 --- a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/search_after_bulk_create.ts +++ b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/search_after_bulk_create.ts @@ -70,6 +70,7 @@ export const searchAfterAndBulkCreate = async ({ interval, buildRuleMessage, }); + const tuplesToBeLogged = [...totalToFromTuples]; logger.debug(buildRuleMessage(`totalToFromTuples: ${totalToFromTuples.length}`)); while (totalToFromTuples.length > 0) { @@ -294,5 +295,6 @@ export const searchAfterAndBulkCreate = async ({ } } logger.debug(buildRuleMessage(`[+] completed bulk index of ${toReturn.createdSignalsCount}`)); + toReturn.totalToFromTuples = tuplesToBeLogged; return toReturn; }; diff --git a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts index d08ab66af56836..2b0abdfdfa090b 100644 --- a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts +++ b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/signal_rule_alert_type.ts @@ -670,6 +670,21 @@ export const signalRulesAlertType = ({ lastLookBackDate: result.lastLookBackDate?.toISOString(), }); } + + // adding this log line so we can get some information from cloud + logger.info( + buildRuleMessage( + `[+] Finished indexing ${result.createdSignalsCount} ${ + !isEmpty(result.totalToFromTuples) + ? `signals searched between date ranges ${JSON.stringify( + result.totalToFromTuples, + null, + 2 + )}` + : '' + }` + ) + ); } else { const errorMessage = buildRuleMessage( 'Bulk Indexing of signals failed:', diff --git a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/types.ts b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/types.ts index 5ae411678aa03d..cb955673a7ea6a 100644 --- a/x-pack/plugins/security_solution/server/lib/detection_engine/signals/types.ts +++ b/x-pack/plugins/security_solution/server/lib/detection_engine/signals/types.ts @@ -5,7 +5,7 @@ */ import { DslQuery, Filter } from 'src/plugins/data/common'; -import moment from 'moment'; +import moment, { Moment } from 'moment'; import { Status } from '../../../../common/detection_engine/schemas/common/schemas'; import { RulesSchema } from '../../../../common/detection_engine/schemas/response/rules_schema'; import { @@ -263,6 +263,11 @@ export interface SearchAfterAndBulkCreateReturnType { createdSignalsCount: number; createdSignals: SignalHit[]; errors: string[]; + totalToFromTuples?: Array<{ + to: Moment | undefined; + from: Moment | undefined; + maxSignals: number; + }>; } export interface ThresholdAggregationBucket extends TermAggregationBucket { diff --git a/x-pack/plugins/security_solution/server/search_strategy/security_solution/index.ts b/x-pack/plugins/security_solution/server/search_strategy/security_solution/index.ts index 8b2cce01cf07a6..d25f1aaccc5e7f 100644 --- a/x-pack/plugins/security_solution/server/search_strategy/security_solution/index.ts +++ b/x-pack/plugins/security_solution/server/search_strategy/security_solution/index.ts @@ -37,7 +37,7 @@ export const securitySolutionSearchStrategyProvider = { + const http = httpServiceMock.createStartContract(); + + beforeEach(() => jest.resetAllMocks()); + + describe('getIncidentTypes', () => { + test('should call get choices API', async () => { + const abortCtrl = new AbortController(); + http.post.mockResolvedValueOnce(incidentTypesResponse); + const res = await getIncidentTypes({ + http, + signal: abortCtrl.signal, + connectorId: 'test', + }); + + expect(res).toEqual(incidentTypesResponse); + expect(http.post).toHaveBeenCalledWith('/api/actions/action/test/_execute', { + body: '{"params":{"subAction":"incidentTypes","subActionParams":{}}}', + signal: abortCtrl.signal, + }); + }); + }); + + describe('getSeverity', () => { + test('should call get choices API', async () => { + const abortCtrl = new AbortController(); + http.post.mockResolvedValueOnce(severityResponse); + const res = await getSeverity({ + http, + signal: abortCtrl.signal, + connectorId: 'test', + }); + + expect(res).toEqual(severityResponse); + expect(http.post).toHaveBeenCalledWith('/api/actions/action/test/_execute', { + body: '{"params":{"subAction":"severity","subActionParams":{}}}', + signal: abortCtrl.signal, + }); + }); + }); +}); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/resilient/config.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/resilient/config.ts index a2054585c19b87..9717d594b20ecb 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/resilient/config.ts +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/resilient/config.ts @@ -15,24 +15,4 @@ export const connectorConfiguration = { enabledInConfig: true, enabledInLicense: true, minimumLicenseRequired: 'platinum', - fields: { - name: { - label: i18n.MAPPING_FIELD_NAME, - validSourceFields: ['title', 'description'], - defaultSourceField: 'title', - defaultActionType: 'overwrite', - }, - description: { - label: i18n.MAPPING_FIELD_DESC, - validSourceFields: ['title', 'description'], - defaultSourceField: 'description', - defaultActionType: 'overwrite', - }, - comments: { - label: i18n.MAPPING_FIELD_COMMENTS, - validSourceFields: ['comments'], - defaultSourceField: 'comments', - defaultActionType: 'append', - }, - }, }; diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/api.test.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/api.test.ts new file mode 100644 index 00000000000000..e87b84439f6f8a --- /dev/null +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/api.test.ts @@ -0,0 +1,69 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { httpServiceMock } from '../../../../../../../../src/core/public/mocks'; +import { getChoices } from './api'; + +const choicesResponse = { + status: 'ok', + data: [ + { + dependent_value: '', + label: '1 - Critical', + value: '1', + element: 'priority', + }, + { + dependent_value: '', + label: '2 - High', + value: '2', + element: 'priority', + }, + { + dependent_value: '', + label: '3 - Moderate', + value: '3', + element: 'priority', + }, + { + dependent_value: '', + label: '4 - Low', + value: '4', + element: 'priority', + }, + { + dependent_value: '', + label: '5 - Planning', + value: '5', + element: 'priority', + }, + ], +}; + +describe('ServiceNow API', () => { + const http = httpServiceMock.createStartContract(); + + beforeEach(() => jest.resetAllMocks()); + + describe('getChoices', () => { + test('should call get choices API', async () => { + const abortCtrl = new AbortController(); + http.post.mockResolvedValueOnce(choicesResponse); + const res = await getChoices({ + http, + signal: abortCtrl.signal, + connectorId: 'test', + fields: ['priority'], + }); + + expect(res).toEqual(choicesResponse); + expect(http.post).toHaveBeenCalledWith('/api/actions/action/test/_execute', { + body: '{"params":{"subAction":"getChoices","subActionParams":{"fields":["priority"]}}}', + signal: abortCtrl.signal, + }); + }); + }); +}); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/api.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/api.ts new file mode 100644 index 00000000000000..ecfc66f1b0391c --- /dev/null +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/api.ts @@ -0,0 +1,27 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { HttpSetup } from 'kibana/public'; +import { BASE_ACTION_API_PATH } from '../../../constants'; + +export async function getChoices({ + http, + signal, + connectorId, + fields, +}: { + http: HttpSetup; + signal: AbortSignal; + connectorId: string; + fields: string[]; +}): Promise> { + return await http.post(`${BASE_ACTION_API_PATH}/action/${connectorId}/_execute`, { + body: JSON.stringify({ + params: { subAction: 'getChoices', subActionParams: { fields } }, + }), + signal, + }); +} diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/config.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/config.ts index 7f810cf5eb38fd..6920ee71144a59 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/config.ts +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/config.ts @@ -7,32 +7,24 @@ import * as i18n from './translations'; import logo from './logo.svg'; -export const connectorConfiguration = { +export const serviceNowITSMConfiguration = { id: '.servicenow', - name: i18n.SERVICENOW_TITLE, + name: i18n.SERVICENOW_ITSM_TITLE, + desc: i18n.SERVICENOW_ITSM_DESC, + logo, + enabled: true, + enabledInConfig: true, + enabledInLicense: true, + minimumLicenseRequired: 'platinum', +}; + +export const serviceNowSIRConfiguration = { + id: '.servicenow-sir', + name: i18n.SERVICENOW_SIR_TITLE, + desc: i18n.SERVICENOW_SIR_DESC, logo, enabled: true, enabledInConfig: true, enabledInLicense: true, minimumLicenseRequired: 'platinum', - fields: { - short_description: { - label: i18n.MAPPING_FIELD_SHORT_DESC, - validSourceFields: ['title', 'description'], - defaultSourceField: 'title', - defaultActionType: 'overwrite', - }, - description: { - label: i18n.MAPPING_FIELD_DESC, - validSourceFields: ['title', 'description'], - defaultSourceField: 'description', - defaultActionType: 'overwrite', - }, - comments: { - label: i18n.MAPPING_FIELD_COMMENTS, - validSourceFields: ['comments'], - defaultSourceField: 'comments', - defaultActionType: 'append', - }, - }, }; diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/index.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/index.ts index 65bb3ae4f5a37b..e1f66e506ed8ef 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/index.ts +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/index.ts @@ -4,4 +4,4 @@ * you may not use this file except in compliance with the Elastic License. */ -export { getActionType as getServiceNowActionType } from './servicenow'; +export { getServiceNowITSMActionType, getServiceNowSIRActionType } from './servicenow'; diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.test.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.test.tsx index dfa9bf56cc7a93..ce69f428e10a1e 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.test.tsx +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.test.tsx @@ -8,102 +8,110 @@ import { registerBuiltInActionTypes } from '.././index'; import { ActionTypeModel } from '../../../../types'; import { ServiceNowActionConnector } from './types'; -const ACTION_TYPE_ID = '.servicenow'; -let actionTypeModel: ActionTypeModel; +const SERVICENOW_ITSM_ACTION_TYPE_ID = '.servicenow'; +const SERVICENOW_SIR_ACTION_TYPE_ID = '.servicenow-sir'; +let actionTypeRegistry: TypeRegistry; beforeAll(() => { - const actionTypeRegistry = new TypeRegistry(); + actionTypeRegistry = new TypeRegistry(); registerBuiltInActionTypes({ actionTypeRegistry }); - const getResult = actionTypeRegistry.get(ACTION_TYPE_ID); - if (getResult !== null) { - actionTypeModel = getResult; - } }); describe('actionTypeRegistry.get() works', () => { - test('action type static data is as expected', () => { - expect(actionTypeModel.id).toEqual(ACTION_TYPE_ID); + [SERVICENOW_ITSM_ACTION_TYPE_ID, SERVICENOW_SIR_ACTION_TYPE_ID].forEach((id) => { + test(`${id}: action type static data is as expected`, () => { + const actionTypeModel = actionTypeRegistry.get(id); + expect(actionTypeModel.id).toEqual(id); + }); }); }); describe('servicenow connector validation', () => { - test('connector validation succeeds when connector config is valid', () => { - const actionConnector = { - secrets: { - username: 'user', - password: 'pass', - }, - id: 'test', - actionTypeId: '.servicenow', - name: 'ServiceNow', - isPreconfigured: false, - config: { - apiUrl: 'https://dev94428.service-now.com/', - }, - } as ServiceNowActionConnector; + [SERVICENOW_ITSM_ACTION_TYPE_ID, SERVICENOW_SIR_ACTION_TYPE_ID].forEach((id) => { + test(`${id}: connector validation succeeds when connector config is valid`, () => { + const actionTypeModel = actionTypeRegistry.get(id); + const actionConnector = { + secrets: { + username: 'user', + password: 'pass', + }, + id: 'test', + actionTypeId: id, + name: 'ServiceNow', + isPreconfigured: false, + config: { + apiUrl: 'https://dev94428.service-now.com/', + }, + } as ServiceNowActionConnector; - expect(actionTypeModel.validateConnector(actionConnector)).toEqual({ - config: { - errors: { - apiUrl: [], + expect(actionTypeModel.validateConnector(actionConnector)).toEqual({ + config: { + errors: { + apiUrl: [], + }, }, - }, - secrets: { - errors: { - username: [], - password: [], + secrets: { + errors: { + username: [], + password: [], + }, }, - }, + }); }); - }); - test('connector validation fails when connector config is not valid', () => { - const actionConnector = ({ - secrets: { - username: 'user', - }, - id: '.servicenow', - actionTypeId: '.servicenow', - name: 'servicenow', - config: {}, - } as unknown) as ServiceNowActionConnector; + test(`${id}: connector validation fails when connector config is not valid`, () => { + const actionTypeModel = actionTypeRegistry.get(id); + const actionConnector = ({ + secrets: { + username: 'user', + }, + id, + actionTypeId: id, + name: 'servicenow', + config: {}, + } as unknown) as ServiceNowActionConnector; - expect(actionTypeModel.validateConnector(actionConnector)).toEqual({ - config: { - errors: { - apiUrl: ['URL is required.'], + expect(actionTypeModel.validateConnector(actionConnector)).toEqual({ + config: { + errors: { + apiUrl: ['URL is required.'], + }, }, - }, - secrets: { - errors: { - username: [], - password: ['Password is required.'], + secrets: { + errors: { + username: [], + password: ['Password is required.'], + }, }, - }, + }); }); }); }); describe('servicenow action params validation', () => { - test('action params validation succeeds when action params is valid', () => { - const actionParams = { - subActionParams: { incident: { short_description: 'some title {{test}}' }, comments: [] }, - }; + [SERVICENOW_ITSM_ACTION_TYPE_ID, SERVICENOW_SIR_ACTION_TYPE_ID].forEach((id) => { + test(`${id}: action params validation succeeds when action params is valid`, () => { + const actionTypeModel = actionTypeRegistry.get(id); + const actionParams = { + subActionParams: { incident: { short_description: 'some title {{test}}' }, comments: [] }, + }; - expect(actionTypeModel.validateParams(actionParams)).toEqual({ - errors: { ['subActionParams.incident.short_description']: [] }, + expect(actionTypeModel.validateParams(actionParams)).toEqual({ + errors: { ['subActionParams.incident.short_description']: [] }, + }); }); - }); - test('params validation fails when body is not valid', () => { - const actionParams = { - subActionParams: { incident: { short_description: '' }, comments: [] }, - }; + test(`${id}: params validation fails when body is not valid`, () => { + const actionTypeModel = actionTypeRegistry.get(id); + const actionParams = { + subActionParams: { incident: { short_description: '' }, comments: [] }, + }; - expect(actionTypeModel.validateParams(actionParams)).toEqual({ - errors: { - ['subActionParams.incident.short_description']: ['Short description is required.'], - }, + expect(actionTypeModel.validateParams(actionParams)).toEqual({ + errors: { + ['subActionParams.incident.short_description']: ['Short description is required.'], + }, + }); }); }); }); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.tsx index 4389abff72fcda..1b968cfff5d014 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.tsx +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow.tsx @@ -10,13 +10,14 @@ import { ActionTypeModel, ConnectorValidationResult, } from '../../../../types'; -import { connectorConfiguration } from './config'; +import { serviceNowITSMConfiguration, serviceNowSIRConfiguration } from './config'; import logo from './logo.svg'; import { ServiceNowActionConnector, ServiceNowConfig, ServiceNowSecrets, - ServiceNowActionParams, + ServiceNowITSMActionParams, + ServiceNowSIRActionParams, } from './types'; import * as i18n from './translations'; import { isValidUrl } from '../../../lib/value_validators'; @@ -60,19 +61,21 @@ const validateConnector = ( return validationResult; }; -export function getActionType(): ActionTypeModel< +export function getServiceNowITSMActionType(): ActionTypeModel< ServiceNowConfig, ServiceNowSecrets, - ServiceNowActionParams + ServiceNowITSMActionParams > { return { - id: connectorConfiguration.id, + id: serviceNowITSMConfiguration.id, iconClass: logo, - selectMessage: i18n.SERVICENOW_DESC, - actionTypeTitle: connectorConfiguration.name, + selectMessage: serviceNowITSMConfiguration.desc, + actionTypeTitle: serviceNowITSMConfiguration.name, validateConnector, actionConnectorFields: lazy(() => import('./servicenow_connectors')), - validateParams: (actionParams: ServiceNowActionParams): GenericValidationResult => { + validateParams: ( + actionParams: ServiceNowITSMActionParams + ): GenericValidationResult => { const errors = { // eslint-disable-next-line @typescript-eslint/naming-convention 'subActionParams.incident.short_description': new Array(), @@ -89,6 +92,39 @@ export function getActionType(): ActionTypeModel< } return validationResult; }, - actionParamsFields: lazy(() => import('./servicenow_params')), + actionParamsFields: lazy(() => import('./servicenow_itsm_params')), + }; +} + +export function getServiceNowSIRActionType(): ActionTypeModel< + ServiceNowConfig, + ServiceNowSecrets, + ServiceNowSIRActionParams +> { + return { + id: serviceNowSIRConfiguration.id, + iconClass: logo, + selectMessage: serviceNowSIRConfiguration.desc, + actionTypeTitle: serviceNowSIRConfiguration.name, + validateConnector, + actionConnectorFields: lazy(() => import('./servicenow_connectors')), + validateParams: (actionParams: ServiceNowSIRActionParams): GenericValidationResult => { + const errors = { + // eslint-disable-next-line @typescript-eslint/naming-convention + 'subActionParams.incident.short_description': new Array(), + }; + const validationResult = { + errors, + }; + if ( + actionParams.subActionParams && + actionParams.subActionParams.incident && + !actionParams.subActionParams.incident.short_description?.length + ) { + errors['subActionParams.incident.short_description'].push(i18n.TITLE_REQUIRED); + } + return validationResult; + }, + actionParamsFields: lazy(() => import('./servicenow_sir_params')), }; } diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_params.test.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_itsm_params.test.tsx similarity index 53% rename from x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_params.test.tsx rename to x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_itsm_params.test.tsx index 5519d7498a85ed..51318e14a2cfdb 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_params.test.tsx +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_itsm_params.test.tsx @@ -5,8 +5,18 @@ */ import React from 'react'; import { mount } from 'enzyme'; -import ServiceNowParamsFields from './servicenow_params'; +import { act } from '@testing-library/react'; + import { ActionConnector } from '../../../../types'; +import { useGetChoices } from './use_get_choices'; +import ServiceNowITSMParamsFields from './servicenow_itsm_params'; +import { Choice } from './types'; + +jest.mock('./use_get_choices'); +jest.mock('../../../../common/lib/kibana'); + +const useGetChoicesMock = useGetChoices as jest.Mock; + const actionParams = { subAction: 'pushToService', subActionParams: { @@ -16,7 +26,6 @@ const actionParams = { severity: '1', urgency: '2', impact: '3', - savedObjectId: '123', externalId: null, }, comments: [], @@ -31,6 +40,7 @@ const connector: ActionConnector = { name: 'Test', isPreconfigured: false, }; + const editAction = jest.fn(); const defaultProps = { actionConnector: connector, @@ -40,31 +50,71 @@ const defaultProps = { index: 0, messageVariables: [], }; -describe('ServiceNowParamsFields renders', () => { + +const useGetChoicesResponse = { + isLoading: false, + choices: ['severity', 'urgency', 'impact'] + .map((element) => [ + { + dependent_value: '', + label: '1 - Critical', + value: '1', + element, + }, + { + dependent_value: '', + label: '2 - High', + value: '2', + element, + }, + { + dependent_value: '', + label: '3 - Moderate', + value: '3', + element, + }, + { + dependent_value: '', + label: '4 - Low', + value: '4', + element, + }, + ]) + .flat(), +}; + +describe('ServiceNowITSMParamsFields renders', () => { + let onChoices = (choices: Choice[]) => {}; + beforeEach(() => { jest.clearAllMocks(); + useGetChoicesMock.mockImplementation((args) => { + onChoices = args.onSuccess; + return useGetChoicesResponse; + }); }); + test('all params fields is rendered', () => { - const wrapper = mount(); - expect(wrapper.find('[data-test-subj="urgencySelect"]').length > 0).toBeTruthy(); - expect(wrapper.find('[data-test-subj="severitySelect"]').first().prop('value')).toStrictEqual( - '1' - ); - expect(wrapper.find('[data-test-subj="impactSelect"]').length > 0).toBeTruthy(); - expect(wrapper.find('[data-test-subj="short_descriptionInput"]').length > 0).toBeTruthy(); - expect(wrapper.find('[data-test-subj="descriptionTextArea"]').length > 0).toBeTruthy(); - expect(wrapper.find('[data-test-subj="commentsTextArea"]').length > 0).toBeTruthy(); + const wrapper = mount(); + expect(wrapper.find('[data-test-subj="urgencySelect"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="severitySelect"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="impactSelect"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="short_descriptionInput"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="descriptionTextArea"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="commentsTextArea"]').exists()).toBeTruthy(); }); + test('If short_description has errors, form row is invalid', () => { const newProps = { ...defaultProps, // eslint-disable-next-line @typescript-eslint/naming-convention errors: { 'subActionParams.incident.short_description': ['error'] }, }; - const wrapper = mount(); + const wrapper = mount(); const title = wrapper.find('[data-test-subj="short_descriptionInput"]').first(); expect(title.prop('isInvalid')).toBeTruthy(); }); + test('When subActionParams is undefined, set to default', () => { const { subActionParams, ...newParams } = actionParams; @@ -72,12 +122,13 @@ describe('ServiceNowParamsFields renders', () => { ...defaultProps, actionParams: newParams, }; - mount(); + mount(); expect(editAction.mock.calls[0][1]).toEqual({ incident: {}, comments: [], }); }); + test('When subAction is undefined, set to default', () => { const { subAction, ...newParams } = actionParams; @@ -85,11 +136,12 @@ describe('ServiceNowParamsFields renders', () => { ...defaultProps, actionParams: newParams, }; - mount(); + mount(); expect(editAction.mock.calls[0][1]).toEqual('pushToService'); }); + test('Resets fields when connector changes', () => { - const wrapper = mount(); + const wrapper = mount(); expect(editAction.mock.calls.length).toEqual(0); wrapper.setProps({ actionConnector: { ...connector, id: '1234' } }); expect(editAction.mock.calls.length).toEqual(1); @@ -98,6 +150,52 @@ describe('ServiceNowParamsFields renders', () => { comments: [], }); }); + + test('it transforms the urgencies to options correctly', async () => { + const wrapper = mount(); + act(() => { + onChoices(useGetChoicesResponse.choices); + }); + + wrapper.update(); + expect(wrapper.find('[data-test-subj="urgencySelect"]').first().prop('options')).toEqual([ + { value: '1', text: '1 - Critical' }, + { value: '2', text: '2 - High' }, + { value: '3', text: '3 - Moderate' }, + { value: '4', text: '4 - Low' }, + ]); + }); + + test('it transforms the severities to options correctly', async () => { + const wrapper = mount(); + act(() => { + onChoices(useGetChoicesResponse.choices); + }); + + wrapper.update(); + expect(wrapper.find('[data-test-subj="severitySelect"]').first().prop('options')).toEqual([ + { value: '1', text: '1 - Critical' }, + { value: '2', text: '2 - High' }, + { value: '3', text: '3 - Moderate' }, + { value: '4', text: '4 - Low' }, + ]); + }); + + test('it transforms the impacts to options correctly', async () => { + const wrapper = mount(); + act(() => { + onChoices(useGetChoicesResponse.choices); + }); + + wrapper.update(); + expect(wrapper.find('[data-test-subj="impactSelect"]').first().prop('options')).toEqual([ + { value: '1', text: '1 - Critical' }, + { value: '2', text: '2 - High' }, + { value: '3', text: '3 - Moderate' }, + { value: '4', text: '4 - Low' }, + ]); + }); + describe('UI updates', () => { const changeEvent = { target: { value: 'Bug' } } as React.ChangeEvent; const simpleFields = [ @@ -107,22 +205,25 @@ describe('ServiceNowParamsFields renders', () => { { dataTestSubj: '[data-test-subj="severitySelect"]', key: 'severity' }, { dataTestSubj: '[data-test-subj="impactSelect"]', key: 'impact' }, ]; + simpleFields.forEach((field) => test(`${field.key} update triggers editAction :D`, () => { - const wrapper = mount(); + const wrapper = mount(); const theField = wrapper.find(field.dataTestSubj).first(); theField.prop('onChange')!(changeEvent); expect(editAction.mock.calls[0][1].incident[field.key]).toEqual(changeEvent.target.value); }) ); + test('A comment triggers editAction', () => { - const wrapper = mount(); + const wrapper = mount(); const comments = wrapper.find('textarea[data-test-subj="commentsTextArea"]'); expect(comments.simulate('change', changeEvent)); expect(editAction.mock.calls[0][1].comments.length).toEqual(1); }); + test('An empty comment does not trigger editAction', () => { - const wrapper = mount(); + const wrapper = mount(); const emptyComment = { target: { value: '' } }; const comments = wrapper.find('[data-test-subj="commentsTextArea"] textarea'); expect(comments.simulate('change', emptyComment)); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_params.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_itsm_params.tsx similarity index 64% rename from x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_params.tsx rename to x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_itsm_params.tsx index 3e6b443d790a99..658b964f8b91d8 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_params.tsx +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_itsm_params.tsx @@ -4,8 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -import React, { Fragment, useCallback, useEffect, useMemo, useRef } from 'react'; -import { i18n } from '@kbn/i18n'; +import React, { Fragment, useCallback, useEffect, useMemo, useRef, useState } from 'react'; import { EuiFormRow, EuiSelect, @@ -14,38 +13,29 @@ import { EuiSpacer, EuiTitle, } from '@elastic/eui'; +import { useKibana } from '../../../../common/lib/kibana'; import { ActionParamsProps } from '../../../../types'; -import { ServiceNowActionParams } from './types'; +import { ServiceNowITSMActionParams, Choice, Options } from './types'; import { TextAreaWithMessageVariables } from '../../text_area_with_message_variables'; import { TextFieldWithMessageVariables } from '../../text_field_with_message_variables'; +import { useGetChoices } from './use_get_choices'; +import * as i18n from './translations'; -const selectOptions = [ - { - value: '1', - text: i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.severitySelectHighOptionLabel', - { defaultMessage: 'High' } - ), - }, - { - value: '2', - text: i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.severitySelectMediumOptionLabel', - { defaultMessage: 'Medium' } - ), - }, - { - value: '3', - text: i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.severitySelectLawOptionLabel', - { defaultMessage: 'Low' } - ), - }, -]; +const useGetChoicesFields = ['urgency', 'severity', 'impact']; +const defaultOptions: Options = { + urgency: [], + severity: [], + impact: [], +}; const ServiceNowParamsFields: React.FunctionComponent< - ActionParamsProps + ActionParamsProps > = ({ actionConnector, actionParams, editAction, index, errors, messageVariables }) => { + const { + http, + notifications: { toasts }, + } = useKibana().services; + const actionConnectorRef = useRef(actionConnector?.id ?? ''); const { incident, comments } = useMemo( () => @@ -53,10 +43,12 @@ const ServiceNowParamsFields: React.FunctionComponent< (({ incident: {}, comments: [], - } as unknown) as ServiceNowActionParams['subActionParams']), + } as unknown) as ServiceNowITSMActionParams['subActionParams']), [actionParams.subActionParams] ); + const [options, setOptions] = useState(defaultOptions); + const editSubActionProperty = useCallback( (key: string, value: any) => { const newProps = @@ -80,6 +72,28 @@ const ServiceNowParamsFields: React.FunctionComponent< [editSubActionProperty] ); + const onChoicesSuccess = (choices: Choice[]) => + setOptions( + choices.reduce( + (acc, choice) => ({ + ...acc, + [choice.element]: [ + ...(acc[choice.element] != null ? acc[choice.element] : []), + { value: choice.value, text: choice.label }, + ], + }), + defaultOptions + ) + ); + + const { isLoading: isLoadingChoices } = useGetChoices({ + http, + toastNotifications: toasts, + actionConnector, + fields: useGetChoicesFields, + onSuccess: onChoicesSuccess, + }); + useEffect(() => { if (actionConnector != null && actionConnectorRef.current !== actionConnector.id) { actionConnectorRef.current = actionConnector.id; @@ -94,6 +108,7 @@ const ServiceNowParamsFields: React.FunctionComponent< } // eslint-disable-next-line react-hooks/exhaustive-deps }, [actionConnector]); + useEffect(() => { if (!actionParams.subAction) { editAction('subAction', 'pushToService', index); @@ -114,64 +129,47 @@ const ServiceNowParamsFields: React.FunctionComponent< return ( -

- {i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.title', - { defaultMessage: 'Incident' } - )} -

+

{i18n.INCIDENT}

 - + editSubActionProperty('urgency', e.target.value)} /> - + editSubActionProperty('severity', e.target.value)} /> - + editSubActionProperty('impact', e.target.value)} /> @@ -185,10 +183,7 @@ const ServiceNowParamsFields: React.FunctionComponent< errors['subActionParams.incident.short_description'].length > 0 && incident.short_description !== undefined } - label={i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.titleFieldLabel', - { defaultMessage: 'Short description (required)' } - )} + label={i18n.SHORT_DESCRIPTION_LABEL} > 0 ? comments[0].comment : undefined} - label={i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.serviceNow.commentsTextAreaFieldLabel', - { defaultMessage: 'Additional comments' } - )} + label={i18n.COMMENTS_LABEL} /> ); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_sir_params.test.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_sir_params.test.tsx new file mode 100644 index 00000000000000..72dfd63da3d4e7 --- /dev/null +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_sir_params.test.tsx @@ -0,0 +1,311 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ +import React from 'react'; +import { mount } from 'enzyme'; +import { act } from '@testing-library/react'; + +import { ActionConnector } from '../../../../types'; +import { useGetChoices } from './use_get_choices'; +import ServiceNowSIRParamsFields from './servicenow_sir_params'; +import { Choice } from './types'; + +jest.mock('./use_get_choices'); +jest.mock('../../../../common/lib/kibana'); + +const useGetChoicesMock = useGetChoices as jest.Mock; + +const actionParams = { + subAction: 'pushToService', + subActionParams: { + incident: { + short_description: 'sn title', + description: 'some description', + category: 'Denial of Service', + dest_ip: '192.168.1.1', + source_ip: '192.168.1.2', + malware_hash: '098f6bcd4621d373cade4e832627b4f6', + malware_url: 'https://attack.com', + priority: '1', + subcategory: '20', + externalId: null, + }, + comments: [], + }, +}; + +const connector: ActionConnector = { + secrets: {}, + config: {}, + id: 'test', + actionTypeId: '.test', + name: 'Test', + isPreconfigured: false, +}; + +const editAction = jest.fn(); +const defaultProps = { + actionConnector: connector, + actionParams, + errors: { ['subActionParams.incident.short_description']: [] }, + editAction, + index: 0, + messageVariables: [], +}; + +const choicesResponse = { + isLoading: false, + choices: [ + { + dependent_value: '', + label: 'Priviledge Escalation', + value: 'Priviledge Escalation', + element: 'category', + }, + { + dependent_value: '', + label: 'Criminal activity/investigation', + value: 'Criminal activity/investigation', + element: 'category', + }, + { + dependent_value: '', + label: 'Denial of Service', + value: 'Denial of Service', + element: 'category', + }, + { + dependent_value: 'Denial of Service', + label: 'Inbound or outbound', + value: '12', + element: 'subcategory', + }, + { + dependent_value: 'Denial of Service', + label: 'Single or distributed (DoS or DDoS)', + value: '26', + element: 'subcategory', + }, + { + dependent_value: 'Denial of Service', + label: 'Inbound DDos', + value: 'inbound_ddos', + element: 'subcategory', + }, + { + dependent_value: '', + label: '1 - Critical', + value: '1', + element: 'priority', + }, + { + dependent_value: '', + label: '2 - High', + value: '2', + element: 'priority', + }, + { + dependent_value: '', + label: '3 - Moderate', + value: '3', + element: 'priority', + }, + { + dependent_value: '', + label: '4 - Low', + value: '4', + element: 'priority', + }, + { + dependent_value: '', + label: '5 - Planning', + value: '5', + element: 'priority', + }, + ], +}; + +describe('ServiceNowSIRParamsFields renders', () => { + let onChoicesSuccess = (choices: Choice[]) => {}; + + beforeEach(() => { + jest.clearAllMocks(); + useGetChoicesMock.mockImplementation((args) => { + onChoicesSuccess = args.onSuccess; + return choicesResponse; + }); + }); + + test('all params fields is rendered', () => { + const wrapper = mount(); + expect(wrapper.find('[data-test-subj="short_descriptionInput"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="source_ipInput"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="dest_ipInput"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="malware_urlInput"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="malware_hashInput"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="prioritySelect"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="categorySelect"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="subcategorySelect"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="descriptionTextArea"]').exists()).toBeTruthy(); + expect(wrapper.find('[data-test-subj="commentsTextArea"]').exists()).toBeTruthy(); + }); + + test('If short_description has errors, form row is invalid', () => { + const newProps = { + ...defaultProps, + // eslint-disable-next-line @typescript-eslint/naming-convention + errors: { 'subActionParams.incident.short_description': ['error'] }, + }; + const wrapper = mount(); + const title = wrapper.find('[data-test-subj="short_descriptionInput"]').first(); + expect(title.prop('isInvalid')).toBeTruthy(); + }); + + test('When subActionParams is undefined, set to default', () => { + const { subActionParams, ...newParams } = actionParams; + + const newProps = { + ...defaultProps, + actionParams: newParams, + }; + mount(); + expect(editAction.mock.calls[0][1]).toEqual({ + incident: {}, + comments: [], + }); + }); + + test('When subAction is undefined, set to default', () => { + const { subAction, ...newParams } = actionParams; + + const newProps = { + ...defaultProps, + actionParams: newParams, + }; + mount(); + expect(editAction.mock.calls[0][1]).toEqual('pushToService'); + }); + + test('Resets fields when connector changes', () => { + const wrapper = mount(); + expect(editAction.mock.calls.length).toEqual(0); + wrapper.setProps({ actionConnector: { ...connector, id: '1234' } }); + expect(editAction.mock.calls.length).toEqual(1); + expect(editAction.mock.calls[0][1]).toEqual({ + incident: {}, + comments: [], + }); + }); + + test('it transforms the categories to options correctly', async () => { + const wrapper = mount(); + act(() => { + onChoicesSuccess(choicesResponse.choices); + }); + + wrapper.update(); + expect(wrapper.find('[data-test-subj="categorySelect"]').first().prop('options')).toEqual([ + { value: 'Priviledge Escalation', text: 'Priviledge Escalation' }, + { + value: 'Criminal activity/investigation', + text: 'Criminal activity/investigation', + }, + { value: 'Denial of Service', text: 'Denial of Service' }, + ]); + }); + + test('it transforms the subcategories to options correctly', async () => { + const wrapper = mount(); + act(() => { + onChoicesSuccess(choicesResponse.choices); + }); + + wrapper.update(); + expect(wrapper.find('[data-test-subj="subcategorySelect"]').first().prop('options')).toEqual([ + { + text: 'Inbound or outbound', + value: '12', + }, + { + text: 'Single or distributed (DoS or DDoS)', + value: '26', + }, + { + text: 'Inbound DDos', + value: 'inbound_ddos', + }, + ]); + }); + + test('it transforms the priorities to options correctly', async () => { + const wrapper = mount(); + act(() => { + onChoicesSuccess(choicesResponse.choices); + }); + + wrapper.update(); + expect(wrapper.find('[data-test-subj="prioritySelect"]').first().prop('options')).toEqual([ + { + text: '1 - Critical', + value: '1', + }, + { + text: '2 - High', + value: '2', + }, + { + text: '3 - Moderate', + value: '3', + }, + { + text: '4 - Low', + value: '4', + }, + { + text: '5 - Planning', + value: '5', + }, + ]); + }); + + describe('UI updates', () => { + const changeEvent = { target: { value: 'Bug' } } as React.ChangeEvent; + const simpleFields = [ + { dataTestSubj: 'input[data-test-subj="short_descriptionInput"]', key: 'short_description' }, + { dataTestSubj: 'textarea[data-test-subj="descriptionTextArea"]', key: 'description' }, + { dataTestSubj: '[data-test-subj="source_ipInput"]', key: 'source_ip' }, + { dataTestSubj: '[data-test-subj="dest_ipInput"]', key: 'dest_ip' }, + { dataTestSubj: '[data-test-subj="malware_urlInput"]', key: 'malware_url' }, + { dataTestSubj: '[data-test-subj="malware_hashInput"]', key: 'malware_hash' }, + { dataTestSubj: '[data-test-subj="prioritySelect"]', key: 'priority' }, + { dataTestSubj: '[data-test-subj="categorySelect"]', key: 'category' }, + { dataTestSubj: '[data-test-subj="subcategorySelect"]', key: 'subcategory' }, + ]; + + simpleFields.forEach((field) => + test(`${field.key} update triggers editAction :D`, () => { + const wrapper = mount(); + const theField = wrapper.find(field.dataTestSubj).first(); + theField.prop('onChange')!(changeEvent); + expect(editAction.mock.calls[0][1].incident[field.key]).toEqual(changeEvent.target.value); + }) + ); + + test('A comment triggers editAction', () => { + const wrapper = mount(); + const comments = wrapper.find('textarea[data-test-subj="commentsTextArea"]'); + expect(comments.simulate('change', changeEvent)); + expect(editAction.mock.calls[0][1].comments.length).toEqual(1); + }); + + test('An empty comment does not trigger editAction', () => { + const wrapper = mount(); + const emptyComment = { target: { value: '' } }; + const comments = wrapper.find('[data-test-subj="commentsTextArea"] textarea'); + expect(comments.simulate('change', emptyComment)); + expect(editAction.mock.calls.length).toEqual(0); + }); + }); +}); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_sir_params.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_sir_params.tsx new file mode 100644 index 00000000000000..26957d828f5e0b --- /dev/null +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/servicenow_sir_params.tsx @@ -0,0 +1,295 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React, { Fragment, useCallback, useEffect, useMemo, useRef, useState } from 'react'; +import { + EuiFormRow, + EuiSelect, + EuiFlexGroup, + EuiFlexItem, + EuiSpacer, + EuiTitle, + EuiSelectOption, +} from '@elastic/eui'; +import { useKibana } from '../../../../common/lib/kibana'; +import { ActionParamsProps } from '../../../../types'; +import { TextAreaWithMessageVariables } from '../../text_area_with_message_variables'; +import { TextFieldWithMessageVariables } from '../../text_field_with_message_variables'; + +import * as i18n from './translations'; +import { useGetChoices } from './use_get_choices'; +import { ServiceNowSIRActionParams, Fields, Choice } from './types'; + +const useGetChoicesFields = ['category', 'subcategory', 'priority']; +const defaultFields: Fields = { + category: [], + subcategory: [], + priority: [], +}; + +const choicesToEuiOptions = (choices: Choice[]): EuiSelectOption[] => + choices.map((choice) => ({ value: choice.value, text: choice.label })); + +const ServiceNowSIRParamsFields: React.FunctionComponent< + ActionParamsProps +> = ({ actionConnector, actionParams, editAction, index, errors, messageVariables }) => { + const { + http, + notifications: { toasts }, + } = useKibana().services; + + const actionConnectorRef = useRef(actionConnector?.id ?? ''); + const { incident, comments } = useMemo( + () => + actionParams.subActionParams ?? + (({ + incident: {}, + comments: [], + } as unknown) as ServiceNowSIRActionParams['subActionParams']), + [actionParams.subActionParams] + ); + + const [choices, setChoices] = useState(defaultFields); + + const editSubActionProperty = useCallback( + (key: string, value: any) => { + const newProps = + key !== 'comments' + ? { + incident: { ...incident, [key]: value }, + comments, + } + : { incident, [key]: value }; + editAction('subActionParams', newProps, index); + }, + [comments, editAction, incident, index] + ); + + const editComment = useCallback( + (key, value) => { + if (value.length > 0) { + editSubActionProperty(key, [{ commentId: '1', comment: value }]); + } + }, + [editSubActionProperty] + ); + + const onChoicesSuccess = useCallback((values: Choice[]) => { + setChoices( + values.reduce( + (acc, value) => ({ + ...acc, + [value.element]: [...(acc[value.element] != null ? acc[value.element] : []), value], + }), + defaultFields + ) + ); + }, []); + + const { isLoading: isLoadingChoices } = useGetChoices({ + http, + toastNotifications: toasts, + actionConnector, + // Not having a memoized fields variable will cause infinitive API calls. + fields: useGetChoicesFields, + onSuccess: onChoicesSuccess, + }); + + const categoryOptions = useMemo(() => choicesToEuiOptions(choices.category), [choices.category]); + const priorityOptions = useMemo(() => choicesToEuiOptions(choices.priority), [choices.priority]); + + const subcategoryOptions = useMemo( + () => + choicesToEuiOptions( + choices.subcategory.filter( + (subcategory) => subcategory.dependent_value === incident.category + ) + ), + [choices.subcategory, incident.category] + ); + + useEffect(() => { + if (actionConnector != null && actionConnectorRef.current !== actionConnector.id) { + actionConnectorRef.current = actionConnector.id; + editAction( + 'subActionParams', + { + incident: {}, + comments: [], + }, + index + ); + } + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [actionConnector]); + + useEffect(() => { + if (!actionParams.subAction) { + editAction('subAction', 'pushToService', index); + } + if (!actionParams.subActionParams) { + editAction( + 'subActionParams', + { + incident: {}, + comments: [], + }, + index + ); + } + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [actionParams]); + + return ( + + +

{i18n.INCIDENT}

+ + + 0 && + incident.short_description !== undefined + } + label={i18n.SHORT_DESCRIPTION_LABEL} + > + + + + + + + + + + + + + + + + + + + + + { + editAction( + 'subActionParams', + { + incident: { ...incident, priority: e.target.value }, + comments, + }, + index + ); + }} + /> + + + + + + { + editAction( + 'subActionParams', + { + incident: { ...incident, category: e.target.value, subcategory: null }, + comments, + }, + index + ); + }} + /> + + + + + editSubActionProperty('subcategory', e.target.value)} + /> + + + + + + 0 ? comments[0].comment : undefined} + label={i18n.COMMENTS_LABEL} + /> + + ); +}; + +// eslint-disable-next-line import/no-default-export +export { ServiceNowSIRParamsFields as default }; diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/translations.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/translations.ts index c84a916c0fef4e..c8bc2f427bde2a 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/translations.ts +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/translations.ts @@ -6,17 +6,31 @@ import { i18n } from '@kbn/i18n'; -export const SERVICENOW_DESC = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.selectMessageText', +export const SERVICENOW_ITSM_DESC = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.serviceNowITSM.selectMessageText', { - defaultMessage: 'Create an incident in ServiceNow.', + defaultMessage: 'Create an incident in ServiceNow ITSM.', } ); -export const SERVICENOW_TITLE = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.actionTypeTitle', +export const SERVICENOW_SIR_DESC = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.serviceNowSIR.selectMessageText', { - defaultMessage: 'ServiceNow', + defaultMessage: 'Create an incident in ServiceNow SIR.', + } +); + +export const SERVICENOW_ITSM_TITLE = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.serviceNowITSM.actionTypeTitle', + { + defaultMessage: 'ServiceNow ITSM', + } +); + +export const SERVICENOW_SIR_TITLE = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.serviceNowSIR.actionTypeTitle', + { + defaultMessage: 'ServiceNow SIR', } ); @@ -98,65 +112,114 @@ export const PASSWORD_REQUIRED = i18n.translate( } ); -export const API_TOKEN_LABEL = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.apiTokenTextFieldLabel', +export const TITLE_REQUIRED = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.common.requiredShortDescTextField', { - defaultMessage: 'Api token', + defaultMessage: 'Short description is required.', } ); -export const API_TOKEN_REQUIRED = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requiredApiTokenTextField', +export const SOURCE_IP_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.sourceIPTitle', { - defaultMessage: 'Api token is required.', + defaultMessage: 'Source IP', } ); -export const EMAIL_LABEL = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.emailTextFieldLabel', +export const DEST_IP_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.destinationIPTitle', { - defaultMessage: 'Email', + defaultMessage: 'Destination IP', } ); -export const EMAIL_REQUIRED = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.requiredEmailTextField', +export const INCIDENT = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.title', { - defaultMessage: 'Email is required.', + defaultMessage: 'Incident', } ); -export const MAPPING_FIELD_SHORT_DESC = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.mappingFieldShortDescription', +export const SHORT_DESCRIPTION_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.titleFieldLabel', { - defaultMessage: 'Short Description', + defaultMessage: 'Short description (required)', } ); -export const MAPPING_FIELD_DESC = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.mappingFieldDescription', +export const DESCRIPTION_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.descriptionTextAreaFieldLabel', { defaultMessage: 'Description', } ); -export const MAPPING_FIELD_COMMENTS = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.mappingFieldComments', +export const COMMENTS_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.commentsTextAreaFieldLabel', { - defaultMessage: 'Comments', + defaultMessage: 'Additional comments', } ); -export const DESCRIPTION_REQUIRED = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.common.requiredDescriptionTextField', +export const MALWARE_URL_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.malwareURLTitle', { - defaultMessage: 'Description is required.', + defaultMessage: 'Malware URL', } ); -export const TITLE_REQUIRED = i18n.translate( - 'xpack.triggersActionsUI.components.builtinActionTypes.common.requiredShortDescTextField', +export const MALWARE_HASH_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.malwareHashTitle', { - defaultMessage: 'Short description is required.', + defaultMessage: 'Malware hash', + } +); + +export const CHOICES_API_ERROR = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.unableToGetChoicesMessage', + { + defaultMessage: 'Unable to get choices', + } +); + +export const CATEGORY_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.categoryTitle', + { + defaultMessage: 'Category', + } +); + +export const SUBCATEGORY_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.subcategoryTitle', + { + defaultMessage: 'Subcategory', + } +); + +export const URGENCY_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.urgencySelectFieldLabel', + { + defaultMessage: 'Urgency', + } +); + +export const SEVERITY_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.severitySelectFieldLabel', + { + defaultMessage: 'Severity', + } +); + +export const IMPACT_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.impactSelectFieldLabel', + { + defaultMessage: 'Impact', + } +); + +export const PRIORITY_LABEL = i18n.translate( + 'xpack.triggersActionsUI.components.builtinActionTypes.servicenow.prioritySelectFieldLabel', + { + defaultMessage: 'Priority', } ); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/types.ts b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/types.ts index ae03680a805341..be9a7c634af8a9 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/types.ts +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/types.ts @@ -4,18 +4,27 @@ * you may not use this file except in compliance with the Elastic License. */ +import { EuiSelectOption } from '@elastic/eui'; import { UserConfiguredActionConnector } from '../../../../types'; -// eslint-disable-next-line @kbn/eslint/no-restricted-paths -import { ExecutorSubActionPushParams } from '../../../../../../actions/server/builtin_action_types/servicenow/types'; +import { + ExecutorSubActionPushParamsITSM, + ExecutorSubActionPushParamsSIR, + // eslint-disable-next-line @kbn/eslint/no-restricted-paths +} from '../../../../../../actions/server/builtin_action_types/servicenow/types'; export type ServiceNowActionConnector = UserConfiguredActionConnector< ServiceNowConfig, ServiceNowSecrets >; -export interface ServiceNowActionParams { +export interface ServiceNowITSMActionParams { subAction: string; - subActionParams: ExecutorSubActionPushParams; + subActionParams: ExecutorSubActionPushParamsITSM; +} + +export interface ServiceNowSIRActionParams { + subAction: string; + subActionParams: ExecutorSubActionPushParamsSIR; } export interface ServiceNowConfig { @@ -26,3 +35,13 @@ export interface ServiceNowSecrets { username: string; password: string; } + +export interface Choice { + value: string; + label: string; + element: string; + dependent_value: string; +} + +export type Fields = Record; +export type Options = Record; diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/use_get_choices.test.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/use_get_choices.test.tsx new file mode 100644 index 00000000000000..4e8061ebaa6e99 --- /dev/null +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/use_get_choices.test.tsx @@ -0,0 +1,164 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { renderHook } from '@testing-library/react-hooks'; + +import { useKibana } from '../../../../common/lib/kibana'; +import { ActionConnector } from '../../../../types'; +import { useGetChoices, UseGetChoices, UseGetChoicesProps } from './use_get_choices'; +import { getChoices } from './api'; + +jest.mock('./api'); +jest.mock('../../../../common/lib/kibana'); + +const useKibanaMock = useKibana as jest.Mocked; +const getChoicesMock = getChoices as jest.Mock; +const onSuccess = jest.fn(); + +const actionConnector = { + secrets: { + username: 'user', + password: 'pass', + }, + id: 'test', + actionTypeId: '.servicenow', + name: 'ServiceNow ITSM', + isPreconfigured: false, + config: { + apiUrl: 'https://dev94428.service-now.com/', + }, +} as ActionConnector; + +const getChoicesResponse = [ + { + dependent_value: '', + label: 'Priviledge Escalation', + value: 'Priviledge Escalation', + element: 'category', + }, + { + dependent_value: '', + label: 'Criminal activity/investigation', + value: 'Criminal activity/investigation', + element: 'category', + }, + { + dependent_value: '', + label: 'Denial of Service', + value: 'Denial of Service', + element: 'category', + }, +]; + +describe('useGetChoices', () => { + const { services } = useKibanaMock(); + getChoicesMock.mockResolvedValue({ + data: getChoicesResponse, + }); + + beforeEach(() => { + jest.clearAllMocks(); + }); + + const fields = ['priority']; + + it('init', async () => { + const { result, waitForNextUpdate } = renderHook(() => + useGetChoices({ + http: services.http, + actionConnector, + toastNotifications: services.notifications.toasts, + fields, + onSuccess, + }) + ); + + await waitForNextUpdate(); + + expect(result.current).toEqual({ + isLoading: false, + choices: getChoicesResponse, + }); + }); + + it('returns an empty array when connector is not presented', async () => { + const { result } = renderHook(() => + useGetChoices({ + http: services.http, + actionConnector: undefined, + toastNotifications: services.notifications.toasts, + fields, + onSuccess, + }) + ); + + expect(result.current).toEqual({ + isLoading: false, + choices: [], + }); + }); + + it('it calls onSuccess', async () => { + const { waitForNextUpdate } = renderHook(() => + useGetChoices({ + http: services.http, + actionConnector, + toastNotifications: services.notifications.toasts, + fields, + onSuccess, + }) + ); + + await waitForNextUpdate(); + + expect(onSuccess).toHaveBeenCalledWith(getChoicesResponse); + }); + + it('it displays an error when service fails', async () => { + getChoicesMock.mockResolvedValue({ + status: 'error', + serviceMessage: 'An error occurred', + }); + + const { waitForNextUpdate } = renderHook(() => + useGetChoices({ + http: services.http, + actionConnector, + toastNotifications: services.notifications.toasts, + fields, + onSuccess, + }) + ); + + await waitForNextUpdate(); + + expect(services.notifications.toasts.addDanger).toHaveBeenCalledWith({ + text: 'An error occurred', + title: 'Unable to get choices', + }); + }); + + it('it displays an error when http throws an error', async () => { + getChoicesMock.mockImplementation(() => { + throw new Error('An error occurred'); + }); + + renderHook(() => + useGetChoices({ + http: services.http, + actionConnector, + toastNotifications: services.notifications.toasts, + fields, + onSuccess, + }) + ); + + expect(services.notifications.toasts.addDanger).toHaveBeenCalledWith({ + text: 'An error occurred', + title: 'Unable to get choices', + }); + }); +}); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/use_get_choices.tsx b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/use_get_choices.tsx new file mode 100644 index 00000000000000..0e4338cec0e186 --- /dev/null +++ b/x-pack/plugins/triggers_actions_ui/public/application/components/builtin_action_types/servicenow/use_get_choices.tsx @@ -0,0 +1,99 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { useState, useEffect, useRef, useCallback } from 'react'; +import { HttpSetup, ToastsApi } from 'kibana/public'; +import { ActionConnector } from '../../../../types'; +import { getChoices } from './api'; +import { Choice } from './types'; +import * as i18n from './translations'; + +export interface UseGetChoicesProps { + http: HttpSetup; + toastNotifications: Pick< + ToastsApi, + 'get$' | 'add' | 'remove' | 'addSuccess' | 'addWarning' | 'addDanger' | 'addError' + >; + actionConnector?: ActionConnector; + fields: string[]; + onSuccess?: (choices: Choice[]) => void; +} + +export interface UseGetChoices { + choices: Choice[]; + isLoading: boolean; +} + +export const useGetChoices = ({ + http, + actionConnector, + toastNotifications, + fields, + onSuccess, +}: UseGetChoicesProps): UseGetChoices => { + const [isLoading, setIsLoading] = useState(false); + const [choices, setChoices] = useState([]); + const didCancel = useRef(false); + const abortCtrl = useRef(new AbortController()); + + const fetchData = useCallback(async () => { + if (!actionConnector) { + setIsLoading(false); + return; + } + + try { + didCancel.current = false; + abortCtrl.current.abort(); + abortCtrl.current = new AbortController(); + setIsLoading(true); + + const res = await getChoices({ + http, + signal: abortCtrl.current.signal, + connectorId: actionConnector.id, + fields, + }); + + if (!didCancel.current) { + setIsLoading(false); + setChoices(res.data ?? []); + if (res.status && res.status === 'error') { + toastNotifications.addDanger({ + title: i18n.CHOICES_API_ERROR, + text: `${res.serviceMessage ?? res.message}`, + }); + } else if (onSuccess) { + onSuccess(res.data ?? []); + } + } + } catch (error) { + if (!didCancel.current) { + setIsLoading(false); + toastNotifications.addDanger({ + title: i18n.CHOICES_API_ERROR, + text: error.message, + }); + } + } + }, [actionConnector, http, fields, onSuccess, toastNotifications]); + + useEffect(() => { + fetchData(); + + return () => { + didCancel.current = true; + abortCtrl.current.abort(); + setIsLoading(false); + }; + // eslint-disable-next-line react-hooks/exhaustive-deps + }, []); + + return { + choices, + isLoading, + }; +}; diff --git a/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/action_form.test.tsx b/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/action_form.test.tsx index 13443e0b682458..9cde2802e0c404 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/action_form.test.tsx +++ b/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/action_form.test.tsx @@ -550,7 +550,9 @@ describe('action_form', () => { ]); expect(setHasActionsWithBrokenConnector).toHaveBeenLastCalledWith(true); expect(wrapper.find(EuiAccordion)).toHaveLength(3); - expect(wrapper.find(`div[data-test-subj="alertActionAccordionCallout"]`)).toHaveLength(2); + expect( + wrapper.find(`EuiIconTip[data-test-subj="alertActionAccordionErrorTooltip"]`) + ).toHaveLength(2); }); }); }); diff --git a/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/action_form.tsx b/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/action_form.tsx index 2145443ba044ce..ed6ea6a73f2424 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/action_form.tsx +++ b/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/action_form.tsx @@ -34,7 +34,11 @@ import { ActionTypeForm, ActionTypeFormProps } from './action_type_form'; import { AddConnectorInline } from './connector_add_inline'; import { actionTypeCompare } from '../../lib/action_type_compare'; import { checkActionFormActionTypeEnabled } from '../../lib/check_action_type_enabled'; -import { VIEW_LICENSE_OPTIONS_LINK, DEFAULT_HIDDEN_ACTION_TYPES } from '../../../common/constants'; +import { + VIEW_LICENSE_OPTIONS_LINK, + DEFAULT_HIDDEN_ACTION_TYPES, + DEFAULT_HIDDEN_ONLY_ON_ALERTS_ACTION_TYPES, +} from '../../../common/constants'; import { ActionGroup, AlertActionParam } from '../../../../../alerts/common'; import { useKibana } from '../../../common/lib/kibana'; import { DefaultActionParamsGetter } from '../../lib/get_defaults_for_action_params'; @@ -230,9 +234,15 @@ export const ActionForm = ({ .list() /** * TODO: Remove when cases connector is available across Kibana. Issue: https://github.com/elastic/kibana/issues/82502. + * TODO: Need to decide about ServiceNow SIR connector. * If actionTypes are set, hidden connectors are filtered out. Otherwise, they are not. */ - .filter(({ id }) => actionTypes ?? !DEFAULT_HIDDEN_ACTION_TYPES.includes(id)) + .filter( + ({ id }) => + actionTypes ?? + (!DEFAULT_HIDDEN_ACTION_TYPES.includes(id) && + !DEFAULT_HIDDEN_ONLY_ON_ALERTS_ACTION_TYPES.includes(id)) + ) .filter((item) => actionTypesIndex[item.id]) .filter((item) => !!item.actionParamsFields) .sort((a, b) => @@ -308,6 +318,7 @@ export const ActionForm = ({ key={`action-form-action-at-${index}`} actionTypeRegistry={actionTypeRegistry} emptyActionsIds={emptyActionsIds} + connectors={connectors} onDeleteConnector={() => { const updatedActions = actions.filter( (_item: AlertAction, i: number) => i !== index @@ -330,6 +341,9 @@ export const ActionForm = ({ }); setAddModalVisibility(true); }} + onSelectConnector={(connectorId: string) => { + setActionIdByIndex(connectorId, index); + }} /> ); } diff --git a/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/connector_add_inline.tsx b/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/connector_add_inline.tsx index 6ffe730658d3d7..c2e96e6f3c0efd 100644 --- a/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/connector_add_inline.tsx +++ b/x-pack/plugins/triggers_actions_ui/public/application/sections/action_connector_form/connector_add_inline.tsx @@ -4,7 +4,7 @@ * you may not use this file except in compliance with the Elastic License. */ -import React, { Fragment } from 'react'; +import React, { Fragment, useEffect, useState } from 'react'; import { i18n } from '@kbn/i18n'; import { FormattedMessage } from '@kbn/i18n/react'; import { @@ -18,8 +18,13 @@ import { EuiEmptyPrompt, EuiCallOut, EuiText, + EuiFormRow, + EuiButtonEmpty, + EuiComboBox, + EuiComboBoxOptionOption, + EuiIconTip, } from '@elastic/eui'; -import { AlertAction, ActionTypeIndex } from '../../../types'; +import { AlertAction, ActionTypeIndex, ActionConnector } from '../../../types'; import { hasSaveActionsCapability } from '../../lib/capabilities'; import { ActionAccordionFormProps } from './action_form'; import { useKibana } from '../../../common/lib/kibana'; @@ -27,9 +32,11 @@ import { useKibana } from '../../../common/lib/kibana'; type AddConnectorInFormProps = { actionTypesIndex: ActionTypeIndex; actionItem: AlertAction; + connectors: ActionConnector[]; index: number; onAddConnector: () => void; onDeleteConnector: () => void; + onSelectConnector: (connectorId: string) => void; emptyActionsIds: string[]; } & Pick; @@ -37,8 +44,10 @@ export const AddConnectorInline = ({ actionTypesIndex, actionItem, index, + connectors, onAddConnector, onDeleteConnector, + onSelectConnector, actionTypeRegistry, emptyActionsIds, }: AddConnectorInFormProps) => { @@ -46,10 +55,14 @@ export const AddConnectorInline = ({ application: { capabilities }, } = useKibana().services; const canSave = hasSaveActionsCapability(capabilities); + const [connectorOptionsList, setConnectorOptionsList] = useState([]); + const [isEmptyActionId, setIsEmptyActionId] = useState(false); + const [errors, setErrors] = useState([]); const actionTypeName = actionTypesIndex ? actionTypesIndex[actionItem.actionTypeId].name : actionItem.actionTypeId; + const actionType = actionTypesIndex[actionItem.actionTypeId]; const actionTypeRegistered = actionTypeRegistry.get(actionItem.actionTypeId); const noConnectorsLabel = ( @@ -61,6 +74,92 @@ export const AddConnectorInline = ({ }} /> ); + + const unableToLoadConnectorLabel = ( + + + + ); + + useEffect(() => { + if (connectors) { + const altConnectorOptions = connectors + .filter( + (connector) => + connector.actionTypeId === actionItem.actionTypeId && + // include only enabled by config connectors or preconfigured + (actionType?.enabledInConfig || connector.isPreconfigured) + ) + .map(({ name, id, isPreconfigured }) => ({ + label: `${name} ${isPreconfigured ? '(preconfigured)' : ''}`, + key: id, + id, + })); + setConnectorOptionsList(altConnectorOptions); + + if (altConnectorOptions.length > 0) { + setErrors([`Unable to load ${actionTypeRegistered.actionTypeTitle} connector`]); + } + } + + setIsEmptyActionId(!!emptyActionsIds.find((emptyId: string) => actionItem.id === emptyId)); + // eslint-disable-next-line react-hooks/exhaustive-deps + }, []); + + const connectorsDropdown = ( + + + + } + labelAppend={ + + + + } + error={errors} + isInvalid={errors.length > 0} + > + { + // On selecting a option from this combo box, this component will + // be removed but the EuiComboBox performs some additional updates on + // closing the dropdown. Wrapping in a `setTimeout` to avoid `React state + // update on an unmounted component` warnings. + setTimeout(() => { + onSelectConnector(selectedOptions[0].id ?? ''); + }); + }} + isClearable={false} + /> + + + + ); + return ( + {!isEmptyActionId && ( + + + } + /> + + )} } extraAction={ @@ -106,38 +221,27 @@ export const AddConnectorInline = ({ paddingSize="l" > {canSave ? ( - actionItem.id === emptyId) ? ( - noConnectorsLabel - ) : ( - - ) - } - actions={[ - - - , - ]} - /> + connectorOptionsList.length > 0 ? ( + connectorsDropdown + ) : ( + + + + } + /> + ) ) : (

diff --git a/x-pack/plugins/triggers_actions_ui/public/common/constants/index.ts b/x-pack/plugins/triggers_actions_ui/public/common/constants/index.ts index 833ed915fad597..8832f8b826eabd 100644 --- a/x-pack/plugins/triggers_actions_ui/public/common/constants/index.ts +++ b/x-pack/plugins/triggers_actions_ui/public/common/constants/index.ts @@ -11,3 +11,5 @@ export { builtInGroupByTypes } from './group_by_types'; export const VIEW_LICENSE_OPTIONS_LINK = 'https://www.elastic.co/subscriptions'; // TODO: Remove when cases connector is available across Kibana. Issue: https://github.com/elastic/kibana/issues/82502. export const DEFAULT_HIDDEN_ACTION_TYPES = ['.case']; +// Action types included in this array will be hidden only from the alert's action type node list +export const DEFAULT_HIDDEN_ONLY_ON_ALERTS_ACTION_TYPES = ['.servicenow-sir']; diff --git a/x-pack/plugins/triggers_actions_ui/public/common/index.ts b/x-pack/plugins/triggers_actions_ui/public/common/index.ts index 86c33a373753fc..c7f43fedb6f034 100644 --- a/x-pack/plugins/triggers_actions_ui/public/common/index.ts +++ b/x-pack/plugins/triggers_actions_ui/public/common/index.ts @@ -10,6 +10,6 @@ export * from './index_controls'; export * from './lib'; export * from './types'; -export { connectorConfiguration as ServiceNowConnectorConfiguration } from '../application/components/builtin_action_types/servicenow/config'; +export { serviceNowITSMConfiguration as ServiceNowITSMConnectorConfiguration } from '../application/components/builtin_action_types/servicenow/config'; export { connectorConfiguration as JiraConnectorConfiguration } from '../application/components/builtin_action_types/jira/config'; export { connectorConfiguration as ResilientConnectorConfiguration } from '../application/components/builtin_action_types/resilient/config'; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.tsx deleted file mode 100644 index be4f0fc62271d2..00000000000000 --- a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.tsx +++ /dev/null @@ -1,219 +0,0 @@ -/* - * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one - * or more contributor license agreements. Licensed under the Elastic License; - * you may not use this file except in compliance with the Elastic License. - */ - -import React, { useContext, useEffect, useState } from 'react'; -import { - EuiButtonIcon, - EuiFlexGroup, - EuiFlexItem, - EuiImage, - EuiSpacer, - EuiText, - EuiLoadingSpinner, -} from '@elastic/eui'; -import useIntersection from 'react-use/lib/useIntersection'; -import moment from 'moment'; -import styled from 'styled-components'; -import { FormattedMessage } from '@kbn/i18n/react'; -import { Ping } from '../../../../../common/runtime_types/ping'; -import { getShortTimeStamp } from '../../../overview/monitor_list/columns/monitor_status_column'; -import { euiStyled } from '../../../../../../../../src/plugins/kibana_react/common'; -import { useFetcher, FETCH_STATUS } from '../../../../../../observability/public'; -import { getJourneyScreenshot } from '../../../../state/api/journey'; -import { UptimeSettingsContext } from '../../../../contexts'; - -const StepImage = styled(EuiImage)` - &&& { - display: flex; - figcaption { - white-space: nowrap; - align-self: center; - margin-left: 8px; - margin-top: 8px; - text-decoration: none !important; - } - } -`; - -const StepDiv = styled.div` - figure.euiImage { - div.stepArrowsFullScreen { - display: none; - } - } - - figure.euiImage-isFullScreen { - div.stepArrowsFullScreen { - display: flex; - } - } - position: relative; - div.stepArrows { - display: none; - } - :hover { - div.stepArrows { - display: flex; - } - } -`; - -interface Props { - timestamp: string; - ping: Ping; -} - -export const PingTimestamp = ({ timestamp, ping }: Props) => { - const [stepNo, setStepNo] = useState(1); - - const [stepImages, setStepImages] = useState([]); - - const intersectionRef = React.useRef(null); - - const { basePath } = useContext(UptimeSettingsContext); - - const imgPath = basePath + `/api/uptime/journey/screenshot/${ping.monitor.check_group}/${stepNo}`; - - const intersection = useIntersection(intersectionRef, { - root: null, - rootMargin: '0px', - threshold: 1, - }); - - const { data, status } = useFetcher(() => { - if (intersection && intersection.intersectionRatio === 1 && !stepImages[stepNo - 1]) - return getJourneyScreenshot(imgPath); - }, [intersection?.intersectionRatio, stepNo]); - - useEffect(() => { - if (data) { - setStepImages((prevState) => [...prevState, data?.src]); - } - }, [data]); - - const imgSrc = stepImages[stepNo] || data?.src; - - const isLoading = status === FETCH_STATUS.LOADING; - const isPending = status === FETCH_STATUS.PENDING; - - const captionContent = `Step:${stepNo} ${data?.stepName}`; - - const ImageCaption = ( - <> -

- {imgSrc && ( - - - { - setStepNo(stepNo - 1); - }} - iconType="arrowLeft" - aria-label="Next" - /> - - - {captionContent} - - - { - setStepNo(stepNo + 1); - }} - iconType="arrowRight" - aria-label="Next" - /> - - - )} -
- {/* TODO: Add link to details page once it's available */} - {getShortTimeStamp(moment(timestamp))} - - - ); - - return ( - - {imgSrc ? ( - - ) : ( - - - {isLoading || isPending ? ( - - ) : ( - - )} - - {ImageCaption} - - )} - - - { - setStepNo(stepNo - 1); - }} - iconType="arrowLeft" - aria-label="Next" - /> - - - { - setStepNo(stepNo + 1); - }} - iconType="arrowRight" - aria-label="Next" - /> - - - - ); -}; - -const BorderedText = euiStyled(EuiText)` - width: 120px; - text-align: center; - border: 1px solid ${(props) => props.theme.eui.euiColorLightShade}; -`; - -export const NoImageAvailable = () => { - return ( - - - - - - ); -}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/index.ts b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/index.ts new file mode 100644 index 00000000000000..db9c18e30cfc15 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/index.ts @@ -0,0 +1,7 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +export { PingTimestamp } from './ping_timestamp'; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.test.tsx new file mode 100644 index 00000000000000..c8acfd48a9913d --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.test.tsx @@ -0,0 +1,88 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { fireEvent, waitFor } from '@testing-library/react'; +import React from 'react'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { NavButtons, NavButtonsProps } from './nav_buttons'; + +describe('NavButtons', () => { + let defaultProps: NavButtonsProps; + + beforeEach(() => { + defaultProps = { + maxSteps: 3, + stepNumber: 2, + setStepNumber: jest.fn(), + setIsImagePopoverOpen: jest.fn(), + }; + }); + + it('labels prev and next buttons', () => { + const { getByLabelText } = render(); + + expect(getByLabelText('Previous step')); + expect(getByLabelText('Next step')); + }); + + it('increments step number on next click', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Next step'); + + fireEvent.click(nextButton); + + await waitFor(() => { + expect(defaultProps.setStepNumber).toHaveBeenCalledTimes(1); + expect(defaultProps.setStepNumber).toHaveBeenCalledWith(3); + }); + }); + + it('decrements step number on prev click', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Previous step'); + + fireEvent.click(nextButton); + + await waitFor(() => { + expect(defaultProps.setStepNumber).toHaveBeenCalledTimes(1); + expect(defaultProps.setStepNumber).toHaveBeenCalledWith(1); + }); + }); + + it('disables `next` button on final step', () => { + defaultProps.stepNumber = 3; + + const { getByLabelText } = render(); + + // getByLabelText('Next step'); + expect(getByLabelText('Next step')).toHaveAttribute('disabled'); + expect(getByLabelText('Previous step')).not.toHaveAttribute('disabled'); + }); + + it('disables `prev` button on final step', () => { + defaultProps.stepNumber = 1; + + const { getByLabelText } = render(); + + expect(getByLabelText('Next step')).not.toHaveAttribute('disabled'); + expect(getByLabelText('Previous step')).toHaveAttribute('disabled'); + }); + + it('opens popover when mouse enters', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Next step'); + + fireEvent.mouseEnter(nextButton); + + await waitFor(() => { + expect(defaultProps.setIsImagePopoverOpen).toHaveBeenCalledTimes(1); + expect(defaultProps.setIsImagePopoverOpen).toHaveBeenCalledWith(true); + }); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.tsx new file mode 100644 index 00000000000000..1c24caba6a9171 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/nav_buttons.tsx @@ -0,0 +1,56 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiButtonIcon, EuiFlexItem, EuiFlexGroup } from '@elastic/eui'; +import React from 'react'; +import { nextAriaLabel, prevAriaLabel } from './translations'; + +export interface NavButtonsProps { + maxSteps?: number; + setIsImagePopoverOpen: React.Dispatch>; + setStepNumber: React.Dispatch>; + stepNumber: number; +} + +export const NavButtons: React.FC = ({ + maxSteps, + setIsImagePopoverOpen, + setStepNumber, + stepNumber, +}) => ( + setIsImagePopoverOpen(true)} + style={{ position: 'absolute', bottom: 0, left: 30 }} + > + + { + setStepNumber(stepNumber - 1); + }} + iconType="arrowLeft" + aria-label={prevAriaLabel} + /> + + + { + setStepNumber(stepNumber + 1); + }} + iconType="arrowRight" + aria-label={nextAriaLabel} + /> + + +); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.test.tsx new file mode 100644 index 00000000000000..17e679846a66df --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.test.tsx @@ -0,0 +1,17 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React from 'react'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { NoImageAvailable } from './no_image_available'; + +describe('NoImageAvailable', () => { + it('renders expected text', () => { + const { getByText } = render(); + + expect(getByText('No image available')); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.tsx new file mode 100644 index 00000000000000..2498e07969f111 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_available.tsx @@ -0,0 +1,29 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiText } from '@elastic/eui'; +import { FormattedMessage } from '@kbn/i18n/react'; +import React from 'react'; +import { euiStyled } from '../../../../../../../../../src/plugins/kibana_react/common'; + +const BorderedText = euiStyled(EuiText)` + width: 120px; + text-align: center; + border: 1px solid ${(props) => props.theme.eui.euiColorLightShade}; +`; + +export const NoImageAvailable = () => { + return ( + + + + + + ); +}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.test.tsx new file mode 100644 index 00000000000000..24080e2f4061d0 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.test.tsx @@ -0,0 +1,44 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React from 'react'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { NoImageDisplay, NoImageDisplayProps } from './no_image_display'; +import { imageLoadingSpinnerAriaLabel } from './translations'; + +describe('NoImageDisplay', () => { + let defaultProps: NoImageDisplayProps; + beforeEach(() => { + defaultProps = { + imageCaption:
test caption
, + isLoading: false, + isPending: false, + }; + }); + + it('renders a loading spinner for loading state', () => { + defaultProps.isLoading = true; + const { getByText, getByLabelText } = render(); + + expect(getByLabelText(imageLoadingSpinnerAriaLabel)); + expect(getByText('test caption')); + }); + + it('renders a loading spinner for pending state', () => { + defaultProps.isPending = true; + const { getByText, getByLabelText } = render(); + + expect(getByLabelText(imageLoadingSpinnerAriaLabel)); + expect(getByText('test caption')); + }); + + it('renders no image available when not loading or pending', () => { + const { getByText } = render(); + + expect(getByText('No image available')); + expect(getByText('test caption')); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.tsx new file mode 100644 index 00000000000000..185f488d5acd29 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/no_image_display.tsx @@ -0,0 +1,39 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiFlexItem, EuiFlexGroup, EuiLoadingSpinner } from '@elastic/eui'; +import React from 'react'; +import { NoImageAvailable } from './no_image_available'; +import { imageLoadingSpinnerAriaLabel } from './translations'; + +export interface NoImageDisplayProps { + imageCaption: JSX.Element; + isLoading: boolean; + isPending: boolean; +} + +export const NoImageDisplay: React.FC = ({ + imageCaption, + isLoading, + isPending, +}) => { + return ( + + + {isLoading || isPending ? ( + + ) : ( + + )} + + {imageCaption} + + ); +}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.test.tsx similarity index 70% rename from x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.test.tsx rename to x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.test.tsx index 1baeb8a69d34cc..a934f6fa39b22a 100644 --- a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp.test.tsx +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.test.tsx @@ -5,16 +5,17 @@ */ import React from 'react'; +import { fireEvent, waitFor } from '@testing-library/react'; import { PingTimestamp } from './ping_timestamp'; -import { mockReduxHooks } from '../../../../lib/helper/test_helpers'; -import { render } from '../../../../lib/helper/rtl_helpers'; -import { Ping } from '../../../../../common/runtime_types/ping'; -import * as observabilityPublic from '../../../../../../observability/public'; +import { mockReduxHooks } from '../../../../../lib/helper/test_helpers'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { Ping } from '../../../../../../common/runtime_types/ping'; +import * as observabilityPublic from '../../../../../../../observability/public'; mockReduxHooks(); -jest.mock('../../../../../../observability/public', () => { - const originalModule = jest.requireActual('../../../../../../observability/public'); +jest.mock('../../../../../../../observability/public', () => { + const originalModule = jest.requireActual('../../../../../../../observability/public'); return { ...originalModule, @@ -92,4 +93,26 @@ describe('Ping Timestamp component', () => { const { container } = render(); expect(container.querySelector('img')?.src).toBe(src); }); + + it('displays popover image when mouse enters img caption, and hides onLeave', async () => { + const src = 'http://sample.com/sampleImageSrc.png'; + jest.spyOn(observabilityPublic, 'useFetcher').mockReturnValue({ + status: FETCH_STATUS.SUCCESS, + data: { src }, + refetch: () => null, + }); + const { getByAltText, getByText, queryByAltText } = render( + + ); + const caption = getByText('Nov 26, 2020 10:28:56 AM'); + fireEvent.mouseEnter(caption); + + const altText = `A larger version of the screenshot for this journey step's thumbnail.`; + + await waitFor(() => getByAltText(altText)); + + fireEvent.mouseLeave(caption); + + await waitFor(() => expect(queryByAltText(altText)).toBeNull()); + }); }); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.tsx new file mode 100644 index 00000000000000..6d605f25f6f683 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/ping_timestamp.tsx @@ -0,0 +1,120 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import React, { useContext, useEffect, useState } from 'react'; +import useIntersection from 'react-use/lib/useIntersection'; +import styled from 'styled-components'; +import { Ping } from '../../../../../../common/runtime_types/ping'; +import { useFetcher, FETCH_STATUS } from '../../../../../../../observability/public'; +import { getJourneyScreenshot } from '../../../../../state/api/journey'; +import { UptimeSettingsContext } from '../../../../../contexts'; +import { NavButtons } from './nav_buttons'; +import { NoImageDisplay } from './no_image_display'; +import { StepImageCaption } from './step_image_caption'; +import { StepImagePopover } from './step_image_popover'; +import { formatCaptionContent } from './translations'; + +const StepDiv = styled.div` + figure.euiImage { + div.stepArrowsFullScreen { + display: none; + } + } + + figure.euiImage-isFullScreen { + div.stepArrowsFullScreen { + display: flex; + } + } + position: relative; + div.stepArrows { + display: none; + } + :hover { + div.stepArrows { + display: flex; + } + } +`; + +interface Props { + timestamp: string; + ping: Ping; +} + +export const PingTimestamp = ({ timestamp, ping }: Props) => { + const [stepNumber, setStepNumber] = useState(1); + const [isImagePopoverOpen, setIsImagePopoverOpen] = useState(false); + + const [stepImages, setStepImages] = useState([]); + + const intersectionRef = React.useRef(null); + + const { basePath } = useContext(UptimeSettingsContext); + + const imgPath = `${basePath}/api/uptime/journey/screenshot/${ping.monitor.check_group}/${stepNumber}`; + + const intersection = useIntersection(intersectionRef, { + root: null, + rootMargin: '0px', + threshold: 1, + }); + + const { data, status } = useFetcher(() => { + if (intersection && intersection.intersectionRatio === 1 && !stepImages[stepNumber - 1]) + return getJourneyScreenshot(imgPath); + }, [intersection?.intersectionRatio, stepNumber]); + + useEffect(() => { + if (data) { + setStepImages((prevState) => [...prevState, data?.src]); + } + }, [data]); + + const imgSrc = stepImages[stepNumber] || data?.src; + + const captionContent = formatCaptionContent(stepNumber, data?.maxSteps); + + const ImageCaption = ( + + ); + + return ( + setIsImagePopoverOpen(true)} + onMouseLeave={() => setIsImagePopoverOpen(false)} + ref={intersectionRef} + > + {imgSrc ? ( + + ) : ( + + )} + + + ); +}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.test.tsx new file mode 100644 index 00000000000000..ef1d0cb388a189 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.test.tsx @@ -0,0 +1,89 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { fireEvent, waitFor } from '@testing-library/react'; +import React from 'react'; +import { render } from '../../../../../lib/helper/rtl_helpers'; +import { StepImageCaption, StepImageCaptionProps } from './step_image_caption'; + +describe('StepImageCaption', () => { + let defaultProps: StepImageCaptionProps; + + beforeEach(() => { + defaultProps = { + captionContent: 'test caption content', + imgSrc: 'http://sample.com/sampleImageSrc.png', + maxSteps: 3, + setStepNumber: jest.fn(), + stepNumber: 2, + timestamp: '2020-11-26T15:28:56.896Z', + }; + }); + + it('labels prev and next buttons', () => { + const { getByLabelText } = render(); + + expect(getByLabelText('Previous step')); + expect(getByLabelText('Next step')); + }); + + it('increments step number on next click', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Next step'); + + fireEvent.click(nextButton); + + await waitFor(() => { + expect(defaultProps.setStepNumber).toHaveBeenCalledTimes(1); + expect(defaultProps.setStepNumber).toHaveBeenCalledWith(3); + }); + }); + + it('decrements step number on prev click', async () => { + const { getByLabelText } = render(); + + const nextButton = getByLabelText('Previous step'); + + fireEvent.click(nextButton); + + await waitFor(() => { + expect(defaultProps.setStepNumber).toHaveBeenCalledTimes(1); + expect(defaultProps.setStepNumber).toHaveBeenCalledWith(1); + }); + }); + + it('disables `next` button on final step', () => { + defaultProps.stepNumber = 3; + + const { getByLabelText } = render(); + + // getByLabelText('Next step'); + expect(getByLabelText('Next step')).toHaveAttribute('disabled'); + expect(getByLabelText('Previous step')).not.toHaveAttribute('disabled'); + }); + + it('disables `prev` button on final step', () => { + defaultProps.stepNumber = 1; + + const { getByLabelText } = render(); + + expect(getByLabelText('Next step')).not.toHaveAttribute('disabled'); + expect(getByLabelText('Previous step')).toHaveAttribute('disabled'); + }); + + it('renders a timestamp', () => { + const { getByText } = render(); + + getByText('Nov 26, 2020 10:28:56 AM'); + }); + + it('renders caption content', () => { + const { getByText } = render(); + + getByText('test caption content'); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.tsx new file mode 100644 index 00000000000000..c5da98bacc431c --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_caption.tsx @@ -0,0 +1,68 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiButtonIcon, EuiFlexGroup, EuiFlexItem, EuiSpacer, EuiText } from '@elastic/eui'; +import React from 'react'; +import moment from 'moment'; +import { nextAriaLabel, prevAriaLabel } from './translations'; +import { getShortTimeStamp } from '../../../../overview/monitor_list/columns/monitor_status_column'; + +export interface StepImageCaptionProps { + captionContent: string; + imgSrc?: string; + maxSteps?: number; + setStepNumber: React.Dispatch>; + stepNumber: number; + timestamp: string; +} + +export const StepImageCaption: React.FC = ({ + captionContent, + imgSrc, + maxSteps, + setStepNumber, + stepNumber, + timestamp, +}) => { + return ( + <> +
+ {imgSrc && ( + + + { + setStepNumber(stepNumber - 1); + }} + iconType="arrowLeft" + aria-label={prevAriaLabel} + /> + + + {captionContent} + + + { + setStepNumber(stepNumber + 1); + }} + iconType="arrowRight" + aria-label={nextAriaLabel} + /> + + + )} +
+ {/* TODO: Add link to details page once it's available */} + {getShortTimeStamp(moment(timestamp))} + + + ); +}; diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.test.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.test.tsx new file mode 100644 index 00000000000000..184794c1465aa4 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.test.tsx @@ -0,0 +1,60 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { fireEvent, waitFor } from '@testing-library/react'; +import React from 'react'; +import { StepImagePopover, StepImagePopoverProps } from './step_image_popover'; +import { render } from '../../../../../lib/helper/rtl_helpers'; + +describe('StepImagePopover', () => { + let defaultProps: StepImagePopoverProps; + + beforeEach(() => { + defaultProps = { + captionContent: 'test caption', + imageCaption:
test caption element
, + imgSrc: 'http://sample.com/sampleImageSrc.png', + isImagePopoverOpen: false, + }; + }); + + it('opens displays full-size image on click, hides after close is closed', async () => { + const { getByAltText, getByLabelText, queryByLabelText } = render( + + ); + + const closeFullScreenButton = 'Close full screen test caption image'; + + expect(queryByLabelText(closeFullScreenButton)).toBeNull(); + + const caption = getByAltText('test caption'); + fireEvent.click(caption); + + await waitFor(() => { + const closeButton = getByLabelText(closeFullScreenButton); + fireEvent.click(closeButton); + }); + + await waitFor(() => { + expect(queryByLabelText(closeFullScreenButton)).toBeNull(); + }); + }); + + it('shows the popover when `isOpen` is true', () => { + defaultProps.isImagePopoverOpen = true; + + const { getByAltText } = render(); + + expect(getByAltText(`A larger version of the screenshot for this journey step's thumbnail.`)); + }); + + it('renders caption content', () => { + const { getByRole } = render(); + const image = getByRole('img'); + expect(image).toHaveAttribute('alt', 'test caption'); + expect(image).toHaveAttribute('src', 'http://sample.com/sampleImageSrc.png'); + }); +}); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.tsx b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.tsx new file mode 100644 index 00000000000000..fd7b7e6a886bb3 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/step_image_popover.tsx @@ -0,0 +1,61 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { EuiImage, EuiPopover } from '@elastic/eui'; +import React from 'react'; +import styled from 'styled-components'; +import { fullSizeImageAlt } from './translations'; + +const POPOVER_IMG_HEIGHT = 360; +const POPOVER_IMG_WIDTH = 640; + +const StepImage = styled(EuiImage)` + &&& { + display: flex; + figcaption { + white-space: nowrap; + align-self: center; + margin-left: 8px; + margin-top: 8px; + text-decoration: none !important; + } + } +`; +export interface StepImagePopoverProps { + captionContent: string; + imageCaption: JSX.Element; + imgSrc: string; + isImagePopoverOpen: boolean; +} + +export const StepImagePopover: React.FC = ({ + captionContent, + imageCaption, + imgSrc, + isImagePopoverOpen, +}) => ( + + } + isOpen={isImagePopoverOpen} + > + + +); diff --git a/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/translations.ts b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/translations.ts new file mode 100644 index 00000000000000..ad49143a680572 --- /dev/null +++ b/x-pack/plugins/uptime/public/components/monitor/ping_list/columns/ping_timestamp/translations.ts @@ -0,0 +1,35 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { i18n } from '@kbn/i18n'; + +export const prevAriaLabel = i18n.translate('xpack.uptime.synthetics.prevStepButton.airaLabel', { + defaultMessage: 'Previous step', +}); + +export const nextAriaLabel = i18n.translate('xpack.uptime.synthetics.nextStepButton.ariaLabel', { + defaultMessage: 'Next step', +}); + +export const imageLoadingSpinnerAriaLabel = i18n.translate( + 'xpack.uptime.synthetics.imageLoadingSpinner.ariaLabel', + { + defaultMessage: 'An animated spinner indicating the image is loading', + } +); + +export const fullSizeImageAlt = i18n.translate('xpack.uptime.synthetics.thumbnail.fullSize.alt', { + defaultMessage: `A larger version of the screenshot for this journey step's thumbnail.`, +}); + +export const formatCaptionContent = (stepNumber: number, stepName?: number) => + i18n.translate('xpack.uptime.synthetics.pingTimestamp.captionContent', { + defaultMessage: 'Step: {stepNumber} {stepName}', + values: { + stepNumber, + stepName, + }, + }); diff --git a/x-pack/plugins/uptime/public/components/monitor/synthetics/step_detail/step_detail_container.tsx b/x-pack/plugins/uptime/public/components/monitor/synthetics/step_detail/step_detail_container.tsx index bdc6dbf3f6de22..3d9b646931e7d3 100644 --- a/x-pack/plugins/uptime/public/components/monitor/synthetics/step_detail/step_detail_container.tsx +++ b/x-pack/plugins/uptime/public/components/monitor/synthetics/step_detail/step_detail_container.tsx @@ -32,7 +32,7 @@ export const StepDetailContainer: React.FC = ({ checkGroup, stepIndex }) useEffect(() => { if (checkGroup) { - dispatch(getJourneySteps({ checkGroup })); + dispatch(getJourneySteps({ checkGroup, syntheticEventTypes: ['step/end'] })); } }, [dispatch, checkGroup]); diff --git a/x-pack/plugins/uptime/public/components/monitor/synthetics/step_screenshot_display.tsx b/x-pack/plugins/uptime/public/components/monitor/synthetics/step_screenshot_display.tsx index 3efcff196b55fd..716e877c50943c 100644 --- a/x-pack/plugins/uptime/public/components/monitor/synthetics/step_screenshot_display.tsx +++ b/x-pack/plugins/uptime/public/components/monitor/synthetics/step_screenshot_display.tsx @@ -98,7 +98,7 @@ export const StepScreenshotDisplay: FC = ({ closePopover={() => setIsImagePopoverOpen(false)} isOpen={isImagePopoverOpen} > - { = ({ } ) } - src={imgSrc} + url={imgSrc} style={{ width: POPOVER_IMG_WIDTH, height: POPOVER_IMG_HEIGHT, objectFit: 'contain' }} /> diff --git a/x-pack/plugins/uptime/public/components/settings/types.ts b/x-pack/plugins/uptime/public/components/settings/types.ts index faa1c7e72e47b9..7a3af47524b202 100644 --- a/x-pack/plugins/uptime/public/components/settings/types.ts +++ b/x-pack/plugins/uptime/public/components/settings/types.ts @@ -9,7 +9,7 @@ import { JiraActionTypeId, PagerDutyActionTypeId, ServerLogActionTypeId, - ServiceNowActionTypeId, + ServiceNowITSMActionTypeId as ServiceNowActionTypeId, SlackActionTypeId, TeamsActionTypeId, WebhookActionTypeId, diff --git a/x-pack/plugins/uptime/public/state/actions/journey.ts b/x-pack/plugins/uptime/public/state/actions/journey.ts index 0d35559d97fc38..5931980c569473 100644 --- a/x-pack/plugins/uptime/public/state/actions/journey.ts +++ b/x-pack/plugins/uptime/public/state/actions/journey.ts @@ -9,6 +9,7 @@ import { SyntheticsJourneyApiResponse } from '../../../common/runtime_types'; export interface FetchJourneyStepsParams { checkGroup: string; + syntheticEventTypes?: string[]; } export interface GetJourneyFailPayload { diff --git a/x-pack/plugins/uptime/public/state/api/journey.ts b/x-pack/plugins/uptime/public/state/api/journey.ts index 1aeeb485e481f1..684056b197f938 100644 --- a/x-pack/plugins/uptime/public/state/api/journey.ts +++ b/x-pack/plugins/uptime/public/state/api/journey.ts @@ -16,7 +16,7 @@ export async function fetchJourneySteps( ): Promise { return (await apiService.get( `/api/uptime/journey/${params.checkGroup}`, - undefined, + { syntheticEventTypes: params.syntheticEventTypes }, SyntheticsJourneyApiResponseType )) as SyntheticsJourneyApiResponse; } diff --git a/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.test.ts b/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.test.ts new file mode 100644 index 00000000000000..8c432ff6f1e0fe --- /dev/null +++ b/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.test.ts @@ -0,0 +1,196 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import { getJourneySteps, formatSyntheticEvents } from './get_journey_steps'; +import { getUptimeESMockClient } from './helper'; + +describe('getJourneySteps request module', () => { + describe('formatStepTypes', () => { + it('returns default steps if none are provided', () => { + expect(formatSyntheticEvents()).toMatchInlineSnapshot(` + Array [ + "step/end", + "stderr", + "cmd/status", + "step/screenshot", + ] + `); + }); + + it('returns provided step array if isArray', () => { + expect(formatSyntheticEvents(['step/end', 'stderr'])).toMatchInlineSnapshot(` + Array [ + "step/end", + "stderr", + ] + `); + }); + + it('returns provided step string in an array', () => { + expect(formatSyntheticEvents('step/end')).toMatchInlineSnapshot(` + Array [ + "step/end", + ] + `); + }); + }); + + describe('getJourneySteps', () => { + let data: any; + beforeEach(() => { + data = { + body: { + hits: { + hits: [ + { + _id: 'o6myXncBFt2V8m6r6z-r', + _source: { + '@timestamp': '2021-02-01T17:45:19.001Z', + synthetics: { + package_version: '0.0.1-alpha.8', + journey: { + name: 'inline', + id: 'inline', + }, + step: { + name: 'load homepage', + index: 1, + }, + type: 'step/end', + }, + monitor: { + name: 'My Monitor', + id: 'my-monitor', + check_group: '2bf952dc-64b5-11eb-8b3b-42010a84000d', + type: 'browser', + }, + }, + }, + { + _id: 'IjqzXncBn2sjqrYxYoCG', + _source: { + '@timestamp': '2021-02-01T17:45:49.944Z', + synthetics: { + package_version: '0.0.1-alpha.8', + journey: { + name: 'inline', + id: 'inline', + }, + step: { + name: 'hover over products menu', + index: 2, + }, + type: 'step/end', + }, + monitor: { + name: 'My Monitor', + timespan: { + lt: '2021-02-01T17:46:49.945Z', + gte: '2021-02-01T17:45:49.945Z', + }, + id: 'my-monitor', + check_group: '2bf952dc-64b5-11eb-8b3b-42010a84000d', + type: 'browser', + }, + }, + }, + ], + }, + }, + }; + }); + + it('formats ES result', async () => { + const { esClient: mockEsClient, uptimeEsClient } = getUptimeESMockClient(); + + mockEsClient.search.mockResolvedValueOnce(data as any); + const result: any = await getJourneySteps({ + uptimeEsClient, + checkGroup: '2bf952dc-64b5-11eb-8b3b-42010a84000d', + }); + expect(mockEsClient.search).toHaveBeenCalledTimes(1); + const call: any = mockEsClient.search.mock.calls[0][0]; + + // check that default `synthetics.type` value is supplied, + expect(call.body.query.bool.filter[0]).toMatchInlineSnapshot(` + Object { + "terms": Object { + "synthetics.type": Array [ + "step/end", + "stderr", + "cmd/status", + "step/screenshot", + ], + }, + } + `); + + // given check group is used for the terms filter + expect(call.body.query.bool.filter[1]).toMatchInlineSnapshot(` + Object { + "term": Object { + "monitor.check_group": "2bf952dc-64b5-11eb-8b3b-42010a84000d", + }, + } + `); + + // should sort by step index, then timestamp + expect(call.body.sort).toMatchInlineSnapshot(` + Array [ + Object { + "synthetics.step.index": Object { + "order": "asc", + }, + }, + Object { + "@timestamp": Object { + "order": "asc", + }, + }, + ] + `); + + expect(result).toHaveLength(2); + // `getJourneySteps` is responsible for formatting these fields, so we need to check them + result.forEach((step: any) => { + expect(['2021-02-01T17:45:19.001Z', '2021-02-01T17:45:49.944Z']).toContain(step.timestamp); + expect(['o6myXncBFt2V8m6r6z-r', 'IjqzXncBn2sjqrYxYoCG']).toContain(step.docId); + expect(step.synthetics.screenshotExists).toBeDefined(); + }); + }); + + it('notes screenshot exists when a document of type step/screenshot is included', async () => { + const { esClient: mockEsClient, uptimeEsClient } = getUptimeESMockClient(); + + data.body.hits.hits[0]._source.synthetics.type = 'step/screenshot'; + data.body.hits.hits[0]._source.synthetics.step.index = 2; + mockEsClient.search.mockResolvedValueOnce(data as any); + + const result: any = await getJourneySteps({ + uptimeEsClient, + checkGroup: '2bf952dc-64b5-11eb-8b3b-42010a84000d', + syntheticEventTypes: ['stderr', 'step/end'], + }); + + const call: any = mockEsClient.search.mock.calls[0][0]; + + // assert that filters for only the provided step types are used + expect(call.body.query.bool.filter[0]).toMatchInlineSnapshot(` + Object { + "terms": Object { + "synthetics.type": Array [ + "stderr", + "step/end", + ], + }, + } + `); + + expect(result).toHaveLength(1); + expect(result[0].synthetics.screenshotExists).toBe(true); + }); + }); +}); diff --git a/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.ts b/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.ts index c330e1b66fe93c..60d2a97c99f7d1 100644 --- a/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.ts +++ b/x-pack/plugins/uptime/server/lib/requests/get_journey_steps.ts @@ -9,11 +9,23 @@ import { Ping } from '../../../common/runtime_types'; interface GetJourneyStepsParams { checkGroup: string; + syntheticEventTypes?: string | string[]; } +const defaultEventTypes = ['step/end', 'stderr', 'cmd/status', 'step/screenshot']; + +export const formatSyntheticEvents = (eventTypes?: string | string[]) => { + if (!eventTypes) { + return defaultEventTypes; + } else { + return Array.isArray(eventTypes) ? eventTypes : [eventTypes]; + } +}; + export const getJourneySteps: UMElasticsearchQueryFn = async ({ uptimeEsClient, checkGroup, + syntheticEventTypes, }) => { const params = { query: { @@ -21,7 +33,7 @@ export const getJourneySteps: UMElasticsearchQueryFn checkGroup: schema.string(), _debug: schema.maybe(schema.boolean()), }), + query: schema.object({ + // provides a filter for the types of synthetic events to include + // when fetching a journey's data + syntheticEventTypes: schema.maybe( + schema.oneOf([schema.arrayOf(schema.string()), schema.string()]) + ), + }), }, handler: async ({ uptimeEsClient, request }): Promise => { const { checkGroup } = request.params; + const { syntheticEventTypes } = request.query; const result = await libs.requests.getJourneySteps({ uptimeEsClient, checkGroup, + syntheticEventTypes, }); const details = await libs.requests.getJourneyDetails({ diff --git a/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/plugin.ts b/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/plugin.ts index 18f3c83b001413..dfdacb230763f9 100644 --- a/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/plugin.ts +++ b/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/plugin.ts @@ -38,6 +38,7 @@ export function getAllExternalServiceSimulatorPaths(): string[] { getExternalServiceSimulatorPath(service) ); allPaths.push(`/api/_${NAME}/${ExternalServiceSimulator.SERVICENOW}/api/now/v2/table/incident`); + allPaths.push(`/api/_${NAME}/${ExternalServiceSimulator.SERVICENOW}/api/now/v2/table/sys_choice`); allPaths.push( `/api/_${NAME}/${ExternalServiceSimulator.SERVICENOW}/api/now/v2/table/sys_dictionary` ); diff --git a/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/servicenow_simulation.ts b/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/servicenow_simulation.ts index 2c3138a36f071c..b94bb89fc6f4ab 100644 --- a/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/servicenow_simulation.ts +++ b/x-pack/test/alerting_api_integration/common/fixtures/plugins/actions_simulators/server/servicenow_simulation.ts @@ -127,6 +127,51 @@ export function initPlugin(router: IRouter, path: string) { }); } ); + + router.get( + { + path: `${path}/api/now/v2/table/sys_choice`, + options: { + authRequired: false, + }, + validate: {}, + }, + async function ( + context: RequestHandlerContext, + req: KibanaRequest, + res: KibanaResponseFactory + ): Promise> { + return jsonResponse(res, 200, { + result: [ + { + dependent_value: '', + label: '1 - Critical', + value: '1', + }, + { + dependent_value: '', + label: '2 - High', + value: '2', + }, + { + dependent_value: '', + label: '3 - Moderate', + value: '3', + }, + { + dependent_value: '', + label: '4 - Low', + value: '4', + }, + { + dependent_value: '', + label: '5 - Planning', + value: '5', + }, + ], + }); + } + ); } function jsonResponse(res: KibanaResponseFactory, code: number, object?: Record) { diff --git a/x-pack/test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/servicenow.ts b/x-pack/test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/servicenow.ts index e448ad1f9c2add..5f7146b43bfdb3 100644 --- a/x-pack/test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/servicenow.ts +++ b/x-pack/test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/servicenow.ts @@ -216,7 +216,7 @@ export default function servicenowTest({ getService }: FtrProviderContext) { // Cannot destructure property 'value' of 'undefined' as it is undefined. // // The error seems to come from the exact same place in the code based on the - // exact same circomstances: + // exact same circumstances: // // https://github.com/elastic/kibana/blob/b0a223ebcbac7e404e8ae6da23b2cc6a4b509ff1/packages/kbn-config-schema/src/types/literal_type.ts#L28 // @@ -247,7 +247,7 @@ export default function servicenowTest({ getService }: FtrProviderContext) { status: 'error', retry: false, message: - 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subAction]: expected value to equal [pushToService]', + 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subAction]: expected value to equal [pushToService]\n- [4.subAction]: expected value to equal [getChoices]', }); }); }); @@ -265,7 +265,7 @@ export default function servicenowTest({ getService }: FtrProviderContext) { status: 'error', retry: false, message: - 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.incident.short_description]: expected value of type [string] but got [undefined]', + 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.incident.short_description]: expected value of type [string] but got [undefined]\n- [4.subAction]: expected value to equal [getChoices]', }); }); }); @@ -288,7 +288,7 @@ export default function servicenowTest({ getService }: FtrProviderContext) { status: 'error', retry: false, message: - 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.incident.short_description]: expected value of type [string] but got [undefined]', + 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.incident.short_description]: expected value of type [string] but got [undefined]\n- [4.subAction]: expected value to equal [getChoices]', }); }); }); @@ -315,7 +315,7 @@ export default function servicenowTest({ getService }: FtrProviderContext) { status: 'error', retry: false, message: - 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.comments]: types that failed validation:\n - [subActionParams.comments.0.0.commentId]: expected value of type [string] but got [undefined]\n - [subActionParams.comments.1]: expected value to equal [null]', + 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.comments]: types that failed validation:\n - [subActionParams.comments.0.0.commentId]: expected value of type [string] but got [undefined]\n - [subActionParams.comments.1]: expected value to equal [null]\n- [4.subAction]: expected value to equal [getChoices]', }); }); }); @@ -342,10 +342,33 @@ export default function servicenowTest({ getService }: FtrProviderContext) { status: 'error', retry: false, message: - 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.comments]: types that failed validation:\n - [subActionParams.comments.0.0.comment]: expected value of type [string] but got [undefined]\n - [subActionParams.comments.1]: expected value to equal [null]', + 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subActionParams.comments]: types that failed validation:\n - [subActionParams.comments.0.0.comment]: expected value of type [string] but got [undefined]\n - [subActionParams.comments.1]: expected value to equal [null]\n- [4.subAction]: expected value to equal [getChoices]', }); }); }); + + describe('getChoices', () => { + it('should fail when field is not provided', async () => { + await supertest + .post(`/api/actions/action/${simulatedActionId}/_execute`) + .set('kbn-xsrf', 'foo') + .send({ + params: { + subAction: 'getChoices', + subActionParams: {}, + }, + }) + .then((resp: any) => { + expect(resp.body).to.eql({ + actionId: simulatedActionId, + status: 'error', + retry: false, + message: + 'error validating action params: types that failed validation:\n- [0.subAction]: expected value to equal [getFields]\n- [1.subAction]: expected value to equal [getIncident]\n- [2.subAction]: expected value to equal [handshake]\n- [3.subAction]: expected value to equal [pushToService]\n- [4.subActionParams.fields]: expected value of type [array] but got [undefined]', + }); + }); + }); + }); }); describe('Execution', () => { @@ -376,6 +399,54 @@ export default function servicenowTest({ getService }: FtrProviderContext) { }, }); }); + + describe('getChoices', () => { + it('should get choices', async () => { + const { body: result } = await supertest + .post(`/api/actions/action/${simulatedActionId}/_execute`) + .set('kbn-xsrf', 'foo') + .send({ + params: { + subAction: 'getChoices', + subActionParams: { fields: ['priority'] }, + }, + }) + .expect(200); + + expect(proxyHaveBeenCalled).to.equal(true); + expect(result).to.eql({ + status: 'ok', + actionId: simulatedActionId, + data: [ + { + dependent_value: '', + label: '1 - Critical', + value: '1', + }, + { + dependent_value: '', + label: '2 - High', + value: '2', + }, + { + dependent_value: '', + label: '3 - Moderate', + value: '3', + }, + { + dependent_value: '', + label: '4 - Low', + value: '4', + }, + { + dependent_value: '', + label: '5 - Planning', + value: '5', + }, + ], + }); + }); + }); }); after(() => { diff --git a/x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/index.ts b/x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/index.ts index 2a256266697e62..e1502b496f77ed 100644 --- a/x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/index.ts +++ b/x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/index.ts @@ -10,7 +10,12 @@ import { setupSpacesAndUsers, tearDown } from '..'; // eslint-disable-next-line import/no-default-export export default function alertingTests({ loadTestFile, getService }: FtrProviderContext) { describe('Alerts', () => { - describe('legacy alerts', () => { + // FLAKY: https://github.com/elastic/kibana/issues/86952 + describe.skip('legacy alerts', () => { + before(async () => { + await setupSpacesAndUsers(getService); + }); + after(async () => { await tearDown(getService); }); diff --git a/x-pack/test/api_integration/apis/metrics_ui/log_entries.ts b/x-pack/test/api_integration/apis/metrics_ui/log_entries.ts index 79d5e683444321..0e61e3aaa0754a 100644 --- a/x-pack/test/api_integration/apis/metrics_ui/log_entries.ts +++ b/x-pack/test/api_integration/apis/metrics_ui/log_entries.ts @@ -6,21 +6,17 @@ import expect from '@kbn/expect'; import { v4 as uuidv4 } from 'uuid'; - -import { decodeOrThrow } from '../../../../plugins/infra/common/runtime_types'; - import { LOG_ENTRIES_PATH, logEntriesRequestRT, logEntriesResponseRT, } from '../../../../plugins/infra/common/http_api'; - import { - LogTimestampColumn, LogFieldColumn, LogMessageColumn, + LogTimestampColumn, } from '../../../../plugins/infra/common/log_entry'; - +import { decodeOrThrow } from '../../../../plugins/infra/common/runtime_types'; import { FtrProviderContext } from '../../ftr_provider_context'; const KEY_WITHIN_DATA_RANGE = { diff --git a/x-pack/test/apm_api_integration/tests/service_overview/dependencies/index.ts b/x-pack/test/apm_api_integration/tests/service_overview/dependencies/index.ts index b3e7e0672fc7fe..fe32e4493b6e4c 100644 --- a/x-pack/test/apm_api_integration/tests/service_overview/dependencies/index.ts +++ b/x-pack/test/apm_api_integration/tests/service_overview/dependencies/index.ts @@ -6,7 +6,7 @@ import expect from '@kbn/expect'; import url from 'url'; -import { sortBy, pick, last } from 'lodash'; +import { sortBy, pick, last, omit } from 'lodash'; import { ValuesType } from 'utility-types'; import { registry } from '../../../common/registry'; import { Maybe } from '../../../../../plugins/apm/typings/common'; @@ -306,7 +306,7 @@ export default function ApiTest({ getService }: FtrProviderContext) { before(async () => { response = await supertest.get( url.format({ - pathname: `/api/apm/services/opbeans-java/dependencies`, + pathname: `/api/apm/services/opbeans-python/dependencies`, query: { start, end, @@ -323,14 +323,41 @@ export default function ApiTest({ getService }: FtrProviderContext) { it('returns at least one item', () => { expect(response.body.length).to.be.greaterThan(0); + + expectSnapshot( + omit(response.body[0], [ + 'errorRate.timeseries', + 'throughput.timeseries', + 'latency.timeseries', + ]) + ).toMatchInline(` + Object { + "errorRate": Object { + "value": 0, + }, + "impact": 1.97910470896139, + "latency": Object { + "value": 1043.99015586546, + }, + "name": "redis", + "spanSubtype": "redis", + "spanType": "db", + "throughput": Object { + "value": 40.6333333333333, + }, + "type": "external", + } + `); }); it('returns the right names', () => { const names = response.body.map((item) => item.name); expectSnapshot(names.sort()).toMatchInline(` Array [ - "opbeans-go", + "elasticsearch", + "opbeans-java", "postgresql", + "redis", ] `); }); @@ -342,7 +369,7 @@ export default function ApiTest({ getService }: FtrProviderContext) { expectSnapshot(serviceNames.sort()).toMatchInline(` Array [ - "opbeans-go", + "opbeans-java", ] `); }); @@ -356,32 +383,89 @@ export default function ApiTest({ getService }: FtrProviderContext) { expectSnapshot(latencyValues).toMatchInline(` Array [ Object { - "latency": 38506.4285714286, - "name": "opbeans-go", + "latency": 2568.40816326531, + "name": "elasticsearch", + }, + Object { + "latency": 25593.875, + "name": "opbeans-java", }, Object { - "latency": 5908.77272727273, + "latency": 28885.3293963255, "name": "postgresql", }, + Object { + "latency": 1043.99015586546, + "name": "redis", + }, ] `); }); it('returns the right throughput values', () => { const throughputValues = sortBy( - response.body.map((item) => ({ name: item.name, latency: item.throughput.value })), + response.body.map((item) => ({ name: item.name, throughput: item.throughput.value })), 'name' ); expectSnapshot(throughputValues).toMatchInline(` Array [ Object { - "latency": 0.466666666666667, - "name": "opbeans-go", + "name": "elasticsearch", + "throughput": 13.0666666666667, + }, + Object { + "name": "opbeans-java", + "throughput": 0.533333333333333, }, Object { - "latency": 3.66666666666667, "name": "postgresql", + "throughput": 50.8, + }, + Object { + "name": "redis", + "throughput": 40.6333333333333, + }, + ] + `); + }); + + it('returns the right impact values', () => { + const impactValues = sortBy( + response.body.map((item) => ({ + name: item.name, + impact: item.impact, + latency: item.latency.value, + throughput: item.throughput.value, + })), + 'name' + ); + + expectSnapshot(impactValues).toMatchInline(` + Array [ + Object { + "impact": 1.36961744704522, + "latency": 2568.40816326531, + "name": "elasticsearch", + "throughput": 13.0666666666667, + }, + Object { + "impact": 0, + "latency": 25593.875, + "name": "opbeans-java", + "throughput": 0.533333333333333, + }, + Object { + "impact": 100, + "latency": 28885.3293963255, + "name": "postgresql", + "throughput": 50.8, + }, + Object { + "impact": 1.97910470896139, + "latency": 1043.99015586546, + "name": "redis", + "throughput": 40.6333333333333, }, ] `); diff --git a/x-pack/test/functional/apps/dashboard/_async_dashboard.ts b/x-pack/test/functional/apps/dashboard/_async_dashboard.ts index fceccb4609bd75..8851c83dea1ffa 100644 --- a/x-pack/test/functional/apps/dashboard/_async_dashboard.ts +++ b/x-pack/test/functional/apps/dashboard/_async_dashboard.ts @@ -12,7 +12,6 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { const retry = getService('retry'); const browser = getService('browser'); const kibanaServer = getService('kibanaServer'); - const esArchiver = getService('esArchiver'); const log = getService('log'); const pieChart = getService('pieChart'); const find = getService('find'); @@ -30,7 +29,6 @@ export default function ({ getService, getPageObjects }: FtrProviderContext) { describe('sample data dashboard', function describeIndexTests() { before(async () => { - await esArchiver.emptyKibanaIndex(); await PageObjects.common.sleep(5000); await PageObjects.common.navigateToUrl('home', '/tutorial_directory/sampleData', { useActualUrl: true, diff --git a/x-pack/test/functional/apps/dashboard/dashboard_lens_by_value.ts b/x-pack/test/functional/apps/dashboard/dashboard_lens_by_value.ts new file mode 100644 index 00000000000000..0a153aecec323e --- /dev/null +++ b/x-pack/test/functional/apps/dashboard/dashboard_lens_by_value.ts @@ -0,0 +1,72 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License; + * you may not use this file except in compliance with the Elastic License. + */ + +import expect from '@kbn/expect'; +import { FtrProviderContext } from '../../ftr_provider_context'; + +export default function ({ getPageObjects, getService }: FtrProviderContext) { + const PageObjects = getPageObjects(['common', 'dashboard', 'visualize', 'lens']); + + const find = getService('find'); + const esArchiver = getService('esArchiver'); + const dashboardPanelActions = getService('dashboardPanelActions'); + const dashboardVisualizations = getService('dashboardVisualizations'); + + describe('dashboard lens by value', function () { + before(async () => { + await esArchiver.loadIfNeeded('logstash_functional'); + await esArchiver.loadIfNeeded('lens/basic'); + await PageObjects.common.navigateToApp('dashboard'); + await PageObjects.dashboard.preserveCrossAppState(); + await PageObjects.dashboard.clickNewDashboard(); + }); + + it('can add a lens panel by value', async () => { + await dashboardVisualizations.ensureNewVisualizationDialogIsShowing(); + await PageObjects.lens.createAndAddLensFromDashboard({}); + const newPanelCount = await PageObjects.dashboard.getPanelCount(); + expect(newPanelCount).to.eql(1); + }); + + it('edits to a by value lens panel are properly applied', async () => { + await PageObjects.dashboard.waitForRenderComplete(); + await dashboardPanelActions.openContextMenu(); + await dashboardPanelActions.clickEdit(); + await PageObjects.lens.switchToVisualization('donut'); + await PageObjects.lens.saveAndReturn(); + await PageObjects.dashboard.waitForRenderComplete(); + + const pieExists = await find.existsByCssSelector('.lnsPieExpression__container'); + expect(pieExists).to.be(true); + }); + + it('editing and saving a lens by value panel retains number of panels', async () => { + const originalPanelCount = await PageObjects.dashboard.getPanelCount(); + await PageObjects.dashboard.waitForRenderComplete(); + await dashboardPanelActions.openContextMenu(); + await dashboardPanelActions.clickEdit(); + await PageObjects.lens.switchToVisualization('treemap'); + await PageObjects.lens.saveAndReturn(); + await PageObjects.dashboard.waitForRenderComplete(); + const newPanelCount = await PageObjects.dashboard.getPanelCount(); + expect(newPanelCount).to.eql(originalPanelCount); + }); + + it('updates panel on dashboard when a by value panel is saved to library', async () => { + const newTitle = 'look out library, here I come!'; + const originalPanelCount = await PageObjects.dashboard.getPanelCount(); + await PageObjects.dashboard.waitForRenderComplete(); + await dashboardPanelActions.openContextMenu(); + await dashboardPanelActions.clickEdit(); + await PageObjects.lens.save(newTitle, false, true); + await PageObjects.dashboard.waitForRenderComplete(); + const newPanelCount = await PageObjects.dashboard.getPanelCount(); + expect(newPanelCount).to.eql(originalPanelCount); + const titles = await PageObjects.dashboard.getPanelTitles(); + expect(titles.indexOf(newTitle)).to.not.be(-1); + }); + }); +} diff --git a/x-pack/test/functional/apps/dashboard/index.ts b/x-pack/test/functional/apps/dashboard/index.ts index 1ba87f89762a1b..d6c0c4394e24a4 100644 --- a/x-pack/test/functional/apps/dashboard/index.ts +++ b/x-pack/test/functional/apps/dashboard/index.ts @@ -15,5 +15,6 @@ export default function ({ loadTestFile }: FtrProviderContext) { loadTestFile(require.resolve('./drilldowns')); loadTestFile(require.resolve('./sync_colors')); loadTestFile(require.resolve('./_async_dashboard')); + loadTestFile(require.resolve('./dashboard_lens_by_value')); }); } diff --git a/x-pack/test/functional/apps/dashboard_mode/dashboard_empty_screen.js b/x-pack/test/functional/apps/dashboard_mode/dashboard_empty_screen.js index bd35374643e9b8..9408bee7dc8685 100644 --- a/x-pack/test/functional/apps/dashboard_mode/dashboard_empty_screen.js +++ b/x-pack/test/functional/apps/dashboard_mode/dashboard_empty_screen.js @@ -7,7 +7,6 @@ import expect from '@kbn/expect'; export default function ({ getPageObjects, getService }) { - const log = getService('log'); const testSubjects = getService('testSubjects'); const esArchiver = getService('esArchiver'); const dashboardVisualizations = getService('dashboardVisualizations'); @@ -27,40 +26,12 @@ export default function ({ getPageObjects, getService }) { await PageObjects.dashboard.gotoDashboardLandingPage(); }); - async function createAndAddLens(title, saveAsNew = false, redirectToOrigin = true) { - log.debug(`createAndAddLens(${title})`); - const inViewMode = await PageObjects.dashboard.getIsInViewMode(); - if (inViewMode) { - await PageObjects.dashboard.switchToEditMode(); - } - await PageObjects.visualize.clickLensWidget(); - await PageObjects.lens.goToTimeRange(); - await PageObjects.lens.configureDimension({ - dimension: 'lnsXY_xDimensionPanel > lns-empty-dimension', - operation: 'date_histogram', - field: '@timestamp', - }); - - await PageObjects.lens.configureDimension({ - dimension: 'lnsXY_yDimensionPanel > lns-empty-dimension', - operation: 'avg', - field: 'bytes', - }); - - await PageObjects.lens.configureDimension({ - dimension: 'lnsXY_splitDimensionPanel > lns-empty-dimension', - operation: 'terms', - field: 'ip', - }); - await PageObjects.lens.save(title, saveAsNew, redirectToOrigin); - } - it('adds Lens visualization to empty dashboard', async () => { const title = 'Dashboard Test Lens'; await testSubjects.exists('addVisualizationButton'); await testSubjects.click('addVisualizationButton'); await dashboardVisualizations.ensureNewVisualizationDialogIsShowing(); - await createAndAddLens(title); + await PageObjects.lens.createAndAddLensFromDashboard({ title, redirectToOrigin: true }); await PageObjects.dashboard.waitForRenderComplete(); await testSubjects.exists(`embeddablePanelHeading-${title}`); }); @@ -118,7 +89,7 @@ export default function ({ getPageObjects, getService }) { await testSubjects.exists('dashboardAddNewPanelButton'); await testSubjects.click('dashboardAddNewPanelButton'); await dashboardVisualizations.ensureNewVisualizationDialogIsShowing(); - await createAndAddLens(title, false, false); + await PageObjects.lens.createAndAddLensFromDashboard({ title }); await PageObjects.lens.notLinkedToOriginatingApp(); await PageObjects.common.navigateToApp('dashboard'); }); diff --git a/x-pack/test/functional/apps/ml/data_frame_analytics/cloning.ts b/x-pack/test/functional/apps/ml/data_frame_analytics/cloning.ts index 532de930bc1a16..91ca0e6f32fd8a 100644 --- a/x-pack/test/functional/apps/ml/data_frame_analytics/cloning.ts +++ b/x-pack/test/functional/apps/ml/data_frame_analytics/cloning.ts @@ -13,7 +13,8 @@ export default function ({ getService }: FtrProviderContext) { const esArchiver = getService('esArchiver'); const ml = getService('ml'); - describe('jobs cloning supported by UI form', function () { + // Failing ES promotion, see https://github.com/elastic/kibana/issues/89980 + describe.skip('jobs cloning supported by UI form', function () { const testDataList: Array<{ suiteTitle: string; archive: string; diff --git a/x-pack/test/functional/es_archives/visualize/default/data.json b/x-pack/test/functional/es_archives/visualize/default/data.json index 26b033e28b4da7..fe29bad0fa3814 100644 --- a/x-pack/test/functional/es_archives/visualize/default/data.json +++ b/x-pack/test/functional/es_archives/visualize/default/data.json @@ -125,8 +125,26 @@ { "type": "doc", "value": { - "index": ".kibana", - "type": "doc", + "id": "custom-space:index-pattern:metricbeat-*", + "index": ".kibana_1", + "source": { + "index-pattern": { + "fieldFormatMap": "{\"aerospike.namespace.device.available.pct\":{\"id\":\"percent\",\"params\":{}},\"aerospike.namespace.device.free.pct\":{\"id\":\"percent\",\"params\":{}},\"aerospike.namespace.device.total.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aerospike.namespace.device.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aerospike.namespace.memory.free.pct\":{\"id\":\"percent\",\"params\":{}},\"aerospike.namespace.memory.used.data.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aerospike.namespace.memory.used.index.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aerospike.namespace.memory.used.sindex.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aerospike.namespace.memory.used.total.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.ec2.diskio.read.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.ec2.diskio.write.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.ec2.network.in.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.ec2.network.out.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.rds.cpu.total.pct\":{\"id\":\"percent\",\"params\":{}},\"aws.rds.disk_usage.bin_log.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.rds.free_local_storage.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.rds.free_storage.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.rds.freeable_memory.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.rds.latency.commit\":{\"id\":\"duration\",\"params\":{}},\"aws.rds.latency.ddl\":{\"id\":\"duration\",\"params\":{}},\"aws.rds.latency.dml\":{\"id\":\"duration\",\"params\":{}},\"aws.rds.latency.insert\":{\"id\":\"duration\",\"params\":{}},\"aws.rds.latency.read\":{\"id\":\"duration\",\"params\":{}},\"aws.rds.latency.select\":{\"id\":\"duration\",\"params\":{}},\"aws.rds.latency.update\":{\"id\":\"duration\",\"params\":{}},\"aws.rds.latency.write\":{\"id\":\"duration\",\"params\":{}},\"aws.rds.replica_lag.sec\":{\"id\":\"duration\",\"params\":{}},\"aws.rds.swap_usage.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.rds.volume_used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.s3_daily_storage.bucket.size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.s3_request.downloaded.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.s3_request.latency.first_byte.ms\":{\"id\":\"duration\",\"params\":{}},\"aws.s3_request.latency.total_request.ms\":{\"id\":\"duration\",\"params\":{}},\"aws.s3_request.requests.select_returned.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.s3_request.requests.select_scanned.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.s3_request.uploaded.bytes\":{\"id\":\"bytes\",\"params\":{}},\"aws.sqs.oldest_message_age.sec\":{\"id\":\"duration\",\"params\":{}},\"aws.sqs.sent_message_size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.cluster_disk.available.bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.cluster_disk.total.bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.cluster_disk.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.cluster_status.degraded.ratio\":{\"id\":\"percent\",\"params\":{}},\"ceph.cluster_status.misplace.ratio\":{\"id\":\"percent\",\"params\":{}},\"ceph.cluster_status.pg.avail_bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.cluster_status.pg.data_bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.cluster_status.pg.total_bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.cluster_status.pg.used_bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.cluster_status.traffic.read_bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.cluster_status.traffic.write_bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.monitor_health.store_stats.log.bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.monitor_health.store_stats.misc.bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.monitor_health.store_stats.sst.bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.monitor_health.store_stats.total.bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.osd_df.available.bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.osd_df.total.byte\":{\"id\":\"bytes\",\"params\":{}},\"ceph.osd_df.used.byte\":{\"id\":\"bytes\",\"params\":{}},\"ceph.osd_df.used.pct\":{\"id\":\"percent\",\"params\":{}},\"ceph.pool_disk.stats.available.bytes\":{\"id\":\"bytes\",\"params\":{}},\"ceph.pool_disk.stats.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"client.bytes\":{\"id\":\"bytes\",\"params\":{}},\"client.nat.port\":{\"id\":\"string\",\"params\":{}},\"client.port\":{\"id\":\"string\",\"params\":{}},\"coredns.stats.dns.request.duration.ns.sum\":{\"id\":\"duration\",\"params\":{}},\"couchbase.bucket.data.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.bucket.disk.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.bucket.memory.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.bucket.quota.ram.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.bucket.quota.use.pct\":{\"id\":\"percent\",\"params\":{}},\"couchbase.cluster.hdd.free.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.cluster.hdd.quota.total.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.cluster.hdd.total.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.cluster.hdd.used.by_data.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.cluster.hdd.used.value.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.cluster.ram.quota.total.per_node.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.cluster.ram.quota.total.value.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.cluster.ram.quota.used.per_node.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.cluster.ram.quota.used.value.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.cluster.ram.total.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.cluster.ram.used.by_data.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.cluster.ram.used.value.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.node.couch.docs.data_size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.node.couch.docs.disk_size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"couchbase.node.mcd_memory.allocated.bytes\":{\"id\":\"bytes\",\"params\":{}},\"destination.bytes\":{\"id\":\"bytes\",\"params\":{}},\"destination.nat.port\":{\"id\":\"string\",\"params\":{}},\"destination.port\":{\"id\":\"string\",\"params\":{}},\"docker.cpu.core.*.pct\":{\"id\":\"percent\",\"params\":{}},\"docker.cpu.kernel.pct\":{\"id\":\"percent\",\"params\":{}},\"docker.cpu.system.pct\":{\"id\":\"percent\",\"params\":{}},\"docker.cpu.total.pct\":{\"id\":\"percent\",\"params\":{}},\"docker.cpu.user.pct\":{\"id\":\"percent\",\"params\":{}},\"docker.diskio.read.bytes\":{\"id\":\"bytes\",\"params\":{}},\"docker.diskio.summary.bytes\":{\"id\":\"bytes\",\"params\":{}},\"docker.diskio.write.bytes\":{\"id\":\"bytes\",\"params\":{}},\"docker.memory.commit.peak\":{\"id\":\"bytes\",\"params\":{}},\"docker.memory.commit.total\":{\"id\":\"bytes\",\"params\":{}},\"docker.memory.limit\":{\"id\":\"bytes\",\"params\":{}},\"docker.memory.private_working_set.total\":{\"id\":\"bytes\",\"params\":{}},\"docker.memory.rss.pct\":{\"id\":\"percent\",\"params\":{}},\"docker.memory.rss.total\":{\"id\":\"bytes\",\"params\":{}},\"docker.memory.usage.max\":{\"id\":\"bytes\",\"params\":{}},\"docker.memory.usage.pct\":{\"id\":\"percent\",\"params\":{}},\"docker.memory.usage.total\":{\"id\":\"bytes\",\"params\":{}},\"docker.network.in.bytes\":{\"id\":\"bytes\",\"params\":{}},\"docker.network.inbound.bytes\":{\"id\":\"bytes\",\"params\":{}},\"docker.network.out.bytes\":{\"id\":\"bytes\",\"params\":{}},\"docker.network.outbound.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.index.summary.primaries.segments.memory.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.index.summary.primaries.store.size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.index.summary.total.segments.memory.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.index.summary.total.store.size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.index.total.segments.memory.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.index.total.store.size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.jvm.memory.heap.init.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.jvm.memory.heap.max.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.jvm.memory.nonheap.init.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.jvm.memory.nonheap.max.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.fs.summary.available.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.fs.summary.free.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.fs.summary.total.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.indices.segments.memory.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.jvm.mem.pools.old.max.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.jvm.mem.pools.old.peak.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.jvm.mem.pools.old.peak_max.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.jvm.mem.pools.old.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.jvm.mem.pools.survivor.max.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.jvm.mem.pools.survivor.peak.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.jvm.mem.pools.survivor.peak_max.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.jvm.mem.pools.survivor.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.jvm.mem.pools.young.max.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.jvm.mem.pools.young.peak.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.jvm.mem.pools.young.peak_max.bytes\":{\"id\":\"bytes\",\"params\":{}},\"elasticsearch.node.stats.jvm.mem.pools.young.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"etcd.disk.mvcc_db_total_size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"etcd.memory.go_memstats_alloc.bytes\":{\"id\":\"bytes\",\"params\":{}},\"etcd.network.client_grpc_received.bytes\":{\"id\":\"bytes\",\"params\":{}},\"etcd.network.client_grpc_sent.bytes\":{\"id\":\"bytes\",\"params\":{}},\"event.duration\":{\"id\":\"duration\",\"params\":{\"inputFormat\":\"nanoseconds\",\"outputFormat\":\"asMilliseconds\",\"outputPrecision\":1}},\"event.sequence\":{\"id\":\"string\",\"params\":{}},\"event.severity\":{\"id\":\"string\",\"params\":{}},\"golang.heap.allocations.active\":{\"id\":\"bytes\",\"params\":{}},\"golang.heap.allocations.allocated\":{\"id\":\"bytes\",\"params\":{}},\"golang.heap.allocations.idle\":{\"id\":\"bytes\",\"params\":{}},\"golang.heap.allocations.total\":{\"id\":\"bytes\",\"params\":{}},\"golang.heap.gc.next_gc_limit\":{\"id\":\"bytes\",\"params\":{}},\"golang.heap.system.obtained\":{\"id\":\"bytes\",\"params\":{}},\"golang.heap.system.released\":{\"id\":\"bytes\",\"params\":{}},\"golang.heap.system.stack\":{\"id\":\"bytes\",\"params\":{}},\"golang.heap.system.total\":{\"id\":\"bytes\",\"params\":{}},\"haproxy.info.idle.pct\":{\"id\":\"percent\",\"params\":{}},\"haproxy.info.memory.max.bytes\":{\"id\":\"bytes\",\"params\":{}},\"haproxy.info.ssl.frontend.session_reuse.pct\":{\"id\":\"percent\",\"params\":{}},\"haproxy.stat.compressor.bypassed.bytes\":{\"id\":\"bytes\",\"params\":{}},\"haproxy.stat.compressor.in.bytes\":{\"id\":\"bytes\",\"params\":{}},\"haproxy.stat.compressor.out.bytes\":{\"id\":\"bytes\",\"params\":{}},\"haproxy.stat.compressor.response.bytes\":{\"id\":\"bytes\",\"params\":{}},\"haproxy.stat.in.bytes\":{\"id\":\"bytes\",\"params\":{}},\"haproxy.stat.out.bytes\":{\"id\":\"bytes\",\"params\":{}},\"haproxy.stat.throttle.pct\":{\"id\":\"percent\",\"params\":{}},\"http.request.body.bytes\":{\"id\":\"bytes\",\"params\":{}},\"http.request.bytes\":{\"id\":\"bytes\",\"params\":{}},\"http.response.body.bytes\":{\"id\":\"bytes\",\"params\":{}},\"http.response.bytes\":{\"id\":\"bytes\",\"params\":{}},\"http.response.status_code\":{\"id\":\"string\",\"params\":{}},\"kibana.stats.process.memory.heap.size_limit.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kibana.stats.process.memory.heap.total.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kibana.stats.process.memory.heap.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.apiserver.http.request.size.bytes.sum\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.apiserver.http.response.size.bytes.sum\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.apiserver.process.memory.resident.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.apiserver.process.memory.virtual.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.container.cpu.usage.limit.pct\":{\"id\":\"percent\",\"params\":{}},\"kubernetes.container.cpu.usage.node.pct\":{\"id\":\"percent\",\"params\":{}},\"kubernetes.container.logs.available.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.container.logs.capacity.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.container.logs.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.container.memory.available.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.container.memory.limit.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.container.memory.request.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.container.memory.rss.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.container.memory.usage.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.container.memory.usage.limit.pct\":{\"id\":\"percent\",\"params\":{}},\"kubernetes.container.memory.usage.node.pct\":{\"id\":\"percent\",\"params\":{}},\"kubernetes.container.memory.workingset.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.container.rootfs.available.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.container.rootfs.capacity.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.container.rootfs.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.controllermanager.http.request.size.bytes.sum\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.controllermanager.http.response.size.bytes.sum\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.controllermanager.process.memory.resident.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.controllermanager.process.memory.virtual.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.node.fs.available.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.node.fs.capacity.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.node.fs.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.node.memory.allocatable.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.node.memory.available.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.node.memory.capacity.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.node.memory.rss.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.node.memory.usage.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.node.memory.workingset.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.node.network.rx.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.node.network.tx.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.node.runtime.imagefs.available.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.node.runtime.imagefs.capacity.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.node.runtime.imagefs.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.pod.cpu.usage.limit.pct\":{\"id\":\"percent\",\"params\":{}},\"kubernetes.pod.cpu.usage.node.pct\":{\"id\":\"percent\",\"params\":{}},\"kubernetes.pod.memory.available.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.pod.memory.rss.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.pod.memory.usage.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.pod.memory.usage.limit.pct\":{\"id\":\"percent\",\"params\":{}},\"kubernetes.pod.memory.usage.node.pct\":{\"id\":\"percent\",\"params\":{}},\"kubernetes.pod.memory.working_set.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.pod.network.rx.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.pod.network.tx.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.proxy.http.request.size.bytes.sum\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.proxy.http.response.size.bytes.sum\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.proxy.process.memory.resident.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.proxy.process.memory.virtual.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.scheduler.http.request.size.bytes.sum\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.scheduler.http.response.size.bytes.sum\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.scheduler.process.memory.resident.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.scheduler.process.memory.virtual.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.system.memory.rss.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.system.memory.usage.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.system.memory.workingset.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.volume.fs.available.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.volume.fs.capacity.bytes\":{\"id\":\"bytes\",\"params\":{}},\"kubernetes.volume.fs.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.dbstats.avg_obj_size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.dbstats.data_size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.dbstats.extent_free_list.size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.dbstats.file_size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.dbstats.index_size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.dbstats.storage_size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.replstatus.headroom.max\":{\"id\":\"duration\",\"params\":{}},\"mongodb.replstatus.headroom.min\":{\"id\":\"duration\",\"params\":{}},\"mongodb.replstatus.lag.max\":{\"id\":\"duration\",\"params\":{}},\"mongodb.replstatus.lag.min\":{\"id\":\"duration\",\"params\":{}},\"mongodb.replstatus.oplog.size.allocated\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.replstatus.oplog.size.used\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.status.extra_info.heap_usage.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.status.network.in.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.status.network.out.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.status.wired_tiger.cache.dirty.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.status.wired_tiger.cache.maximum.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.status.wired_tiger.cache.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.status.wired_tiger.log.max_file_size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.status.wired_tiger.log.size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mongodb.status.wired_tiger.log.write.bytes\":{\"id\":\"bytes\",\"params\":{}},\"mysql.status.bytes.received\":{\"id\":\"bytes\",\"params\":{}},\"mysql.status.bytes.sent\":{\"id\":\"bytes\",\"params\":{}},\"nats.stats.cpu\":{\"id\":\"percent\",\"params\":{}},\"nats.stats.in.bytes\":{\"id\":\"bytes\",\"params\":{}},\"nats.stats.mem.bytes\":{\"id\":\"bytes\",\"params\":{}},\"nats.stats.out.bytes\":{\"id\":\"bytes\",\"params\":{}},\"nats.stats.uptime\":{\"id\":\"duration\",\"params\":{}},\"nats.subscriptions.cache.hit_rate\":{\"id\":\"percent\",\"params\":{}},\"network.bytes\":{\"id\":\"bytes\",\"params\":{}},\"oracle.tablespace.data_file.size.bytes\":{\"id\":\"bytes\",\"params\":{}},\"oracle.tablespace.data_file.size.free.bytes\":{\"id\":\"bytes\",\"params\":{}},\"oracle.tablespace.data_file.size.max.bytes\":{\"id\":\"bytes\",\"params\":{}},\"oracle.tablespace.space.free.bytes\":{\"id\":\"bytes\",\"params\":{}},\"oracle.tablespace.space.total.bytes\":{\"id\":\"bytes\",\"params\":{}},\"oracle.tablespace.space.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"process.pgid\":{\"id\":\"string\",\"params\":{}},\"process.pid\":{\"id\":\"string\",\"params\":{}},\"process.ppid\":{\"id\":\"string\",\"params\":{}},\"process.thread.id\":{\"id\":\"string\",\"params\":{}},\"rabbitmq.connection.frame_max\":{\"id\":\"bytes\",\"params\":{}},\"rabbitmq.node.disk.free.bytes\":{\"id\":\"bytes\",\"params\":{}},\"rabbitmq.node.disk.free.limit.bytes\":{\"id\":\"bytes\",\"params\":{}},\"rabbitmq.node.gc.reclaimed.bytes\":{\"id\":\"bytes\",\"params\":{}},\"rabbitmq.node.io.read.bytes\":{\"id\":\"bytes\",\"params\":{}},\"rabbitmq.node.io.write.bytes\":{\"id\":\"bytes\",\"params\":{}},\"rabbitmq.node.mem.limit.bytes\":{\"id\":\"bytes\",\"params\":{}},\"rabbitmq.queue.consumers.utilisation.pct\":{\"id\":\"percent\",\"params\":{}},\"rabbitmq.queue.memory.bytes\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.memory.allocator_stats.active\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.memory.allocator_stats.allocated\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.memory.allocator_stats.fragmentation.bytes\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.memory.allocator_stats.resident\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.memory.allocator_stats.rss.bytes\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.memory.fragmentation.bytes\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.memory.max.value\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.memory.used.dataset\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.memory.used.lua\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.memory.used.peak\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.memory.used.rss\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.memory.used.value\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.persistence.aof.buffer.size\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.persistence.aof.copy_on_write.last_size\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.persistence.aof.rewrite.buffer.size\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.persistence.aof.rewrite.current_time.sec\":{\"id\":\"duration\",\"params\":{}},\"redis.info.persistence.aof.rewrite.last_time.sec\":{\"id\":\"duration\",\"params\":{}},\"redis.info.persistence.aof.size.base\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.persistence.aof.size.current\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.persistence.rdb.bgsave.current_time.sec\":{\"id\":\"duration\",\"params\":{}},\"redis.info.persistence.rdb.bgsave.last_time.sec\":{\"id\":\"duration\",\"params\":{}},\"redis.info.persistence.rdb.copy_on_write.last_size\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.replication.backlog.size\":{\"id\":\"bytes\",\"params\":{}},\"redis.info.replication.master.last_io_seconds_ago\":{\"id\":\"duration\",\"params\":{}},\"redis.info.replication.master.sync.last_io_seconds_ago\":{\"id\":\"duration\",\"params\":{}},\"redis.info.replication.master.sync.left_bytes\":{\"id\":\"bytes\",\"params\":{}},\"server.bytes\":{\"id\":\"bytes\",\"params\":{}},\"server.nat.port\":{\"id\":\"string\",\"params\":{}},\"server.port\":{\"id\":\"string\",\"params\":{}},\"source.bytes\":{\"id\":\"bytes\",\"params\":{}},\"source.nat.port\":{\"id\":\"string\",\"params\":{}},\"source.port\":{\"id\":\"string\",\"params\":{}},\"system.core.idle.pct\":{\"id\":\"percent\",\"params\":{}},\"system.core.iowait.pct\":{\"id\":\"percent\",\"params\":{}},\"system.core.irq.pct\":{\"id\":\"percent\",\"params\":{}},\"system.core.nice.pct\":{\"id\":\"percent\",\"params\":{}},\"system.core.softirq.pct\":{\"id\":\"percent\",\"params\":{}},\"system.core.steal.pct\":{\"id\":\"percent\",\"params\":{}},\"system.core.system.pct\":{\"id\":\"percent\",\"params\":{}},\"system.core.user.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.idle.norm.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.idle.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.iowait.norm.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.iowait.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.irq.norm.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.irq.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.nice.norm.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.nice.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.softirq.norm.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.softirq.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.steal.norm.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.steal.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.system.norm.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.system.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.total.norm.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.total.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.user.norm.pct\":{\"id\":\"percent\",\"params\":{}},\"system.cpu.user.pct\":{\"id\":\"percent\",\"params\":{}},\"system.diskio.iostat.read.per_sec.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.diskio.iostat.write.per_sec.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.diskio.read.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.diskio.write.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.entropy.pct\":{\"id\":\"percent\",\"params\":{}},\"system.filesystem.available\":{\"id\":\"bytes\",\"params\":{}},\"system.filesystem.free\":{\"id\":\"bytes\",\"params\":{}},\"system.filesystem.total\":{\"id\":\"bytes\",\"params\":{}},\"system.filesystem.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.filesystem.used.pct\":{\"id\":\"percent\",\"params\":{}},\"system.fsstat.total_size.free\":{\"id\":\"bytes\",\"params\":{}},\"system.fsstat.total_size.total\":{\"id\":\"bytes\",\"params\":{}},\"system.fsstat.total_size.used\":{\"id\":\"bytes\",\"params\":{}},\"system.memory.actual.free\":{\"id\":\"bytes\",\"params\":{}},\"system.memory.actual.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.memory.actual.used.pct\":{\"id\":\"percent\",\"params\":{}},\"system.memory.free\":{\"id\":\"bytes\",\"params\":{}},\"system.memory.hugepages.default_size\":{\"id\":\"bytes\",\"params\":{}},\"system.memory.hugepages.free\":{\"id\":\"number\",\"params\":{}},\"system.memory.hugepages.reserved\":{\"id\":\"number\",\"params\":{}},\"system.memory.hugepages.surplus\":{\"id\":\"number\",\"params\":{}},\"system.memory.hugepages.total\":{\"id\":\"number\",\"params\":{}},\"system.memory.hugepages.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.memory.hugepages.used.pct\":{\"id\":\"percent\",\"params\":{}},\"system.memory.swap.free\":{\"id\":\"bytes\",\"params\":{}},\"system.memory.swap.total\":{\"id\":\"bytes\",\"params\":{}},\"system.memory.swap.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.memory.swap.used.pct\":{\"id\":\"percent\",\"params\":{}},\"system.memory.total\":{\"id\":\"bytes\",\"params\":{}},\"system.memory.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.memory.used.pct\":{\"id\":\"percent\",\"params\":{}},\"system.network.in.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.network.out.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.blkio.total.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.kmem.limit.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.kmem.usage.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.kmem.usage.max.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.kmem_tcp.limit.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.kmem_tcp.usage.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.kmem_tcp.usage.max.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.mem.limit.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.mem.usage.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.mem.usage.max.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.memsw.limit.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.memsw.usage.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.memsw.usage.max.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.stats.active_anon.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.stats.active_file.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.stats.cache.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.stats.hierarchical_memory_limit.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.stats.hierarchical_memsw_limit.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.stats.inactive_anon.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.stats.inactive_file.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.stats.mapped_file.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.stats.rss.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.stats.rss_huge.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.stats.swap.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cgroup.memory.stats.unevictable.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.cpu.total.norm.pct\":{\"id\":\"percent\",\"params\":{}},\"system.process.cpu.total.pct\":{\"id\":\"percent\",\"params\":{}},\"system.process.memory.rss.bytes\":{\"id\":\"bytes\",\"params\":{}},\"system.process.memory.rss.pct\":{\"id\":\"percent\",\"params\":{}},\"system.process.memory.share\":{\"id\":\"bytes\",\"params\":{}},\"system.process.memory.size\":{\"id\":\"bytes\",\"params\":{}},\"system.socket.summary.tcp.memory\":{\"id\":\"bytes\",\"params\":{}},\"system.socket.summary.udp.memory\":{\"id\":\"bytes\",\"params\":{}},\"system.uptime.duration.ms\":{\"id\":\"duration\",\"params\":{\"inputFormat\":\"milliseconds\"}},\"url.port\":{\"id\":\"string\",\"params\":{}},\"vsphere.datastore.capacity.free.bytes\":{\"id\":\"bytes\",\"params\":{}},\"vsphere.datastore.capacity.total.bytes\":{\"id\":\"bytes\",\"params\":{}},\"vsphere.datastore.capacity.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"vsphere.datastore.capacity.used.pct\":{\"id\":\"percent\",\"params\":{}},\"vsphere.host.memory.free.bytes\":{\"id\":\"bytes\",\"params\":{}},\"vsphere.host.memory.total.bytes\":{\"id\":\"bytes\",\"params\":{}},\"vsphere.host.memory.used.bytes\":{\"id\":\"bytes\",\"params\":{}},\"vsphere.virtualmachine.memory.free.guest.bytes\":{\"id\":\"bytes\",\"params\":{}},\"vsphere.virtualmachine.memory.total.guest.bytes\":{\"id\":\"bytes\",\"params\":{}},\"vsphere.virtualmachine.memory.used.guest.bytes\":{\"id\":\"bytes\",\"params\":{}},\"vsphere.virtualmachine.memory.used.host.bytes\":{\"id\":\"bytes\",\"params\":{}},\"windows.service.uptime.ms\":{\"id\":\"duration\",\"params\":{\"inputFormat\":\"milliseconds\"}}}", + "timeFieldName": "@timestamp", + "title": "metricbeat-*" + }, + "migrationVersion": { + "index-pattern": "7.6.0" + }, + "type": "index-pattern", + "updated_at": "2020-01-22T15:34:59.061Z" + } + } +} + +{ + "type": "doc", + "value": { "id": "index-pattern:logstash-*", "index": ".kibana_1", "source": { @@ -279,4 +297,4 @@ "updated_at": "2019-07-17T17:54:26.378Z" } } -} +} \ No newline at end of file diff --git a/x-pack/test/functional/page_objects/lens_page.ts b/x-pack/test/functional/page_objects/lens_page.ts index 17f9fb036129a7..292e91866d6903 100644 --- a/x-pack/test/functional/page_objects/lens_page.ts +++ b/x-pack/test/functional/page_objects/lens_page.ts @@ -16,7 +16,7 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont const find = getService('find'); const comboBox = getService('comboBox'); const browser = getService('browser'); - const PageObjects = getPageObjects(['header', 'timePicker', 'common', 'visualize']); + const PageObjects = getPageObjects(['header', 'timePicker', 'common', 'visualize', 'dashboard']); return logWrapper('lensPage', log, { /** @@ -586,5 +586,49 @@ export function LensPageProvider({ getService, getPageObjects }: FtrProviderCont // TODO: target dimensionTrigger color element after merging https://github.com/elastic/kibana/pull/76871 await testSubjects.getAttribute('colorPickerAnchor', color); }, + + /** + * Creates and saves a lens visualization from a dashboard + * + * @param title - title for the new lens. If left undefined, the panel will be created by value + * @param redirectToOrigin - whether to redirect back to the dashboard after saving the panel + */ + async createAndAddLensFromDashboard({ + title, + redirectToOrigin, + }: { + title?: string; + redirectToOrigin?: boolean; + }) { + log.debug(`createAndAddLens${title}`); + const inViewMode = await PageObjects.dashboard.getIsInViewMode(); + if (inViewMode) { + await PageObjects.dashboard.switchToEditMode(); + } + await PageObjects.visualize.clickLensWidget(); + await this.goToTimeRange(); + await this.configureDimension({ + dimension: 'lnsXY_xDimensionPanel > lns-empty-dimension', + operation: 'date_histogram', + field: '@timestamp', + }); + + await this.configureDimension({ + dimension: 'lnsXY_yDimensionPanel > lns-empty-dimension', + operation: 'avg', + field: 'bytes', + }); + + await this.configureDimension({ + dimension: 'lnsXY_splitDimensionPanel > lns-empty-dimension', + operation: 'terms', + field: 'ip', + }); + if (title) { + await this.save(title, false, redirectToOrigin); + } else { + await this.saveAndReturn(); + } + }, }); } diff --git a/x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/details.ts b/x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/details.ts index 35e2e039690233..07e0ef62ea4db5 100644 --- a/x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/details.ts +++ b/x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/details.ts @@ -21,6 +21,7 @@ export default ({ getPageObjects, getService }: FtrProviderContext) => { const retry = getService('retry'); const find = getService('find'); const supertest = getService('supertest'); + const comboBox = getService('comboBox'); const objectRemover = new ObjectRemover(supertest); async function createActionManualCleanup(overwrites: Record = {}) { @@ -313,15 +314,70 @@ export default ({ getPageObjects, getService }: FtrProviderContext) => { describe('Edit alert with deleted connector', function () { const testRunUuid = uuid.v4(); - after(async () => { + afterEach(async () => { await objectRemover.removeAll(); }); - it('should show and update deleted connectors', async () => { + it('should show and update deleted connectors when there are existing connectors of the same type', async () => { const action = await createActionManualCleanup({ name: `slack-${testRunUuid}-${0}`, }); + await pageObjects.common.navigateToApp('triggersActions'); + const alert = await createAlwaysFiringAlert({ + name: testRunUuid, + actions: [ + { + group: 'default', + id: action.id, + params: { level: 'info', message: ' {{context.message}}' }, + }, + ], + }); + + // refresh to see alert + await browser.refresh(); + await pageObjects.header.waitUntilLoadingHasFinished(); + + // verify content + await testSubjects.existOrFail('alertsList'); + + // delete connector + await pageObjects.triggersActionsUI.changeTabs('connectorsTab'); + await pageObjects.triggersActionsUI.searchConnectors(action.name); + await testSubjects.click('deleteConnector'); + await testSubjects.existOrFail('deleteIdsConfirmation'); + await testSubjects.click('deleteIdsConfirmation > confirmModalConfirmButton'); + await testSubjects.missingOrFail('deleteIdsConfirmation'); + + const toastTitle = await pageObjects.common.closeToast(); + expect(toastTitle).to.eql('Deleted 1 connector'); + + // click on first alert + await pageObjects.triggersActionsUI.changeTabs('alertsTab'); + await pageObjects.triggersActionsUI.clickOnAlertInAlertsList(alert.name); + + const editButton = await testSubjects.find('openEditAlertFlyoutButton'); + await editButton.click(); + expect(await testSubjects.exists('hasActionsDisabled')).to.eql(false); + + expect(await testSubjects.exists('addNewActionConnectorActionGroup-0')).to.eql(false); + expect(await testSubjects.exists('alertActionAccordion-0')).to.eql(true); + + await comboBox.set('selectActionConnector-.slack-0', 'Slack#xyztest (preconfigured)'); + expect(await testSubjects.exists('addNewActionConnectorActionGroup-0')).to.eql(true); + }); + + it('should show and update deleted connectors when there are no existing connectors of the same type', async () => { + const action = await createActionManualCleanup({ + name: `index-${testRunUuid}-${0}`, + actionTypeId: '.index', + config: { + index: `index-${testRunUuid}-${0}`, + }, + secrets: {}, + }); + await pageObjects.common.navigateToApp('triggersActions'); const alert = await createAlwaysFiringAlert({ name: testRunUuid, @@ -373,7 +429,17 @@ export default ({ getPageObjects, getService }: FtrProviderContext) => { await testSubjects.click('createActionConnectorButton-0'); await testSubjects.existOrFail('connectorAddModal'); await testSubjects.setValue('nameInput', 'new connector'); - await testSubjects.setValue('slackWebhookUrlInput', 'https://test'); + await retry.try(async () => { + // At times we find the driver controlling the ComboBox in tests + // can select the wrong item, this ensures we always select the correct index + await comboBox.set('connectorIndexesComboBox', 'test-index'); + expect( + await comboBox.isOptionSelected( + await testSubjects.find('connectorIndexesComboBox'), + 'test-index' + ) + ).to.be(true); + }); await testSubjects.click('connectorAddModal > saveActionButtonModal'); await testSubjects.missingOrFail('deleteIdsConfirmation'); diff --git a/x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/index.ts b/x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/index.ts index 8600cb6c852f50..54ce70911903b1 100644 --- a/x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/index.ts +++ b/x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/index.ts @@ -10,9 +10,9 @@ export default ({ loadTestFile, getService }: FtrProviderContext) => { describe('Actions and Triggers app', function () { this.tags('ciGroup10'); loadTestFile(require.resolve('./home_page')); - loadTestFile(require.resolve('./connectors')); loadTestFile(require.resolve('./alerts_list')); loadTestFile(require.resolve('./alert_create_flyout')); loadTestFile(require.resolve('./details')); + loadTestFile(require.resolve('./connectors')); }); }; diff --git a/x-pack/test/load/runner.ts b/x-pack/test/load/runner.ts index e9a1cadfddc558..52448a6b32a998 100644 --- a/x-pack/test/load/runner.ts +++ b/x-pack/test/load/runner.ts @@ -7,23 +7,57 @@ import { withProcRunner } from '@kbn/dev-utils'; import { resolve } from 'path'; import { REPO_ROOT } from '@kbn/utils'; +import Fs from 'fs'; +import { createFlagError } from '@kbn/dev-utils'; import { FtrProviderContext } from './../functional/ftr_provider_context'; +const baseSimulationPath = 'src/test/scala/org/kibanaLoadTest/simulation'; +const simulationPackage = 'org.kibanaLoadTest.simulation'; +const simulationFIleExtension = '.scala'; +const gatlingProjectRootPath: string = + process.env.GATLING_PROJECT_PATH || resolve(REPO_ROOT, '../kibana-load-testing'); +const simulationEntry: string = process.env.GATLING_SIMULATIONS || 'DemoJourney'; + +if (!Fs.existsSync(gatlingProjectRootPath)) { + throw createFlagError( + `Incorrect path to load testing project: '${gatlingProjectRootPath}'\n + Clone 'elastic/kibana-load-testing' and set path using 'GATLING_PROJECT_PATH' env var` + ); +} + +const dropEmptyLines = (s: string) => s.split(',').filter((i) => i.length > 0); +const simulationClasses = dropEmptyLines(simulationEntry); +const simulationsRootPath = resolve(gatlingProjectRootPath, baseSimulationPath); + +simulationClasses.map((className) => { + const simulationClassPath = resolve( + simulationsRootPath, + className.replace('.', '/') + simulationFIleExtension + ); + if (!Fs.existsSync(simulationClassPath)) { + throw createFlagError(`Simulation class is not found: '${simulationClassPath}'`); + } +}); + +/** + * + * GatlingTestRunner is used to run load simulation against local Kibana instance + * + * Use GATLING_SIMULATIONS to pass comma-separated class names + * Use GATLING_PROJECT_PATH to override path to 'kibana-load-testing' project + */ export async function GatlingTestRunner({ getService }: FtrProviderContext) { const log = getService('log'); - const gatlingProjectRootPath = resolve(REPO_ROOT, '../kibana-load-testing'); await withProcRunner(log, async (procs) => { - await procs.run('gatling', { + await procs.run('mvn: clean compile', { cmd: 'mvn', args: [ - 'clean', - '-q', + '-Dmaven.wagon.http.retryHandler.count=3', '-Dmaven.test.failure.ignore=true', - 'compile', - 'gatling:test', '-q', - '-Dgatling.simulationClass=org.kibanaLoadTest.simulation.DemoJourney', + 'clean', + 'compile', ], cwd: gatlingProjectRootPath, env: { @@ -31,5 +65,20 @@ export async function GatlingTestRunner({ getService }: FtrProviderContext) { }, wait: true, }); + for (const simulationClass of simulationClasses) { + await procs.run('gatling: test', { + cmd: 'mvn', + args: [ + 'gatling:test', + '-q', + `-Dgatling.simulationClass=${simulationPackage}.${simulationClass}`, + ], + cwd: gatlingProjectRootPath, + env: { + ...process.env, + }, + wait: true, + }); + } }); } diff --git a/x-pack/test/reporting_api_integration/reporting_without_security.config.ts b/x-pack/test/reporting_api_integration/reporting_without_security.config.ts index 4a95a15169b590..11182bbcdb3b09 100644 --- a/x-pack/test/reporting_api_integration/reporting_without_security.config.ts +++ b/x-pack/test/reporting_api_integration/reporting_without_security.config.ts @@ -32,6 +32,7 @@ export default async function ({ readConfigFile }: FtrConfigProviderContext) { kbnTestServer: { ...apiConfig.get('kbnTestServer'), serverArgs: [ + `--migrations.enableV2=false`, `--elasticsearch.hosts=${formatUrl(esTestConfig.getUrlParts())}`, `--logging.json=false`, `--server.maxPayloadBytes=1679958`, diff --git a/x-pack/test/tsconfig.json b/x-pack/test/tsconfig.json index ab3488bc3d4c2f..6039cd330c1493 100644 --- a/x-pack/test/tsconfig.json +++ b/x-pack/test/tsconfig.json @@ -21,6 +21,7 @@ { "path": "../../src/plugins/es_ui_shared/tsconfig.json" }, { "path": "../../src/plugins/expressions/tsconfig.json" }, { "path": "../../src/plugins/home/tsconfig.json" }, + { "path": "../../src/plugins/kibana_overview/tsconfig.json" }, { "path": "../../src/plugins/kibana_react/tsconfig.json" }, { "path": "../../src/plugins/kibana_usage_collection/tsconfig.json" }, { "path": "../../src/plugins/kibana_utils/tsconfig.json" }, @@ -40,6 +41,7 @@ { "path": "../plugins/actions/tsconfig.json" }, { "path": "../plugins/alerts/tsconfig.json" }, + { "path": "../plugins/code/tsconfig.json" }, { "path": "../plugins/console_extensions/tsconfig.json" }, { "path": "../plugins/data_enhanced/tsconfig.json" }, { "path": "../plugins/enterprise_search/tsconfig.json" }, @@ -65,6 +67,8 @@ { "path": "../plugins/observability/tsconfig.json" }, { "path": "../plugins/ingest_pipelines/tsconfig.json" }, { "path": "../plugins/license_management/tsconfig.json" }, + { "path": "../plugins/snapshot_restore/tsconfig.json" }, + { "path": "../plugins/grokdebugger/tsconfig.json" }, { "path": "../plugins/painless_lab/tsconfig.json" }, { "path": "../plugins/watcher/tsconfig.json" } ] diff --git a/x-pack/tsconfig.json b/x-pack/tsconfig.json index 37a5a7fc56816d..c73eb37a1138a9 100644 --- a/x-pack/tsconfig.json +++ b/x-pack/tsconfig.json @@ -9,6 +9,7 @@ "plugins/apm/scripts/**/*", "plugins/canvas/**/*", "plugins/console_extensions/**/*", + "plugins/code/**/*", "plugins/data_enhanced/**/*", "plugins/discover_enhanced/**/*", "plugins/dashboard_enhanced/**/*", @@ -43,8 +44,10 @@ "plugins/global_search_bar/**/*", "plugins/ingest_pipelines/**/*", "plugins/license_management/**/*", + "plugins/snapshot_restore/**/*", "plugins/painless_lab/**/*", "plugins/watcher/**/*", + "plugins/grokdebugger/**/*", "test/**/*" ], "compilerOptions": { @@ -64,8 +67,10 @@ { "path": "../src/plugins/es_ui_shared/tsconfig.json" }, { "path": "../src/plugins/expressions/tsconfig.json" }, { "path": "../src/plugins/home/tsconfig.json" }, + { "path": "../src/plugins/index_pattern_management/tsconfig.json" }, { "path": "../src/plugins/inspector/tsconfig.json" }, { "path": "../src/plugins/kibana_legacy/tsconfig.json" }, + { "path": "../src/plugins/kibana_overview/tsconfig.json" }, { "path": "../src/plugins/kibana_react/tsconfig.json" }, { "path": "../src/plugins/kibana_usage_collection/tsconfig.json" }, { "path": "../src/plugins/kibana_utils/tsconfig.json" }, @@ -85,12 +90,12 @@ { "path": "../src/plugins/ui_actions/tsconfig.json" }, { "path": "../src/plugins/url_forwarding/tsconfig.json" }, { "path": "../src/plugins/usage_collection/tsconfig.json" }, - { "path": "../src/plugins/index_pattern_management/tsconfig.json" }, - { "path": "./plugins/actions/tsconfig.json"}, - { "path": "./plugins/alerts/tsconfig.json"}, + { "path": "./plugins/actions/tsconfig.json" }, + { "path": "./plugins/alerts/tsconfig.json" }, { "path": "./plugins/beats_management/tsconfig.json" }, { "path": "./plugins/canvas/tsconfig.json" }, { "path": "./plugins/cloud/tsconfig.json" }, + { "path": "./plugins/code/tsconfig.json" }, { "path": "./plugins/console_extensions/tsconfig.json" }, { "path": "./plugins/data_enhanced/tsconfig.json" }, { "path": "./plugins/discover_enhanced/tsconfig.json" }, @@ -103,28 +108,26 @@ { "path": "./plugins/global_search_providers/tsconfig.json" }, { "path": "./plugins/global_search/tsconfig.json" }, { "path": "./plugins/graph/tsconfig.json" }, + { "path": "./plugins/grokdebugger/tsconfig.json" }, + { "path": "./plugins/ingest_pipelines/tsconfig.json" }, { "path": "./plugins/lens/tsconfig.json" }, { "path": "./plugins/license_management/tsconfig.json" }, { "path": "./plugins/licensing/tsconfig.json" }, { "path": "./plugins/maps_file_upload/tsconfig.json" }, { "path": "./plugins/maps_legacy_licensing/tsconfig.json" }, - { "path": "./plugins/observability/tsconfig.json" }, { "path": "./plugins/maps/tsconfig.json" }, + { "path": "./plugins/observability/tsconfig.json" }, { "path": "./plugins/painless_lab/tsconfig.json" }, { "path": "./plugins/saved_objects_tagging/tsconfig.json" }, { "path": "./plugins/searchprofiler/tsconfig.json" }, { "path": "./plugins/security/tsconfig.json" }, + { "path": "./plugins/snapshot_restore/tsconfig.json" }, { "path": "./plugins/spaces/tsconfig.json" }, { "path": "./plugins/stack_alerts/tsconfig.json" }, { "path": "./plugins/task_manager/tsconfig.json" }, { "path": "./plugins/telemetry_collection_xpack/tsconfig.json" }, { "path": "./plugins/translations/tsconfig.json" }, { "path": "./plugins/triggers_actions_ui/tsconfig.json" }, - { "path": "./plugins/stack_alerts/tsconfig.json" }, - { "path": "./plugins/ingest_pipelines/tsconfig.json" }, - { "path": "./plugins/license_management/tsconfig.json" }, - { "path": "./plugins/painless_lab/tsconfig.json" }, - { "path": "./plugins/triggers_actions_ui/tsconfig.json" }, { "path": "./plugins/ui_actions_enhanced/tsconfig.json" }, { "path": "./plugins/watcher/tsconfig.json" } ] diff --git a/x-pack/tsconfig.refs.json b/x-pack/tsconfig.refs.json index 00effc686183ac..ac82eb6407b1ac 100644 --- a/x-pack/tsconfig.refs.json +++ b/x-pack/tsconfig.refs.json @@ -6,6 +6,7 @@ { "path": "./plugins/beats_management/tsconfig.json" }, { "path": "./plugins/canvas/tsconfig.json" }, { "path": "./plugins/cloud/tsconfig.json" }, + { "path": "./plugins/code/tsconfig.json" }, { "path": "./plugins/console_extensions/tsconfig.json" }, { "path": "./plugins/dashboard_enhanced/tsconfig.json" }, { "path": "./plugins/data_enhanced/tsconfig.json" }, @@ -19,6 +20,7 @@ { "path": "./plugins/global_search_providers/tsconfig.json" }, { "path": "./plugins/global_search/tsconfig.json" }, { "path": "./plugins/graph/tsconfig.json" }, + { "path": "./plugins/grokdebugger/tsconfig.json" }, { "path": "./plugins/ingest_pipelines/tsconfig.json" }, { "path": "./plugins/lens/tsconfig.json" }, { "path": "./plugins/license_management/tsconfig.json" }, @@ -26,12 +28,13 @@ { "path": "./plugins/maps_file_upload/tsconfig.json" }, { "path": "./plugins/maps_legacy_licensing/tsconfig.json" }, { "path": "./plugins/maps/tsconfig.json" }, - { "path": "./plugins/painless_lab/tsconfig.json" }, { "path": "./plugins/observability/tsconfig.json" }, + { "path": "./plugins/painless_lab/tsconfig.json" }, { "path": "./plugins/reporting/tsconfig.json" }, { "path": "./plugins/saved_objects_tagging/tsconfig.json" }, { "path": "./plugins/searchprofiler/tsconfig.json" }, { "path": "./plugins/security/tsconfig.json" }, + { "path": "./plugins/snapshot_restore/tsconfig.json" }, { "path": "./plugins/spaces/tsconfig.json" }, { "path": "./plugins/stack_alerts/tsconfig.json" }, { "path": "./plugins/task_manager/tsconfig.json" },