Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
# Backport This will backport the following commits from `main` to `8.8`: - [[Security Solution] Elastic Security Assistant (#156933)](#156933) <!--- Backport version: 8.9.7 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Garrett Spong","email":"spong@users.noreply.github.com"},"sourceCommit":{"committedDate":"2023-06-02T21:19:10Z","message":"[Security Solution] Elastic Security Assistant (#156933)\n\n## [Security Solution] Elastic Security Assistant\r\n\r\nThe _Elastic Security Assistant_ has entered the chat, integrating generative AI and large language models (LLMs) into the workflows of Elastic Security users.\r\n\r\nBring your alerts, events, rules, and data quality checks into the conversation.\r\n\r\n<https://github.com/elastic/kibana/assets/2946766/31d65c78-5692-4817-b726-820c5df0801c>\r\n\r\nThis PR merges a feature branch developed by @spong and @andrew-goldstein , seeded by @jamesspi 's prototype of the assistant. Connectivity to LLMs is provided the [Generative AI Connector](<#157228>) , developed by @stephmilovic . This PR includes:\r\n\r\n- A new reusable Kibana package containing the assistant: `x-pack/packages/kbn-elastic-assistant`\r\n - See the `How to embed the Assistant in other parts of Kibana` for details\r\n- Assistant integration into Elastic Security Solution workflows (e.g. alerts, cases, Timeline, rules, data quality)\r\n\r\n### An assistant trained on the Elastic stack and Elastic Security\r\n\r\nThe [Generative AI Connector](<#157228>) connects the assistant to OpenAI and Azure OpenAI models trained with knowledge of the Elastic stack and the Elastic Security solution, including:\r\n\r\n- The Elastic open [Detection Rules](https://github.com/elastic/detection-rules)\r\n- The [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/current/index.html)\r\n- Elastic query languages, including [KQL](https://www.elastic.co/guide/en/kibana/current/kuery-query.html), [EQL](https://www.elastic.co/guide/en/elasticsearch/reference/current/eql-syntax.html), and the [Elastic Query DSL](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html)\r\n- [Elasticsearch API documentation](https://www.elastic.co/guide/en/elasticsearch/reference/8.8/rest-apis.html)\r\n\r\nThis training enables the assistant to offer fully interactive chat experiences that include:\r\n\r\n- alert summarization\r\n- interactive query generation\r\n- workflow suggestions\r\n- generating ingestion configurations that conform to the Elastic Common Schema\r\n- your imagination\r\n\r\nusing context from Elastic Security.\r\n\r\n### Take action from your conversations\r\n\r\nThe Actions (from assistant response):\r\n\r\n- Send KQL to Timeline\r\n- Send EQL to Timeline\r\n- Send Elasticsearch DSL to Timeline\r\n- Send Note to timeline\r\n- Create new case\r\n- Add to existing case\r\n- Copy to clipboard\r\n\r\n### Components architecture diagram\r\n\r\n![Untitled Diagram drawio (1)](https://github.com/elastic/kibana/assets/55110838/8f446313-629f-4646-b44d-0cb0ca74aeaf)\r\n\r\n### How to embed the Assistant in other parts of Kibana\r\n\r\nFollow the general instructions in `x-pack/packages/kbn-elastic-assistant/index.ts` to integrate the assistant into a Kibana app.\r\n\r\n#### Step 1 - Wrap your Kibana app in the `AssistantProvider` component\r\n\r\n```ts\r\n// Step 1: Wrap your Kibana app in the `AssistantProvider` component. This typically\r\n// happens in the root of your app. Optionally provide a custom title for the assistant:\r\n\r\n/** provides context (from the app) to the assistant, and injects Kibana services, like `http` */\r\nexport { AssistantProvider } from './impl/assistant_context';\r\n```\r\n\r\n#### Step 2: Add the `AssistantOverlay` component to your app\r\n\r\n```ts\r\n// Step 2: Add the `AssistantOverlay` component to your app. This component displays the assistant\r\n// overlay in a modal, bound to a shortcut key:\r\n\r\n/** modal overlay for Elastic Assistant conversations */\r\nexport { AssistantOverlay } from './impl/assistant/assistant_overlay';\r\n\r\n// In addition to the `AssistantOverlay`, or as an alternative, you may use the `Assistant` component\r\n// to display the assistant without the modal overlay:\r\n\r\n/** this component renders the Assistant without the modal overlay to, for example, render it in a Timeline tab */\r\nexport { Assistant } from './impl/assistant';\r\n```\r\n\r\n#### Step 3: Wherever you want to bring context into the assistant, use the any combination of the following\r\n\r\n```ts\r\n// Step 3: Wherever you want to bring context into the assistant, use the any combination of the following\r\n// components and hooks:\r\n// - `NewChat` component\r\n// - `NewChatById` component\r\n// - `useAssistantOverlay` hook\r\n\r\n/**\r\n * `NewChat` displays a _New chat_ icon button, providing all the context\r\n * necessary to start a new chat. You may optionally style the button icon,\r\n * or override the default _New chat_ text with custom content, like `🪄✨`\r\n *\r\n * USE THIS WHEN: All the data necessary to start a new chat is available\r\n * in the same part of the React tree as the _New chat_ button.\r\n */\r\nexport { NewChat } from './impl/new_chat';\r\n\r\n/**\r\n * `NewChatByID` displays a _New chat_ icon button by providing only the `promptContextId`\r\n * of a context that was (already) registered by the `useAssistantOverlay` hook. You may\r\n * optionally style the button icon, or override the default _New chat_ text with custom\r\n * content, like {'🪄✨'}\r\n *\r\n * USE THIS WHEN: all the data necessary to start a new chat is NOT available\r\n * in the same part of the React tree as the _New chat_ button. When paired\r\n * with the `useAssistantOverlay` hook, this option enables context to be be\r\n * registered where the data is available, and then the _New chat_ button can be displayed\r\n * in another part of the tree.\r\n */\r\nexport { NewChatById } from './impl/new_chat_by_id';\r\n\r\n/**\r\n * `useAssistantOverlay` is a hook that registers context with the assistant overlay, and\r\n * returns an optional `showAssistantOverlay` function to display the assistant overlay.\r\n * As an alterative to using the `showAssistantOverlay` returned from this hook, you may\r\n * use the `NewChatById` component and pass it the `promptContextId` returned by this hook.\r\n *\r\n * USE THIS WHEN: You want to register context in one part of the tree, and then show\r\n * a _New chat_ button in another part of the tree without passing around the data, or when\r\n * you want to build a custom `New chat` button with features not not provided by the\r\n * `NewChat` component.\r\n */\r\nexport { useAssistantOverlay } from './impl/assistant/use_assistant_overlay';\r\n```\r\n\r\nCo-authored-by: Garrett Spong <garrett.spong@elastic.co>\r\nCo-authored-by: Andrew Macri <andrew.macri@elastic.co>","sha":"4e38817a4ddfea8aff336c22788bc3a30ece5908","branchLabelMapping":{"^v8.9.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team: SecuritySolution","release_note:feature","ci:no-auto-commit","v8.9.0","v8.8.1"],"number":156933,"url":"https://github.com/elastic/kibana/pull/156933","mergeCommit":{"message":"[Security Solution] Elastic Security Assistant (#156933)\n\n## [Security Solution] Elastic Security Assistant\r\n\r\nThe _Elastic Security Assistant_ has entered the chat, integrating generative AI and large language models (LLMs) into the workflows of Elastic Security users.\r\n\r\nBring your alerts, events, rules, and data quality checks into the conversation.\r\n\r\n<https://github.com/elastic/kibana/assets/2946766/31d65c78-5692-4817-b726-820c5df0801c>\r\n\r\nThis PR merges a feature branch developed by @spong and @andrew-goldstein , seeded by @jamesspi 's prototype of the assistant. Connectivity to LLMs is provided the [Generative AI Connector](<#157228>) , developed by @stephmilovic . This PR includes:\r\n\r\n- A new reusable Kibana package containing the assistant: `x-pack/packages/kbn-elastic-assistant`\r\n - See the `How to embed the Assistant in other parts of Kibana` for details\r\n- Assistant integration into Elastic Security Solution workflows (e.g. alerts, cases, Timeline, rules, data quality)\r\n\r\n### An assistant trained on the Elastic stack and Elastic Security\r\n\r\nThe [Generative AI Connector](<#157228>) connects the assistant to OpenAI and Azure OpenAI models trained with knowledge of the Elastic stack and the Elastic Security solution, including:\r\n\r\n- The Elastic open [Detection Rules](https://github.com/elastic/detection-rules)\r\n- The [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/current/index.html)\r\n- Elastic query languages, including [KQL](https://www.elastic.co/guide/en/kibana/current/kuery-query.html), [EQL](https://www.elastic.co/guide/en/elasticsearch/reference/current/eql-syntax.html), and the [Elastic Query DSL](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html)\r\n- [Elasticsearch API documentation](https://www.elastic.co/guide/en/elasticsearch/reference/8.8/rest-apis.html)\r\n\r\nThis training enables the assistant to offer fully interactive chat experiences that include:\r\n\r\n- alert summarization\r\n- interactive query generation\r\n- workflow suggestions\r\n- generating ingestion configurations that conform to the Elastic Common Schema\r\n- your imagination\r\n\r\nusing context from Elastic Security.\r\n\r\n### Take action from your conversations\r\n\r\nThe Actions (from assistant response):\r\n\r\n- Send KQL to Timeline\r\n- Send EQL to Timeline\r\n- Send Elasticsearch DSL to Timeline\r\n- Send Note to timeline\r\n- Create new case\r\n- Add to existing case\r\n- Copy to clipboard\r\n\r\n### Components architecture diagram\r\n\r\n![Untitled Diagram drawio (1)](https://github.com/elastic/kibana/assets/55110838/8f446313-629f-4646-b44d-0cb0ca74aeaf)\r\n\r\n### How to embed the Assistant in other parts of Kibana\r\n\r\nFollow the general instructions in `x-pack/packages/kbn-elastic-assistant/index.ts` to integrate the assistant into a Kibana app.\r\n\r\n#### Step 1 - Wrap your Kibana app in the `AssistantProvider` component\r\n\r\n```ts\r\n// Step 1: Wrap your Kibana app in the `AssistantProvider` component. This typically\r\n// happens in the root of your app. Optionally provide a custom title for the assistant:\r\n\r\n/** provides context (from the app) to the assistant, and injects Kibana services, like `http` */\r\nexport { AssistantProvider } from './impl/assistant_context';\r\n```\r\n\r\n#### Step 2: Add the `AssistantOverlay` component to your app\r\n\r\n```ts\r\n// Step 2: Add the `AssistantOverlay` component to your app. This component displays the assistant\r\n// overlay in a modal, bound to a shortcut key:\r\n\r\n/** modal overlay for Elastic Assistant conversations */\r\nexport { AssistantOverlay } from './impl/assistant/assistant_overlay';\r\n\r\n// In addition to the `AssistantOverlay`, or as an alternative, you may use the `Assistant` component\r\n// to display the assistant without the modal overlay:\r\n\r\n/** this component renders the Assistant without the modal overlay to, for example, render it in a Timeline tab */\r\nexport { Assistant } from './impl/assistant';\r\n```\r\n\r\n#### Step 3: Wherever you want to bring context into the assistant, use the any combination of the following\r\n\r\n```ts\r\n// Step 3: Wherever you want to bring context into the assistant, use the any combination of the following\r\n// components and hooks:\r\n// - `NewChat` component\r\n// - `NewChatById` component\r\n// - `useAssistantOverlay` hook\r\n\r\n/**\r\n * `NewChat` displays a _New chat_ icon button, providing all the context\r\n * necessary to start a new chat. You may optionally style the button icon,\r\n * or override the default _New chat_ text with custom content, like `🪄✨`\r\n *\r\n * USE THIS WHEN: All the data necessary to start a new chat is available\r\n * in the same part of the React tree as the _New chat_ button.\r\n */\r\nexport { NewChat } from './impl/new_chat';\r\n\r\n/**\r\n * `NewChatByID` displays a _New chat_ icon button by providing only the `promptContextId`\r\n * of a context that was (already) registered by the `useAssistantOverlay` hook. You may\r\n * optionally style the button icon, or override the default _New chat_ text with custom\r\n * content, like {'🪄✨'}\r\n *\r\n * USE THIS WHEN: all the data necessary to start a new chat is NOT available\r\n * in the same part of the React tree as the _New chat_ button. When paired\r\n * with the `useAssistantOverlay` hook, this option enables context to be be\r\n * registered where the data is available, and then the _New chat_ button can be displayed\r\n * in another part of the tree.\r\n */\r\nexport { NewChatById } from './impl/new_chat_by_id';\r\n\r\n/**\r\n * `useAssistantOverlay` is a hook that registers context with the assistant overlay, and\r\n * returns an optional `showAssistantOverlay` function to display the assistant overlay.\r\n * As an alterative to using the `showAssistantOverlay` returned from this hook, you may\r\n * use the `NewChatById` component and pass it the `promptContextId` returned by this hook.\r\n *\r\n * USE THIS WHEN: You want to register context in one part of the tree, and then show\r\n * a _New chat_ button in another part of the tree without passing around the data, or when\r\n * you want to build a custom `New chat` button with features not not provided by the\r\n * `NewChat` component.\r\n */\r\nexport { useAssistantOverlay } from './impl/assistant/use_assistant_overlay';\r\n```\r\n\r\nCo-authored-by: Garrett Spong <garrett.spong@elastic.co>\r\nCo-authored-by: Andrew Macri <andrew.macri@elastic.co>","sha":"4e38817a4ddfea8aff336c22788bc3a30ece5908"}},"sourceBranch":"main","suggestedTargetBranches":["8.8"],"targetPullRequestStates":[{"branch":"main","label":"v8.9.0","labelRegex":"^v8.9.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/156933","number":156933,"mergeCommit":{"message":"[Security Solution] Elastic Security Assistant (#156933)\n\n## [Security Solution] Elastic Security Assistant\r\n\r\nThe _Elastic Security Assistant_ has entered the chat, integrating generative AI and large language models (LLMs) into the workflows of Elastic Security users.\r\n\r\nBring your alerts, events, rules, and data quality checks into the conversation.\r\n\r\n<https://github.com/elastic/kibana/assets/2946766/31d65c78-5692-4817-b726-820c5df0801c>\r\n\r\nThis PR merges a feature branch developed by @spong and @andrew-goldstein , seeded by @jamesspi 's prototype of the assistant. Connectivity to LLMs is provided the [Generative AI Connector](<#157228>) , developed by @stephmilovic . This PR includes:\r\n\r\n- A new reusable Kibana package containing the assistant: `x-pack/packages/kbn-elastic-assistant`\r\n - See the `How to embed the Assistant in other parts of Kibana` for details\r\n- Assistant integration into Elastic Security Solution workflows (e.g. alerts, cases, Timeline, rules, data quality)\r\n\r\n### An assistant trained on the Elastic stack and Elastic Security\r\n\r\nThe [Generative AI Connector](<#157228>) connects the assistant to OpenAI and Azure OpenAI models trained with knowledge of the Elastic stack and the Elastic Security solution, including:\r\n\r\n- The Elastic open [Detection Rules](https://github.com/elastic/detection-rules)\r\n- The [Elastic Common Schema (ECS)](https://www.elastic.co/guide/en/ecs/current/index.html)\r\n- Elastic query languages, including [KQL](https://www.elastic.co/guide/en/kibana/current/kuery-query.html), [EQL](https://www.elastic.co/guide/en/elasticsearch/reference/current/eql-syntax.html), and the [Elastic Query DSL](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html)\r\n- [Elasticsearch API documentation](https://www.elastic.co/guide/en/elasticsearch/reference/8.8/rest-apis.html)\r\n\r\nThis training enables the assistant to offer fully interactive chat experiences that include:\r\n\r\n- alert summarization\r\n- interactive query generation\r\n- workflow suggestions\r\n- generating ingestion configurations that conform to the Elastic Common Schema\r\n- your imagination\r\n\r\nusing context from Elastic Security.\r\n\r\n### Take action from your conversations\r\n\r\nThe Actions (from assistant response):\r\n\r\n- Send KQL to Timeline\r\n- Send EQL to Timeline\r\n- Send Elasticsearch DSL to Timeline\r\n- Send Note to timeline\r\n- Create new case\r\n- Add to existing case\r\n- Copy to clipboard\r\n\r\n### Components architecture diagram\r\n\r\n![Untitled Diagram drawio (1)](https://github.com/elastic/kibana/assets/55110838/8f446313-629f-4646-b44d-0cb0ca74aeaf)\r\n\r\n### How to embed the Assistant in other parts of Kibana\r\n\r\nFollow the general instructions in `x-pack/packages/kbn-elastic-assistant/index.ts` to integrate the assistant into a Kibana app.\r\n\r\n#### Step 1 - Wrap your Kibana app in the `AssistantProvider` component\r\n\r\n```ts\r\n// Step 1: Wrap your Kibana app in the `AssistantProvider` component. This typically\r\n// happens in the root of your app. Optionally provide a custom title for the assistant:\r\n\r\n/** provides context (from the app) to the assistant, and injects Kibana services, like `http` */\r\nexport { AssistantProvider } from './impl/assistant_context';\r\n```\r\n\r\n#### Step 2: Add the `AssistantOverlay` component to your app\r\n\r\n```ts\r\n// Step 2: Add the `AssistantOverlay` component to your app. This component displays the assistant\r\n// overlay in a modal, bound to a shortcut key:\r\n\r\n/** modal overlay for Elastic Assistant conversations */\r\nexport { AssistantOverlay } from './impl/assistant/assistant_overlay';\r\n\r\n// In addition to the `AssistantOverlay`, or as an alternative, you may use the `Assistant` component\r\n// to display the assistant without the modal overlay:\r\n\r\n/** this component renders the Assistant without the modal overlay to, for example, render it in a Timeline tab */\r\nexport { Assistant } from './impl/assistant';\r\n```\r\n\r\n#### Step 3: Wherever you want to bring context into the assistant, use the any combination of the following\r\n\r\n```ts\r\n// Step 3: Wherever you want to bring context into the assistant, use the any combination of the following\r\n// components and hooks:\r\n// - `NewChat` component\r\n// - `NewChatById` component\r\n// - `useAssistantOverlay` hook\r\n\r\n/**\r\n * `NewChat` displays a _New chat_ icon button, providing all the context\r\n * necessary to start a new chat. You may optionally style the button icon,\r\n * or override the default _New chat_ text with custom content, like `🪄✨`\r\n *\r\n * USE THIS WHEN: All the data necessary to start a new chat is available\r\n * in the same part of the React tree as the _New chat_ button.\r\n */\r\nexport { NewChat } from './impl/new_chat';\r\n\r\n/**\r\n * `NewChatByID` displays a _New chat_ icon button by providing only the `promptContextId`\r\n * of a context that was (already) registered by the `useAssistantOverlay` hook. You may\r\n * optionally style the button icon, or override the default _New chat_ text with custom\r\n * content, like {'🪄✨'}\r\n *\r\n * USE THIS WHEN: all the data necessary to start a new chat is NOT available\r\n * in the same part of the React tree as the _New chat_ button. When paired\r\n * with the `useAssistantOverlay` hook, this option enables context to be be\r\n * registered where the data is available, and then the _New chat_ button can be displayed\r\n * in another part of the tree.\r\n */\r\nexport { NewChatById } from './impl/new_chat_by_id';\r\n\r\n/**\r\n * `useAssistantOverlay` is a hook that registers context with the assistant overlay, and\r\n * returns an optional `showAssistantOverlay` function to display the assistant overlay.\r\n * As an alterative to using the `showAssistantOverlay` returned from this hook, you may\r\n * use the `NewChatById` component and pass it the `promptContextId` returned by this hook.\r\n *\r\n * USE THIS WHEN: You want to register context in one part of the tree, and then show\r\n * a _New chat_ button in another part of the tree without passing around the data, or when\r\n * you want to build a custom `New chat` button with features not not provided by the\r\n * `NewChat` component.\r\n */\r\nexport { useAssistantOverlay } from './impl/assistant/use_assistant_overlay';\r\n```\r\n\r\nCo-authored-by: Garrett Spong <garrett.spong@elastic.co>\r\nCo-authored-by: Andrew Macri <andrew.macri@elastic.co>","sha":"4e38817a4ddfea8aff336c22788bc3a30ece5908"}},{"branch":"8.8","label":"v8.8.1","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
- Loading branch information