-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SecuritySolution][Detections] Adds UI for new Rule Fields: Related Integrations, Required Fields, and Setup #131475
Conversation
9736848
to
4e7ecaf
Compare
0b7e8f9
to
9c62a92
Compare
…ated Integrations, Required Fields, and Setup (#132409) **Addresses partially:** elastic/security-team#2083, elastic/security-team#558, elastic/security-team#2856, elastic/security-team#1801 (internal tickets) ## Summary **TL;DR:** With this PR, it's now possible to specify `related_integrations`, `required_fields`, and `setup` fields in prebuilt rules in https://github.com/elastic/detection-rules. They are returned within rules in the API responses. This PR: - Adds 3 new fields to the model of Security detection rules. These fields are common to all of the rule types we have. - **Related Integrations**. It's a list of Fleet integrations associated with a given rule. It's assumed that if the user installs them, the rule might start to work properly because it will start receiving source events potentially matching the rule's query. - **Required Fields**. It's a list of event fields that must be present in the source indices of a given rule. - **Setup Guide**. It's any instructions for the user for setting up their environment in order to start receiving source events for a given rule. It's a text. Markdown is supported. It's similar to the Investigation Guide that we show on the Details page. - Adjusts API endpoints accordingly: - These fields are for prebuilt rules only and are supposed to be read-only in the UI. - Specifying these fields in the request parameters of the create/update/patch rule API endpoints is not supported. - These fields are returned in all responses that contain rules. If they are missing in a rule, default values are returned (empty array, empty string). - When duplicating a prebuilt rule, these fields are being reset to their default value (empty array, empty string). - Export/Import is supported. Edge case / supported hack: it's possible to specify these fields manually in a ndjson doc and import with a rule. - The fields are being copied to `kibana.alert.rule.parameters` field of an alert document, which is mapped as a flattened field type. No special handling for the new fields was needed there. - Adjusts tests accordingly. ## Related Integrations Example (part of a rule returned from the API): ```json { "related_integrations": [ { "package": "windows", "version": "1.5.x" }, { "package": "azure", "integration": "activitylogs", "version": "~1.1.6" } ], } ``` Schema: ```ts /** * Related integration is a potential dependency of a rule. It's assumed that if the user installs * one of the related integrations of a rule, the rule might start to work properly because it will * have source events (generated by this integration) potentially matching the rule's query. * * NOTE: Proper work is not guaranteed, because a related integration, if installed, can be * configured differently or generate data that is not necessarily relevant for this rule. * * Related integration is a combination of a Fleet package and (optionally) one of the * package's "integrations" that this package contains. It is represented by 3 properties: * * - `package`: name of the package (required, unique id) * - `version`: version of the package (required, semver-compatible) * - `integration`: name of the integration of this package (optional, id within the package) * * There are Fleet packages like `windows` that contain only one integration; in this case, * `integration` should be unspecified. There are also packages like `aws` and `azure` that contain * several integrations; in this case, `integration` should be specified. * * @example * const x: RelatedIntegration = { * package: 'windows', * version: '1.5.x', * }; * * @example * const x: RelatedIntegration = { * package: 'azure', * version: '~1.1.6', * integration: 'activitylogs', * }; */ export type RelatedIntegration = t.TypeOf<typeof RelatedIntegration>; export const RelatedIntegration = t.exact( t.intersection([ t.type({ package: NonEmptyString, version: NonEmptyString, }), t.partial({ integration: NonEmptyString, }), ]) ); ``` ## Required Fields Example (part of a rule returned from the API): ```json { "required_fields": [ { "name": "event.action", "type": "keyword", "ecs": true }, { "name": "event.code", "type": "keyword", "ecs": true }, { "name": "winlog.event_data.AttributeLDAPDisplayName", "type": "keyword", "ecs": false } ], } ``` Schema: ```ts /** * Almost all types of Security rules check source event documents for a match to some kind of * query or filter. If a document has certain field with certain values, then it's a match and * the rule will generate an alert. * * Required field is an event field that must be present in the source indices of a given rule. * * @example * const standardEcsField: RequiredField = { * name: 'event.action', * type: 'keyword', * ecs: true, * }; * * @example * const nonEcsField: RequiredField = { * name: 'winlog.event_data.AttributeLDAPDisplayName', * type: 'keyword', * ecs: false, * }; */ export type RequiredField = t.TypeOf<typeof RequiredField>; export const RequiredField = t.exact( t.type({ name: NonEmptyString, type: NonEmptyString, ecs: t.boolean, }) ); ``` ## Setup Guide Example (part of a rule returned from the API): ```json { "setup": "## Config\n\nThe 'PowerShell Script Block Logging' logging policy must be enabled.\nSteps to implement the logging policy with with Advanced Audit Configuration:\n\n```\nComputer Configuration > \nAdministrative Templates > \nWindows PowerShell > \nTurn on PowerShell Script Block Logging (Enable)\n```\n\nSteps to implement the logging policy via registry:\n\n```\nreg add \"hklm\\SOFTWARE\\Policies\\Microsoft\\Windows\\PowerShell\\ScriptBlockLogging\" /v EnableScriptBlockLogging /t REG_DWORD /d 1\n```\n", } ``` Schema: ```ts /** * Any instructions for the user for setting up their environment in order to start receiving * source events for a given rule. * * It's a multiline text. Markdown is supported. */ export type SetupGuide = t.TypeOf<typeof SetupGuide>; export const SetupGuide = t.string; ``` ## Details on the schema This PR adjusts all the 6 rule schemas we have: 1. Alerting Framework rule `params` schema: - `security_solution/server/lib/detection_engine/schemas/rule_schemas.ts` - `security_solution/server/lib/detection_engine/schemas/rule_converters.ts` 2. HTTP API main old schema: - `security_solution/common/detection_engine/schemas/response/rules_schema.ts` 3. HTTP API main new schema: - `security_solution/common/detection_engine/schemas/request/rule_schemas.ts` 4. Prebuilt rule schema: - `security_solution/common/detection_engine/schemas/request/add_prepackaged_rules_schema.ts` 5. Import rule schema: - `security_solution/common/detection_engine/schemas/request/import_rules_schema.ts` 6. Rule schema used on the frontend side: - `security_solution/public/detections/containers/detection_engine/rules/types.ts` Names of the fields on the HTTP API level: - `related_integrations` - `required_fields` - `setup` Names of the fields on the Alerting Framework level: - `params.relatedIntegrations` - `params.requiredFields` - `params.setup` ## Next steps - Create a new endpoint for returning installed Fleet integrations (gonna be a separate PR). - Rebase #131475 on top of this PR after merge. - Cover the new fields with dedicated tests (gonna be a separate PR). - Update API docs (gonna be a separate PR). - Address the tech debt of having 6 different schemas (gonna create a ticket for that). ### Checklist - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
9c62a92
to
0b48577
Compare
@@ -237,6 +237,8 @@ export const DETECTION_ENGINE_PREPACKAGED_URL = | |||
`${DETECTION_ENGINE_RULES_URL}/prepackaged` as const; | |||
export const DETECTION_ENGINE_PRIVILEGES_URL = `${DETECTION_ENGINE_URL}/privileges` as const; | |||
export const DETECTION_ENGINE_INDEX_URL = `${DETECTION_ENGINE_URL}/index` as const; | |||
export const DETECTION_ENGINE_INSTALLED_INTEGRATIONS_URL = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@banderror -- this is as far as I went with plumbing for the installed_integrations
API. Created this route, and also wired it up to to the useInstalledIntegrations
hook, so should just start working if you return a RelatedIntegrationArray
from this route 🙂
…graiton API, and removes mocked installed data
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
Pinging @elastic/security-solution (Team: SecuritySolution) |
Files by Code Ownerelastic/kibana-app-services
elastic/security-detections-response-rules
elastic/security-solution
|
<PopoverItems | ||
items={groups} | ||
numberOfItemsToDisplay={0} | ||
popoverButtonTitle={`${groups.length} Groups`} | ||
renderItem={renderItem} | ||
dataTestPrefix="groups" | ||
/> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't mind this... 😅
I was testing #131166 the other day and had it with the badge overflow. Since in doing this PR I learned of the PopoverItems
component I figured there was a nice quick fix here:
{numberOfItemsToDisplay !== 0 && ( | ||
<EuiFlexItem grow={1} className="eui-textTruncate"> | ||
<OverflowList items={items.slice(0, numberOfItemsToDisplay)} /> | ||
</EuiFlexItem> | ||
)} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Spacing fix from leftover element when displaying empty items. Current implementation/use on Rules Table for Tags didn't exercise this path.
💛 Build succeeded, but was flakyFailed CI StepsTest Failures
Metrics [docs]Module Count
Async chunks
Page load bundle
History
To update your PR or re-run it, just comment with: cc @spong |
commit bdb4966 Author: Angela Chuang <6295984+angorayc@users.noreply.github.com> Date: Mon May 23 13:13:23 2022 +0100 styling (elastic#132539) Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> commit a807c90 Author: Esteban Beltran <academo@users.noreply.github.com> Date: Mon May 23 13:36:00 2022 +0200 [Cases] Add a key to userActionMarkdown to prevent stale state (elastic#132681) commit ba84602 Author: Tomasz Ciecierski <ciecierskitomek@gmail.com> Date: Mon May 23 13:33:20 2022 +0200 [Osquery] Change prebuilt saved queries to include prebuilt flag (elastic#132651) commit 6b846af Author: Faisal Kanout <faisal.kanout@elastic.co> Date: Mon May 23 14:11:04 2022 +0300 [Actionable Observability] Update the Rule details design and clean up (elastic#132616) * Add rule status in the rule summary * Match design * Remove unused imports * code review commit c993ff2 Author: Byron Hulcher <byron.hulcher@elastic.co> Date: Mon May 23 06:25:17 2022 -0400 [Workplace Search] Add categories to source data for internal connectors (elastic#132671) commit b59fb97 Author: Pablo Machado <pablo.nevesmachado@elastic.co> Date: Mon May 23 12:02:43 2022 +0200 [Security Solution] Update use_url_state to work with new side nav (elastic#132518) * Fix landing pages browser tab title * Fix new navigation url state * Fix unit tests Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> commit 693b3e8 Author: Tomasz Ciecierski <ciecierskitomek@gmail.com> Date: Mon May 23 11:54:29 2022 +0200 [Osquery] Add Osquery to Alert context menu (elastic#131790) commit 2cddced Author: Jordan <51442161+JordanSh@users.noreply.github.com> Date: Mon May 23 12:50:55 2022 +0300 [Cloud Posture] Trendline query changes (elastic#132680) commit 7591fb6 Author: Giorgos Bamparopoulos <georgios.bamparopoulos@elastic.co> Date: Mon May 23 10:37:03 2022 +0100 Fix agent config indicator when applied through fleet integration (elastic#131820) * Fix agent config indicator when applied through fleet integration * Add synthrace scenario Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> commit 37d40d7 Author: Dominique Clarke <dominique.clarke@elastic.co> Date: Mon May 23 04:56:34 2022 -0400 [Synthetics] fix browser type as default in monitor management (elastic#132572) Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> commit ae8b6c8 Author: Lucas F. da Costa <lucas.costa@elastic.co> Date: Mon May 23 09:29:11 2022 +0100 [Uptime] Fix bug causing all monitors to be saved to all locations [solves elastic#132314] (elastic#132325) Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> commit e0944d1 Author: Nodir Latipov <nodir.latypov@gmail.com> Date: Mon May 23 13:27:24 2022 +0500 [Unified search] Use the DataViews service (elastic#130008) * feat: cleanup deprecated service and type * fix: rollback test * refact: replace deprecated type * refact: changed deprecation type * feat: added comments to deprecated imports that can't be cleaned up in this PR * refact: rollback query_string_input.test file commit a3646eb Author: Pablo Machado <machadoum@gmail.com> Date: Mon May 23 10:17:12 2022 +0200 [Security Solutions] Refactor breadcrumbs to support new menu structure (elastic#131624) * Refactor breadcrumbs to support new structure * Fix code style * Fix more code style * Fix unit test Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> commit 9649307 Author: István Zoltán Szabó <istvan.szabo@elastic.co> Date: Mon May 23 10:12:54 2022 +0200 [DOCS] Updates alerting authorization docs with info on retaining API keys (elastic#132402) Co-authored-by: Lisa Cawley <lcawley@elastic.co> commit 40df1f3 Author: Tomasz Ciecierski <ciecierskitomek@gmail.com> Date: Mon May 23 08:45:50 2022 +0200 [Osquery] Add labels, move osquery schema link (elastic#132584) commit fbaf058 Author: Jiawei Wu <74562234+JiaweiWu@users.noreply.github.com> Date: Sun May 22 17:14:23 2022 -0700 [RAM] Add shareable rules list (elastic#132437) * Shareable rules list * Hide snooze panel in rules list * Address comments and added tests * Fix tests * Fix tests * Fix lint * Address design comments and fix tests Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> commit 383239e Author: Kfir Peled <61654899+kfirpeled@users.noreply.github.com> Date: Sun May 22 13:18:42 2022 +0300 [Cloud Posture] Findings - Group by resource - Fixed bug not showing results (elastic#132529) commit fb1eeb0 Author: Georgii Gorbachev <georgii.gorbachev@elastic.co> Date: Sat May 21 00:21:53 2022 +0200 [Security Solution][Detections] Add new fields to the rule model: Related Integrations, Required Fields, and Setup (elastic#132409) **Addresses partially:** elastic/security-team#2083, elastic/security-team#558, elastic/security-team#2856, elastic/security-team#1801 (internal tickets) ## Summary **TL;DR:** With this PR, it's now possible to specify `related_integrations`, `required_fields`, and `setup` fields in prebuilt rules in https://github.com/elastic/detection-rules. They are returned within rules in the API responses. This PR: - Adds 3 new fields to the model of Security detection rules. These fields are common to all of the rule types we have. - **Related Integrations**. It's a list of Fleet integrations associated with a given rule. It's assumed that if the user installs them, the rule might start to work properly because it will start receiving source events potentially matching the rule's query. - **Required Fields**. It's a list of event fields that must be present in the source indices of a given rule. - **Setup Guide**. It's any instructions for the user for setting up their environment in order to start receiving source events for a given rule. It's a text. Markdown is supported. It's similar to the Investigation Guide that we show on the Details page. - Adjusts API endpoints accordingly: - These fields are for prebuilt rules only and are supposed to be read-only in the UI. - Specifying these fields in the request parameters of the create/update/patch rule API endpoints is not supported. - These fields are returned in all responses that contain rules. If they are missing in a rule, default values are returned (empty array, empty string). - When duplicating a prebuilt rule, these fields are being reset to their default value (empty array, empty string). - Export/Import is supported. Edge case / supported hack: it's possible to specify these fields manually in a ndjson doc and import with a rule. - The fields are being copied to `kibana.alert.rule.parameters` field of an alert document, which is mapped as a flattened field type. No special handling for the new fields was needed there. - Adjusts tests accordingly. ## Related Integrations Example (part of a rule returned from the API): ```json { "related_integrations": [ { "package": "windows", "version": "1.5.x" }, { "package": "azure", "integration": "activitylogs", "version": "~1.1.6" } ], } ``` Schema: ```ts /** * Related integration is a potential dependency of a rule. It's assumed that if the user installs * one of the related integrations of a rule, the rule might start to work properly because it will * have source events (generated by this integration) potentially matching the rule's query. * * NOTE: Proper work is not guaranteed, because a related integration, if installed, can be * configured differently or generate data that is not necessarily relevant for this rule. * * Related integration is a combination of a Fleet package and (optionally) one of the * package's "integrations" that this package contains. It is represented by 3 properties: * * - `package`: name of the package (required, unique id) * - `version`: version of the package (required, semver-compatible) * - `integration`: name of the integration of this package (optional, id within the package) * * There are Fleet packages like `windows` that contain only one integration; in this case, * `integration` should be unspecified. There are also packages like `aws` and `azure` that contain * several integrations; in this case, `integration` should be specified. * * @example * const x: RelatedIntegration = { * package: 'windows', * version: '1.5.x', * }; * * @example * const x: RelatedIntegration = { * package: 'azure', * version: '~1.1.6', * integration: 'activitylogs', * }; */ export type RelatedIntegration = t.TypeOf<typeof RelatedIntegration>; export const RelatedIntegration = t.exact( t.intersection([ t.type({ package: NonEmptyString, version: NonEmptyString, }), t.partial({ integration: NonEmptyString, }), ]) ); ``` ## Required Fields Example (part of a rule returned from the API): ```json { "required_fields": [ { "name": "event.action", "type": "keyword", "ecs": true }, { "name": "event.code", "type": "keyword", "ecs": true }, { "name": "winlog.event_data.AttributeLDAPDisplayName", "type": "keyword", "ecs": false } ], } ``` Schema: ```ts /** * Almost all types of Security rules check source event documents for a match to some kind of * query or filter. If a document has certain field with certain values, then it's a match and * the rule will generate an alert. * * Required field is an event field that must be present in the source indices of a given rule. * * @example * const standardEcsField: RequiredField = { * name: 'event.action', * type: 'keyword', * ecs: true, * }; * * @example * const nonEcsField: RequiredField = { * name: 'winlog.event_data.AttributeLDAPDisplayName', * type: 'keyword', * ecs: false, * }; */ export type RequiredField = t.TypeOf<typeof RequiredField>; export const RequiredField = t.exact( t.type({ name: NonEmptyString, type: NonEmptyString, ecs: t.boolean, }) ); ``` ## Setup Guide Example (part of a rule returned from the API): ```json { "setup": "## Config\n\nThe 'PowerShell Script Block Logging' logging policy must be enabled.\nSteps to implement the logging policy with with Advanced Audit Configuration:\n\n```\nComputer Configuration > \nAdministrative Templates > \nWindows PowerShell > \nTurn on PowerShell Script Block Logging (Enable)\n```\n\nSteps to implement the logging policy via registry:\n\n```\nreg add \"hklm\\SOFTWARE\\Policies\\Microsoft\\Windows\\PowerShell\\ScriptBlockLogging\" /v EnableScriptBlockLogging /t REG_DWORD /d 1\n```\n", } ``` Schema: ```ts /** * Any instructions for the user for setting up their environment in order to start receiving * source events for a given rule. * * It's a multiline text. Markdown is supported. */ export type SetupGuide = t.TypeOf<typeof SetupGuide>; export const SetupGuide = t.string; ``` ## Details on the schema This PR adjusts all the 6 rule schemas we have: 1. Alerting Framework rule `params` schema: - `security_solution/server/lib/detection_engine/schemas/rule_schemas.ts` - `security_solution/server/lib/detection_engine/schemas/rule_converters.ts` 2. HTTP API main old schema: - `security_solution/common/detection_engine/schemas/response/rules_schema.ts` 3. HTTP API main new schema: - `security_solution/common/detection_engine/schemas/request/rule_schemas.ts` 4. Prebuilt rule schema: - `security_solution/common/detection_engine/schemas/request/add_prepackaged_rules_schema.ts` 5. Import rule schema: - `security_solution/common/detection_engine/schemas/request/import_rules_schema.ts` 6. Rule schema used on the frontend side: - `security_solution/public/detections/containers/detection_engine/rules/types.ts` Names of the fields on the HTTP API level: - `related_integrations` - `required_fields` - `setup` Names of the fields on the Alerting Framework level: - `params.relatedIntegrations` - `params.requiredFields` - `params.setup` ## Next steps - Create a new endpoint for returning installed Fleet integrations (gonna be a separate PR). - Rebase elastic#131475 on top of this PR after merge. - Cover the new fields with dedicated tests (gonna be a separate PR). - Update API docs (gonna be a separate PR). - Address the tech debt of having 6 different schemas (gonna create a ticket for that). ### Checklist - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios commit 788dd2e Author: Andrew Goldstein <andrew-goldstein@users.noreply.github.com> Date: Fri May 20 16:02:05 2022 -0600 [Security Solution] Fixes sorting and tooltips on columns for non-ECS fields that are only one level deep (elastic#132570) ## [Security Solution] Fixes sorting and tooltips on columns for non-ECS fields that are only one level deep This PR fixes <elastic#132490>, an issue where Timeline columns for non-ECS fields that are only one level deep couldn't be sorted, and displayed incomplete metadata in the column's tooltip. ### Before ![test_field_1_actual_tooltip](https://user-images.githubusercontent.com/4459398/169208299-51d9296a-15e1-4eb0-bc31-a0df6a63f0c5.png) _Before: The column is **not** sortable, and the tooltip displays incomplete metadata_ ### After ![after](https://user-images.githubusercontent.com/4459398/169414767-7274a795-015f-4805-8c3f-b233ead994ea.png) _After: The column is sortable, and the tooltip displays the expected metadata_ ### Desk testing See the _Steps to reproduce_ section of <elastic#132490> for testing details. commit 51ae020 Author: Constance <constancecchen@users.noreply.github.com> Date: Fri May 20 14:30:36 2022 -0700 Upgrade EUI to v55.1.3 (elastic#132451) * Upgrade EUI to 55.1.3 backport * [Deprecation] Remove `watchedItemProps` from EuiContextMenu usage - should no longer be necessary * Update snapshots with new data-popover attr * Fix failing FTR test - Now that EuiContextMenu focus is restored correctly, there is a tooltip around the popover toggle that's blocking an above item that the test wants to click - swapping the order so that the tooltip does not block the clicked item should work * Fix 2nd maps FTR test with blocking tooltip Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> commit 642290b Author: Nathan Reese <reese.nathan@elastic.co> Date: Fri May 20 15:11:15 2022 -0600 [maps] convert ESPewPewSource to typescript (elastic#132656) * [maps] convert ESPewPewSource to typescript * move @ts-expect-error moved by fix commit eb6a061 Author: Brian Seeders <brian.seeders@elastic.co> Date: Fri May 20 16:57:49 2022 -0400 [docs] Add 'yarn dev-docs' for managing and starting dev docs (elastic#132647) commit e0ea600 Author: Hannah Mudge <Heenawter@users.noreply.github.com> Date: Fri May 20 14:55:31 2022 -0600 Add group 6 to FTR config (elastic#132655) commit 41635e2 Author: Karl Godard <karl.godard@elastic.co> Date: Fri May 20 13:35:30 2022 -0700 fixed search highlighting. was only showing highlighted text w/o context (elastic#132650) Co-authored-by: mitodrummer <karlgodard@elastic.co> commit 791ebfa Author: debadair <debadair@elastic.co> Date: Fri May 20 13:34:04 2022 -0700 [DOCS] Remove obsolete license expiration info (elastic#131474) * [DOCS] Remove obsolete license expiration info As of elastic/elasticsearch#79671, Elasticsearch does a more stringent license check rather than operating in a semi-degraded mode. Closes elastic#127845 Closes elastic#125702 * Update docs/management/managing-licenses.asciidoc Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> commit e55bf40 Author: Nathan Reese <reese.nathan@elastic.co> Date: Fri May 20 14:15:00 2022 -0600 [Maps] create MVT_VECTOR when using choropleth wizard (elastic#132648) commit 46cd729 Author: Jan Monschke <jan.monschke@elastic.co> Date: Fri May 20 22:02:00 2022 +0200 [SecuritySolution] Disable agent status filters and timeline interaction (elastic#132586) * fix: disable drag-ability and hover actions for agent statuses The agent fields cannot be queried with ECS and therefore should not provide Filter In/Out functionality nor should users be able to add their representative fields to timeline investigations. Therefore users should not be able to add them to a timeline query by dragging them. * chore: make code more readable commit e857b30 Author: Vadim Kibana <82822460+vadimkibana@users.noreply.github.com> Date: Fri May 20 20:36:59 2022 +0200 remove human-readable automatic slug generation (elastic#132593) * remove human-readable automatic slug generation * make change non-breaking * [CI] Auto-commit changed files from 'node scripts/eslint --no-cache --fix' * remove test Co-authored-by: streamich <streamich@gmail.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> commit 6fc2fff Author: Lisa Cawley <lcawley@elastic.co> Date: Fri May 20 10:48:15 2022 -0700 [ML] Minor edits in prebuilt job descriptions (elastic#132633) commit ecca231 Author: Felix Stürmer <weltenwort@users.noreply.github.com> Date: Fri May 20 19:37:03 2022 +0200 [Stack Monitoring] Convert setup routes to TypeScript (elastic#131265) commit 065ea3e Author: Byron Hulcher <byron.hulcher@elastic.co> Date: Fri May 20 13:12:49 2022 -0400 [Workplace Search] Remove Custom API Source Integration tile (elastic#132538) commit 583d2b7 Author: Byron Hulcher <byron.hulcher@elastic.co> Date: Fri May 20 13:12:32 2022 -0400 [Workplace Search] Add documentation links for v8.3.0 connectors (elastic#132547) commit c244883 Author: Nathan Reese <reese.nathan@elastic.co> Date: Fri May 20 10:35:00 2022 -0600 [maps] show marker size in legend (elastic#132549) * [Maps] size legend * clean-up * refine spacing * clean up * more cleanup * use euiTheme for colors * fix jest test * do not show marker sizes for icons * remove lodash Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> commit d70ae0f Author: Quynh Nguyen <43350163+qn895@users.noreply.github.com> Date: Fri May 20 11:34:35 2022 -0500 [ILM] Add warnings for managed system policies (elastic#132269) * Add warnings to system/managed policies * Fix translations, policies * Add jest tests * Add jest tests to assert new toggle behavior * Add jest tests for edit policy callout * Fix snapshot * [ML] Update jest tests with helper, rename helper for clarity * [ML] Add hook for local storage to remember toggle setting * [ML] Fix naming Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> commit f70b4af Author: Nicolas Chaulet <nicolas.chaulet@elastic.co> Date: Fri May 20 12:22:08 2022 -0400 [Fleet] Fix rolling upgrade CANCEL and UI fixes (elastic#132625) commit d9f141a Author: Kevin Logan <56395104+kevinlog@users.noreply.github.com> Date: Fri May 20 11:37:35 2022 -0400 [Security Solution] Telemetry for Event Filters counts on both user and global entries (elastic#132542) commit 1b4ac7d Author: Yaroslav Kuznietsov <kuznetsov.yaroslav.yk@gmail.com> Date: Fri May 20 17:54:13 2022 +0300 [XY] Reference lines overlay fix. (elastic#132607) commit 759f13f Author: Nicolas Chaulet <nicolas.chaulet@elastic.co> Date: Fri May 20 10:39:09 2022 -0400 [Fleet] Remove reference to non removable package feature (elastic#132458) commit 7e15097 Author: Lisa Cawley <lcawley@elastic.co> Date: Fri May 20 07:32:27 2022 -0700 [ML] Adds placeholder text for testing NLP models (elastic#132486) commit bc31053 Author: Dmitry Tomashevich <39378793+Dmitriynj@users.noreply.github.com> Date: Fri May 20 17:09:20 2022 +0300 [Discover][Alerting] Implement editing of dataView, query & filters (elastic#131688) * [Discover] introduce params editing using unified search * [Discover] fix unit tests * [Discover] fix functional tests * [Discover] fix unit tests * [Discover] return test subject name * [Discover] fix alert functional test * Update x-pack/plugins/stack_alerts/public/alert_types/es_query/expression/search_source_expression_form.tsx Co-authored-by: Julia Rechkunova <julia.rechkunova@gmail.com> * Update x-pack/plugins/stack_alerts/public/alert_types/es_query/expression/search_source_expression_form.tsx Co-authored-by: Matthias Wilhelm <ankertal@gmail.com> * [Discover] hide filter panel options * [Discover] improve functional test * [Discover] apply suggestions * [Discover] change data view selector * [Discover] fix tests * [Discover] apply suggestions, fix lang mode toggler * [Discover] mote interface to types file, clean up diff * [Discover] fix saved query issue * Update x-pack/plugins/stack_alerts/server/alert_types/es_query/alert_type.ts Co-authored-by: Matthias Wilhelm <ankertal@gmail.com> * [Discover] remove zIndex * [Discover] omit null searchType from esQuery completely, add isEsQueryAlert check for useSavedObjectReferences hook * [Discover] set searchType to esQuery when needed * [Discover] fix unit tests * Update x-pack/plugins/stack_alerts/server/alert_types/es_query/alert_type_params.ts Co-authored-by: Matthias Wilhelm <ankertal@gmail.com> * Update x-pack/plugins/stack_alerts/server/alert_types/es_query/alert_type.ts Co-authored-by: Matthias Wilhelm <ankertal@gmail.com> Co-authored-by: Julia Rechkunova <julia.rechkunova@gmail.com> Co-authored-by: Matthias Wilhelm <ankertal@gmail.com> commit d344088 Author: Nathan Reese <reese.nathan@elastic.co> Date: Fri May 20 08:06:25 2022 -0600 [maps] Use label features from ES vector tile search API to fix multiple labels (elastic#132080) * [maps] mvt labels * eslint * only request labels when needed * update vector tile integration tests for hasLabels parameter * [CI] Auto-commit changed files from 'node scripts/precommit_hook.js --ref HEAD~1..HEAD --fix' * fix tests * fix test * only add _mvt_label_position filter when vector tiles are from ES vector tile search API * review feedback * include hasLabels in source data * fix jest test Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> commit 1d8bc7e Author: Shivindera Singh <shivindera@gmail.com> Date: Fri May 20 15:53:00 2022 +0200 hasData service - hit search api in case of an error with resolve api (elastic#132618) commit 7c37eda Author: Tomasz Ciecierski <ciecierskitomek@gmail.com> Date: Fri May 20 15:42:28 2022 +0200 [Osquery] Fix pagination issue on Alert's Osquery Flyout (elastic#132611) commit 2e51140 Author: Katerina Patticha <aikaterini.patticha@elastic.co> Date: Fri May 20 15:34:29 2022 +0200 Show service group icon only when there are service groups (elastic#131138) * Show service group icon when there are service groups * Fix fix errors * Remove additional request and display icon only for the service groups * Revert "Remove additional request and display icon only for the service groups" This reverts commit 7ff2bc9. * Add dependencies Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> commit 24cbb32 Author: Yaroslav Kuznietsov <kuznetsov.yaroslav.yk@gmail.com> Date: Fri May 20 16:27:14 2022 +0300 [XY] `pointsRadius`, `showPoints` and `lineWidth`. (elastic#130391) * Added pointsRadius, showPoints and lineWidth. * Added tests. Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> commit 1c2eb9f Author: Sergi Massaneda <sergi.massaneda@elastic.co> Date: Fri May 20 13:59:56 2022 +0100 [Security Solution] New Side nav integrating links config (elastic#132210) * Update navigation landing pages to use appLinks config * align app links changes * link configs refactor to use updater$ * navigation panel categories * test and type fixes * [CI] Auto-commit changed files from 'node scripts/precommit_hook.js --ref HEAD~1..HEAD --fix' * types changes * shared style change moved to a separate PR * use old deep links * minor changes after ux meeting * add links filtering * remove duplicated categories * temporary increase of plugin size limit * swap management links order * improve performance closing nav panel * test updated * host isolation page filterd and some improvements * remove async from plugin start * move links register from start to mount * restore size limits * Fix use_show_timeline unit tests Co-authored-by: Pablo Neves Machado <pablo.nevesmachado@elastic.co> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> commit 92ac7f9 Author: Katrin Freihofner <katrin.freihofner@elastic.co> Date: Fri May 20 13:51:51 2022 +0200 adds small styling updates to header panels (elastic#132596)
…ated Integrations, Required Fields, and Setup (elastic#132409) **Addresses partially:** elastic/security-team#2083, elastic/security-team#558, elastic/security-team#2856, elastic/security-team#1801 (internal tickets) ## Summary **TL;DR:** With this PR, it's now possible to specify `related_integrations`, `required_fields`, and `setup` fields in prebuilt rules in https://github.com/elastic/detection-rules. They are returned within rules in the API responses. This PR: - Adds 3 new fields to the model of Security detection rules. These fields are common to all of the rule types we have. - **Related Integrations**. It's a list of Fleet integrations associated with a given rule. It's assumed that if the user installs them, the rule might start to work properly because it will start receiving source events potentially matching the rule's query. - **Required Fields**. It's a list of event fields that must be present in the source indices of a given rule. - **Setup Guide**. It's any instructions for the user for setting up their environment in order to start receiving source events for a given rule. It's a text. Markdown is supported. It's similar to the Investigation Guide that we show on the Details page. - Adjusts API endpoints accordingly: - These fields are for prebuilt rules only and are supposed to be read-only in the UI. - Specifying these fields in the request parameters of the create/update/patch rule API endpoints is not supported. - These fields are returned in all responses that contain rules. If they are missing in a rule, default values are returned (empty array, empty string). - When duplicating a prebuilt rule, these fields are being reset to their default value (empty array, empty string). - Export/Import is supported. Edge case / supported hack: it's possible to specify these fields manually in a ndjson doc and import with a rule. - The fields are being copied to `kibana.alert.rule.parameters` field of an alert document, which is mapped as a flattened field type. No special handling for the new fields was needed there. - Adjusts tests accordingly. ## Related Integrations Example (part of a rule returned from the API): ```json { "related_integrations": [ { "package": "windows", "version": "1.5.x" }, { "package": "azure", "integration": "activitylogs", "version": "~1.1.6" } ], } ``` Schema: ```ts /** * Related integration is a potential dependency of a rule. It's assumed that if the user installs * one of the related integrations of a rule, the rule might start to work properly because it will * have source events (generated by this integration) potentially matching the rule's query. * * NOTE: Proper work is not guaranteed, because a related integration, if installed, can be * configured differently or generate data that is not necessarily relevant for this rule. * * Related integration is a combination of a Fleet package and (optionally) one of the * package's "integrations" that this package contains. It is represented by 3 properties: * * - `package`: name of the package (required, unique id) * - `version`: version of the package (required, semver-compatible) * - `integration`: name of the integration of this package (optional, id within the package) * * There are Fleet packages like `windows` that contain only one integration; in this case, * `integration` should be unspecified. There are also packages like `aws` and `azure` that contain * several integrations; in this case, `integration` should be specified. * * @example * const x: RelatedIntegration = { * package: 'windows', * version: '1.5.x', * }; * * @example * const x: RelatedIntegration = { * package: 'azure', * version: '~1.1.6', * integration: 'activitylogs', * }; */ export type RelatedIntegration = t.TypeOf<typeof RelatedIntegration>; export const RelatedIntegration = t.exact( t.intersection([ t.type({ package: NonEmptyString, version: NonEmptyString, }), t.partial({ integration: NonEmptyString, }), ]) ); ``` ## Required Fields Example (part of a rule returned from the API): ```json { "required_fields": [ { "name": "event.action", "type": "keyword", "ecs": true }, { "name": "event.code", "type": "keyword", "ecs": true }, { "name": "winlog.event_data.AttributeLDAPDisplayName", "type": "keyword", "ecs": false } ], } ``` Schema: ```ts /** * Almost all types of Security rules check source event documents for a match to some kind of * query or filter. If a document has certain field with certain values, then it's a match and * the rule will generate an alert. * * Required field is an event field that must be present in the source indices of a given rule. * * @example * const standardEcsField: RequiredField = { * name: 'event.action', * type: 'keyword', * ecs: true, * }; * * @example * const nonEcsField: RequiredField = { * name: 'winlog.event_data.AttributeLDAPDisplayName', * type: 'keyword', * ecs: false, * }; */ export type RequiredField = t.TypeOf<typeof RequiredField>; export const RequiredField = t.exact( t.type({ name: NonEmptyString, type: NonEmptyString, ecs: t.boolean, }) ); ``` ## Setup Guide Example (part of a rule returned from the API): ```json { "setup": "## Config\n\nThe 'PowerShell Script Block Logging' logging policy must be enabled.\nSteps to implement the logging policy with with Advanced Audit Configuration:\n\n```\nComputer Configuration > \nAdministrative Templates > \nWindows PowerShell > \nTurn on PowerShell Script Block Logging (Enable)\n```\n\nSteps to implement the logging policy via registry:\n\n```\nreg add \"hklm\\SOFTWARE\\Policies\\Microsoft\\Windows\\PowerShell\\ScriptBlockLogging\" /v EnableScriptBlockLogging /t REG_DWORD /d 1\n```\n", } ``` Schema: ```ts /** * Any instructions for the user for setting up their environment in order to start receiving * source events for a given rule. * * It's a multiline text. Markdown is supported. */ export type SetupGuide = t.TypeOf<typeof SetupGuide>; export const SetupGuide = t.string; ``` ## Details on the schema This PR adjusts all the 6 rule schemas we have: 1. Alerting Framework rule `params` schema: - `security_solution/server/lib/detection_engine/schemas/rule_schemas.ts` - `security_solution/server/lib/detection_engine/schemas/rule_converters.ts` 2. HTTP API main old schema: - `security_solution/common/detection_engine/schemas/response/rules_schema.ts` 3. HTTP API main new schema: - `security_solution/common/detection_engine/schemas/request/rule_schemas.ts` 4. Prebuilt rule schema: - `security_solution/common/detection_engine/schemas/request/add_prepackaged_rules_schema.ts` 5. Import rule schema: - `security_solution/common/detection_engine/schemas/request/import_rules_schema.ts` 6. Rule schema used on the frontend side: - `security_solution/public/detections/containers/detection_engine/rules/types.ts` Names of the fields on the HTTP API level: - `related_integrations` - `required_fields` - `setup` Names of the fields on the Alerting Framework level: - `params.relatedIntegrations` - `params.requiredFields` - `params.setup` ## Next steps - Create a new endpoint for returning installed Fleet integrations (gonna be a separate PR). - Rebase elastic#131475 on top of this PR after merge. - Cover the new fields with dedicated tests (gonna be a separate PR). - Update API docs (gonna be a separate PR). - Address the tech debt of having 6 different schemas (gonna create a ticket for that). ### Checklist - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Peer-reviewed with @spong and code changes LGTM 👍
🚀 🚀 🚀
…lled Fleet integrations (#132667) **Addresses partially:** elastic/security-team#2856, elastic/security-team#3624 (internal tickets) ## Summary Adds a new detections endpoint that returns installed Fleet integrations. It is to be used on the Rule Management and Rule Details pages (see #131475 for context and screenshots). This endpoint is `internal` - no need to document it. ``` GET /internal/detection_engine/fleet/integrations/installed ``` ```json { "installed_integrations": [ { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "billing", "integration_title": "AWS Billing", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "cloudtrail", "integration_title": "AWS CloudTrail", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "cloudwatch", "integration_title": "AWS CloudWatch", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "dynamodb", "integration_title": "Amazon DynamoDB", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "ebs", "integration_title": "Amazon EBS", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "ec2", "integration_title": "Amazon EC2", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "elb", "integration_title": "AWS ELB", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "lambda", "integration_title": "AWS Lambda", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "natgateway", "integration_title": "Amazon NAT Gateway", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "firewall", "integration_title": "AWS Network Firewall", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "rds", "integration_title": "Amazon RDS", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "s3", "integration_title": "Amazon S3", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "s3_storage_lens", "integration_title": "Amazon S3 Storage Lens", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "sns", "integration_title": "Amazon SNS", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "sqs", "integration_title": "Amazon SQS", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "transitgateway", "integration_title": "AWS Transit Gateway", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "usage", "integration_title": "AWS Usage", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "vpcflow", "integration_title": "Amazon VPC", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "vpn", "integration_title": "Amazon VPN", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "waf", "integration_title": "AWS WAF", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "route53", "integration_title": "AWS Route 53", "is_enabled": false }, { "package_name": "aws", "package_title": "AWS", "package_version": "1.16.1", "integration_name": "cloudfront", "integration_title": "Amazon CloudFront", "is_enabled": true }, { "package_name": "system", "package_title": "System", "package_version": "1.13.0", "is_enabled": true } ] } ``` ## Next steps - Test with users with different privileges (non-superusers). Fleet privileges: none, read, all. Security Solution privileges. SO privileges. - Add filtering by `package_name` and `is_enabled` (will be done in a separate PR). - Add test coverage (will be done in a separate PR). ### Checklist Delete any items that are not applicable to this PR. - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
…o interface (#132847) ## Summary Wires up the new Installed Integrations API added in #132667 to the new Related Integrations UI added in #131475. #### Additional changes include (though not all necessary for this specific PR): - [X] Updates integrations badge icon to `package` on Rules Table - [ ] Add Kibana Advanced Setting for disabling integrations badge on Rules Table - [ ] Add loaders where necessary since there can now be API delay - [ ] Separate description step components to specific files Please see #131475 for screenshots and additional details. #### Steps to test In this initial implementation these new fields are only visible with Prebuilt Rules, and so there is limited API support and currently no UI for editing them. If a Prebuilt Rule is duplicated, these fields are emptied (set to `''` or `[]`). When a Rule is exported these fields are included (as empty values), and it is possible to edit the `ndjson` and re-import and then see these fields for the Custom Rule (but still not editable in the UI). This is expected behavior, and is actually a nice and easy way to test. Here is a sample export you can paste into a `test.ndjson` file and import to test this feature. You can modify the `package`/`version` fields to test corner cases like if a package is installed but it's the wrong version. ``` {"id":"6cc39c80-da3a-11ec-9fce-65c1a0bee904","updated_at":"2022-05-23T01:48:23.422Z","updated_by":"elastic","created_at":"2022-05-23T01:48:20.940Z","created_by":"elastic","name":"Testing #131475, don't mind me...","tags":["Elastic","Endpoint Security"],"interval":"5m","enabled":false,"description":"Generates a detection alert each time an Elastic Endpoint Security alert is received. Enabling this rule allows you to immediately begin investigating your Endpoint alerts.","risk_score":47,"severity":"medium","license":"Elastic License v2","output_index":".siem-signals-default","meta":{"from":"5m"},"rule_name_override":"message","timestamp_override":"event.ingested","author":["Elastic"],"false_positives":[],"from":"now-600s","rule_id":"2c66bf23-6ae9-4eb2-859e-446bea181ae9","max_signals":10000,"risk_score_mapping":[{"field":"event.risk_score","operator":"equals","value":""}],"severity_mapping":[{"field":"event.severity","operator":"equals","severity":"low","value":"21"},{"field":"event.severity","operator":"equals","severity":"medium","value":"47"},{"field":"event.severity","operator":"equals","severity":"high","value":"73"},{"field":"event.severity","operator":"equals","severity":"critical","value":"99"}],"threat":[],"to":"now","references":[],"version":7,"exceptions_list":[{"id":"endpoint_list","list_id":"endpoint_list","namespace_type":"agnostic","type":"endpoint"}],"immutable":false,"related_integrations":[{"package":"system","version":"1.6.4"},{"package":"aws","integration":"cloudtrail","version":"1.11.0"}],"required_fields":[{"ecs":true,"name":"event.code","type":"keyword"},{"ecs":true,"name":"message","type":"match_only_text"},{"ecs":false,"name":"winlog.event_data.AttributeLDAPDisplayName","type":"keyword"},{"ecs":false,"name":"winlog.event_data.AttributeValue","type":"keyword"},{"ecs":false,"name":"winlog.event_data.ShareName","type":"keyword"},{"ecs":false,"name":"winlog.event_data.RelativeTargetName","type":"keyword"},{"ecs":false,"name":"winlog.event_data.AccessList","type":"keyword"}],"setup":"## Config\\n\\nThe 'Audit Detailed File Share' audit policy must be configured (Success Failure).\\nSteps to implement the logging policy with with Advanced Audit Configuration:\\n\\n```\\nComputer Configuration > \\nPolicies > \\nWindows Settings > \\nSecurity Settings > \\nAdvanced Audit Policies Configuration > \\nAudit Policies > \\nObject Access > \\nAudit Detailed File Share (Success,Failure)\\n```\\n\\nThe 'Audit Directory Service Changes' audit policy must be configured (Success Failure).\\nSteps to implement the logging policy with with Advanced Audit Configuration:\\n\\n```\\nComputer Configuration > \\nPolicies > \\nWindows Settings > \\nSecurity Settings > \\nAdvanced Audit Policies Configuration > \\nAudit Policies > \\nDS Access > \\nAudit Directory Service Changes (Success,Failure)\\n```\\n","type":"query","language":"kuery","index":["logs-endpoint.alerts-*"],"query":"event.kind:alert and event.module:(endpoint and not endgame)\\n","filters":[],"throttle":"no_actions","actions":[]} {"exported_count":1,"exported_rules_count":1,"missing_rules":[],"missing_rules_count":0,"exported_exception_list_count":0,"exported_exception_list_item_count":0,"missing_exception_list_item_count":0,"missing_exception_list_items":[],"missing_exception_lists":[],"missing_exception_lists_count":0} ```
## Summary A new Security Solution feature (#131475) was added in `8.3` that displays a field name and icon token using the reusable `FieldIcon` component. In testing an issue was reported (#133291) that the wrong icon token was being displayed. I had [previously updated](https://github.com/elastic/kibana/pull/131475/files#diff-d79a8297783f3177da25dd13fe807425d9136a0e235fe170f7c0a61f2448dacaR23) `FieldIcon` to support `match_only_text` however this new issue was with the `float` type not displaying correctly. After some searching I found the `castEsToKbnFieldTypeName` utility which solved the issue with `float` fields not displaying, but then `match_only_text` field types would not show since it is missing from `ES_FIELD_TYPES` and so would resolve as `unknown`. This PR adds the `match_only_text` [ES type](https://www.elastic.co/guide/en/elasticsearch/reference/current/text.html#match-only-text-field-type) to `ES_FIELD_TYPES` to resolve this missing icon token issue so that `castEsToKbnFieldTypeName` can be used in conjunction with the resuable `FieldIcon` component. I imagine this is fine as it's a sibling type to `text`, but am curious here since `sortable: true,` is set for the KbnFieldType even though it includes `ES_FIELD_TYPES.TEXT` (which is not sortable) as well?
## Summary A new Security Solution feature (#131475) was added in `8.3` that displays a field name and icon token using the reusable `FieldIcon` component. In testing an issue was reported (#133291) that the wrong icon token was being displayed. I had [previously updated](https://github.com/elastic/kibana/pull/131475/files#diff-d79a8297783f3177da25dd13fe807425d9136a0e235fe170f7c0a61f2448dacaR23) `FieldIcon` to support `match_only_text` however this new issue was with the `float` type not displaying correctly. After some searching I found the `castEsToKbnFieldTypeName` utility which solved the issue with `float` fields not displaying, but then `match_only_text` field types would not show since it is missing from `ES_FIELD_TYPES` and so would resolve as `unknown`. This PR adds the `match_only_text` [ES type](https://www.elastic.co/guide/en/elasticsearch/reference/current/text.html#match-only-text-field-type) to `ES_FIELD_TYPES` to resolve this missing icon token issue so that `castEsToKbnFieldTypeName` can be used in conjunction with the resuable `FieldIcon` component. I imagine this is fine as it's a sibling type to `text`, but am curious here since `sortable: true,` is set for the KbnFieldType even though it includes `ES_FIELD_TYPES.TEXT` (which is not sortable) as well? (cherry picked from commit de3410e)
## Summary A new Security Solution feature (#131475) was added in `8.3` that displays a field name and icon token using the reusable `FieldIcon` component. In testing an issue was reported (#133291) that the wrong icon token was being displayed. I had [previously updated](https://github.com/elastic/kibana/pull/131475/files#diff-d79a8297783f3177da25dd13fe807425d9136a0e235fe170f7c0a61f2448dacaR23) `FieldIcon` to support `match_only_text` however this new issue was with the `float` type not displaying correctly. After some searching I found the `castEsToKbnFieldTypeName` utility which solved the issue with `float` fields not displaying, but then `match_only_text` field types would not show since it is missing from `ES_FIELD_TYPES` and so would resolve as `unknown`. This PR adds the `match_only_text` [ES type](https://www.elastic.co/guide/en/elasticsearch/reference/current/text.html#match-only-text-field-type) to `ES_FIELD_TYPES` to resolve this missing icon token issue so that `castEsToKbnFieldTypeName` can be used in conjunction with the resuable `FieldIcon` component. I imagine this is fine as it's a sibling type to `text`, but am curious here since `sortable: true,` is set for the KbnFieldType even though it includes `ES_FIELD_TYPES.TEXT` (which is not sortable) as well? (cherry picked from commit de3410e) Co-authored-by: Garrett Spong <spong@users.noreply.github.com>
@nastasha-solomon I changed the label here to align with our decision to treat Related Integrations, Required Fields, and Setup Guide as new features rather than enhancements in the release notes. |
Summary
Adds UI for new Rule Fields
Related Integrations
,Required Fields
, andSetup
to both the Rules Table and Rule Details pages. On the Rules Table a new column is added that shows the number of related integrations, and upon clicking will show you details about those integrations and links off to the integration page in fleet. On the Rule Details pageSetup
is added as a tab pill within the About section (if provided), andRelated Integrations
andRequired Fields
are displayed in the Definition section.Once package/integration install data is added in #132667, the UI will show the installed status of an integration, and whether or not the installed version satisfies the related integration dependency.
NOTE: Until then, please follow the test instructions below for how to add a custom rule and return mock data to test the
installed/uninstalled
UI.Related Issues
Related Links
Related Integrations
,Required Fields
andSetup
security-docs#2015Steps to test
In this initial implementation these new fields are only visible with Prebuilt Rules, and so there is limited API support and currently no UI for editing them. If a Prebuilt Rule is duplicated, these fields are emptied (set to
''
or[]
). When a Rule is exported these fields are included (as empty values), and it is possible to edit thendjson
and re-import and then see these fields for the Custom Rule (but still not editable in the UI). This is expected behavior, and is actually a nice and easy way to test.Here is a sample export you can paste into an
test.ndjson
file and import to test this feature. You can modify thepackage
/version
fields to test corner cases like if a package is installed but it's the wrong version.Existing plumbing for showing integration install state
This PR includes a
useInstalledIntegrations
hook wired up to theDETECTION_ENGINE_INSTALLED_INTEGRATIONS_URL
route to be added in #132667. I plumbed the initial logic as if that API returned an array of integrations in the same format stored by the rule (i.e. theRelatedIntegrationArray
type), so this will need to be adapted when integrating this feature. There's also apackages[]
that can be provided touseInstalledIntegrations
to constrain the search against all installed integrations, but this may not be used in the initial API.To test the Installed Integrations UI, just uncomment the mock data return in
use_installed_integrations.tsx
.Additional Notes/Todo:
Related Integrations
/Required Fields
. We don't really have an overflow pattern for these description list items, so instead of just adding support for these two fields (as like another description list item that's a popover), would like to solve this for generically for all items.Rule Details
Rule Details without
Installed Integrations
API changesRules Table
Rules Table without
Installed Integrations
API changesVersion mismatch
In cases where the related package/integration is installed, but the version is not satisfied, a warning icon/tooltip will display next to the integration link letting the user know the installed vs targeted version. I just wanted to make sure this case was handled so copy/UI isn't final -- any feedback welcome here! 🙂
Checklist
Delete any items that are not applicable to this PR.
Related Integrations
,Required Fields
andSetup
security-docs#2015(https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))