Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Security Solution][Detections] Add new fields to the rule model: Rel…
…ated Integrations, Required Fields, and Setup (elastic#132409) **Addresses partially:** elastic/security-team#2083, elastic/security-team#558, elastic/security-team#2856, elastic/security-team#1801 (internal tickets) ## Summary **TL;DR:** With this PR, it's now possible to specify `related_integrations`, `required_fields`, and `setup` fields in prebuilt rules in https://github.com/elastic/detection-rules. They are returned within rules in the API responses. This PR: - Adds 3 new fields to the model of Security detection rules. These fields are common to all of the rule types we have. - **Related Integrations**. It's a list of Fleet integrations associated with a given rule. It's assumed that if the user installs them, the rule might start to work properly because it will start receiving source events potentially matching the rule's query. - **Required Fields**. It's a list of event fields that must be present in the source indices of a given rule. - **Setup Guide**. It's any instructions for the user for setting up their environment in order to start receiving source events for a given rule. It's a text. Markdown is supported. It's similar to the Investigation Guide that we show on the Details page. - Adjusts API endpoints accordingly: - These fields are for prebuilt rules only and are supposed to be read-only in the UI. - Specifying these fields in the request parameters of the create/update/patch rule API endpoints is not supported. - These fields are returned in all responses that contain rules. If they are missing in a rule, default values are returned (empty array, empty string). - When duplicating a prebuilt rule, these fields are being reset to their default value (empty array, empty string). - Export/Import is supported. Edge case / supported hack: it's possible to specify these fields manually in a ndjson doc and import with a rule. - The fields are being copied to `kibana.alert.rule.parameters` field of an alert document, which is mapped as a flattened field type. No special handling for the new fields was needed there. - Adjusts tests accordingly. ## Related Integrations Example (part of a rule returned from the API): ```json { "related_integrations": [ { "package": "windows", "version": "1.5.x" }, { "package": "azure", "integration": "activitylogs", "version": "~1.1.6" } ], } ``` Schema: ```ts /** * Related integration is a potential dependency of a rule. It's assumed that if the user installs * one of the related integrations of a rule, the rule might start to work properly because it will * have source events (generated by this integration) potentially matching the rule's query. * * NOTE: Proper work is not guaranteed, because a related integration, if installed, can be * configured differently or generate data that is not necessarily relevant for this rule. * * Related integration is a combination of a Fleet package and (optionally) one of the * package's "integrations" that this package contains. It is represented by 3 properties: * * - `package`: name of the package (required, unique id) * - `version`: version of the package (required, semver-compatible) * - `integration`: name of the integration of this package (optional, id within the package) * * There are Fleet packages like `windows` that contain only one integration; in this case, * `integration` should be unspecified. There are also packages like `aws` and `azure` that contain * several integrations; in this case, `integration` should be specified. * * @example * const x: RelatedIntegration = { * package: 'windows', * version: '1.5.x', * }; * * @example * const x: RelatedIntegration = { * package: 'azure', * version: '~1.1.6', * integration: 'activitylogs', * }; */ export type RelatedIntegration = t.TypeOf<typeof RelatedIntegration>; export const RelatedIntegration = t.exact( t.intersection([ t.type({ package: NonEmptyString, version: NonEmptyString, }), t.partial({ integration: NonEmptyString, }), ]) ); ``` ## Required Fields Example (part of a rule returned from the API): ```json { "required_fields": [ { "name": "event.action", "type": "keyword", "ecs": true }, { "name": "event.code", "type": "keyword", "ecs": true }, { "name": "winlog.event_data.AttributeLDAPDisplayName", "type": "keyword", "ecs": false } ], } ``` Schema: ```ts /** * Almost all types of Security rules check source event documents for a match to some kind of * query or filter. If a document has certain field with certain values, then it's a match and * the rule will generate an alert. * * Required field is an event field that must be present in the source indices of a given rule. * * @example * const standardEcsField: RequiredField = { * name: 'event.action', * type: 'keyword', * ecs: true, * }; * * @example * const nonEcsField: RequiredField = { * name: 'winlog.event_data.AttributeLDAPDisplayName', * type: 'keyword', * ecs: false, * }; */ export type RequiredField = t.TypeOf<typeof RequiredField>; export const RequiredField = t.exact( t.type({ name: NonEmptyString, type: NonEmptyString, ecs: t.boolean, }) ); ``` ## Setup Guide Example (part of a rule returned from the API): ```json { "setup": "## Config\n\nThe 'PowerShell Script Block Logging' logging policy must be enabled.\nSteps to implement the logging policy with with Advanced Audit Configuration:\n\n```\nComputer Configuration > \nAdministrative Templates > \nWindows PowerShell > \nTurn on PowerShell Script Block Logging (Enable)\n```\n\nSteps to implement the logging policy via registry:\n\n```\nreg add \"hklm\\SOFTWARE\\Policies\\Microsoft\\Windows\\PowerShell\\ScriptBlockLogging\" /v EnableScriptBlockLogging /t REG_DWORD /d 1\n```\n", } ``` Schema: ```ts /** * Any instructions for the user for setting up their environment in order to start receiving * source events for a given rule. * * It's a multiline text. Markdown is supported. */ export type SetupGuide = t.TypeOf<typeof SetupGuide>; export const SetupGuide = t.string; ``` ## Details on the schema This PR adjusts all the 6 rule schemas we have: 1. Alerting Framework rule `params` schema: - `security_solution/server/lib/detection_engine/schemas/rule_schemas.ts` - `security_solution/server/lib/detection_engine/schemas/rule_converters.ts` 2. HTTP API main old schema: - `security_solution/common/detection_engine/schemas/response/rules_schema.ts` 3. HTTP API main new schema: - `security_solution/common/detection_engine/schemas/request/rule_schemas.ts` 4. Prebuilt rule schema: - `security_solution/common/detection_engine/schemas/request/add_prepackaged_rules_schema.ts` 5. Import rule schema: - `security_solution/common/detection_engine/schemas/request/import_rules_schema.ts` 6. Rule schema used on the frontend side: - `security_solution/public/detections/containers/detection_engine/rules/types.ts` Names of the fields on the HTTP API level: - `related_integrations` - `required_fields` - `setup` Names of the fields on the Alerting Framework level: - `params.relatedIntegrations` - `params.requiredFields` - `params.setup` ## Next steps - Create a new endpoint for returning installed Fleet integrations (gonna be a separate PR). - Rebase elastic#131475 on top of this PR after merge. - Cover the new fields with dedicated tests (gonna be a separate PR). - Update API docs (gonna be a separate PR). - Address the tech debt of having 6 different schemas (gonna create a ticket for that). ### Checklist - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
- Loading branch information