Skip to content

Commit

Permalink
[8.x] [Security Solution] Integrate Prebuilt Rules Customization UI w…
Browse files Browse the repository at this point in the history
…ith the `_perform` upgrade API (elastic#199761) (elastic#200193)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Security Solution] Integrate Prebuilt Rules Customization UI with
the `_perform` upgrade API
(elastic#199761)](elastic#199761)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Maxim
Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2024-11-14T14:13:20Z","message":"[Security
Solution] Integrate Prebuilt Rules Customization UI with the `_perform`
upgrade API (elastic#199761)\n\n**Partially addresses:**
https://github.com/elastic/kibana/issues/171520\r\n\r\n##
Summary\r\n\r\nThis PR integrates Prebuilt Rules Customization UI
functionality with\r\nthe
`/internal/detection_engine/prebuilt_rules/upgrade/_perform`\r\nPrebuilt
Rules Customization upgrade API.\r\n\r\n> [!CAUTION]\r\n> This PR
doesn't handle rule type changes. Prebuilt rule updates with\r\nrule
type change consider having a NON SOLVABLE conflict and won't
be\r\nupgraded neither individually nor in bulk. Addressing that task
requires\r\nUI and functional changes and will be addressed in a
separate PR.\r\n\r\n## Details\r\n\r\n\r\n## How to test\r\n\r\n- Clear
Elasticsearch data\r\n- Run Elasticsearch and Kibana locally (do not
open Kibana in a web\r\nbrowser)\r\n- Install an outdated version of the
`security_detection_engine` Fleet\r\npackage\r\n ```bash\r\ncurl -X POST
--user elastic:changeme -H 'Content-Type: application/json'\r\n-H
'kbn-xsrf: 123' -H \"elastic-api-version: 2023-10-31\"
-d\r\n'{\"force\":true}'\r\nhttp://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1\r\n
```\r\n- Install prebuilt rules\r\n ```bash\r\ncurl -X POST --user
elastic:changeme -H 'Content-Type: application/json'\r\n-H 'kbn-xsrf:
123' -H \"elastic-api-version: 1\" -d
'{\"mode\":\"ALL_RULES\"}'\r\nhttp://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform\r\n
```\r\n- Open `Detection Rules (SIEM)` Page -> `Rule Updates`\r\n- [ ]
Check update functionality in a flyout\r\n - Pick a rule\r\n - Click on
rule's name\r\n - Make changes to fields in incoming rule updates
updates\r\n - Save field(s) changes\r\n - Press the `Update` button\r\n-
[ ] Check table row rule update\r\n - Pick a rule\r\n - Click on rule's
name\r\n - Make changes to fields in incoming rule updates updates\r\n -
Save field(s) changes\r\n - Close the flyout\r\n - Press the `Update
rule` button in the rule's table row\r\n- [ ] Check bulk rule update\r\n
- Pick a few rules and for each of them do the next steps\r\n - Click on
rule's name\r\n - Make changes to fields in incoming rule updates
updates\r\n - Save field(s) changes\r\n - Close the flyout\r\n - After
press the `Update All` button on the page\r\n- [ ] Check selected rules
bulk update\r\n - Pick a few rules and for each of them do the next
steps\r\n - Click on rule's name\r\n - Make changes to fields in
incoming rule updates updates\r\n - Save field(s) changes\r\n - Close
the flyout\r\n - After select the modified rule updates\r\n - Press the
`Update N selected rule(s)` button on the page\r\n\r\nCo-authored-by:
Dmitrii Shevchenko
<dmitrii.shevchenko@elastic.co>","sha":"1862b5914786a207238c650465c3d6b3f04ab172","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","Feature:Prebuilt Detection
Rules","backport:version","v8.17.0"],"title":"[Security Solution]
Integrate Prebuilt Rules Customization UI with the `_perform` upgrade
API","number":199761,"url":"https://github.com/elastic/kibana/pull/199761","mergeCommit":{"message":"[Security
Solution] Integrate Prebuilt Rules Customization UI with the `_perform`
upgrade API (elastic#199761)\n\n**Partially addresses:**
https://github.com/elastic/kibana/issues/171520\r\n\r\n##
Summary\r\n\r\nThis PR integrates Prebuilt Rules Customization UI
functionality with\r\nthe
`/internal/detection_engine/prebuilt_rules/upgrade/_perform`\r\nPrebuilt
Rules Customization upgrade API.\r\n\r\n> [!CAUTION]\r\n> This PR
doesn't handle rule type changes. Prebuilt rule updates with\r\nrule
type change consider having a NON SOLVABLE conflict and won't
be\r\nupgraded neither individually nor in bulk. Addressing that task
requires\r\nUI and functional changes and will be addressed in a
separate PR.\r\n\r\n## Details\r\n\r\n\r\n## How to test\r\n\r\n- Clear
Elasticsearch data\r\n- Run Elasticsearch and Kibana locally (do not
open Kibana in a web\r\nbrowser)\r\n- Install an outdated version of the
`security_detection_engine` Fleet\r\npackage\r\n ```bash\r\ncurl -X POST
--user elastic:changeme -H 'Content-Type: application/json'\r\n-H
'kbn-xsrf: 123' -H \"elastic-api-version: 2023-10-31\"
-d\r\n'{\"force\":true}'\r\nhttp://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1\r\n
```\r\n- Install prebuilt rules\r\n ```bash\r\ncurl -X POST --user
elastic:changeme -H 'Content-Type: application/json'\r\n-H 'kbn-xsrf:
123' -H \"elastic-api-version: 1\" -d
'{\"mode\":\"ALL_RULES\"}'\r\nhttp://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform\r\n
```\r\n- Open `Detection Rules (SIEM)` Page -> `Rule Updates`\r\n- [ ]
Check update functionality in a flyout\r\n - Pick a rule\r\n - Click on
rule's name\r\n - Make changes to fields in incoming rule updates
updates\r\n - Save field(s) changes\r\n - Press the `Update` button\r\n-
[ ] Check table row rule update\r\n - Pick a rule\r\n - Click on rule's
name\r\n - Make changes to fields in incoming rule updates updates\r\n -
Save field(s) changes\r\n - Close the flyout\r\n - Press the `Update
rule` button in the rule's table row\r\n- [ ] Check bulk rule update\r\n
- Pick a few rules and for each of them do the next steps\r\n - Click on
rule's name\r\n - Make changes to fields in incoming rule updates
updates\r\n - Save field(s) changes\r\n - Close the flyout\r\n - After
press the `Update All` button on the page\r\n- [ ] Check selected rules
bulk update\r\n - Pick a few rules and for each of them do the next
steps\r\n - Click on rule's name\r\n - Make changes to fields in
incoming rule updates updates\r\n - Save field(s) changes\r\n - Close
the flyout\r\n - After select the modified rule updates\r\n - Press the
`Update N selected rule(s)` button on the page\r\n\r\nCo-authored-by:
Dmitrii Shevchenko
<dmitrii.shevchenko@elastic.co>","sha":"1862b5914786a207238c650465c3d6b3f04ab172"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/199761","number":199761,"mergeCommit":{"message":"[Security
Solution] Integrate Prebuilt Rules Customization UI with the `_perform`
upgrade API (elastic#199761)\n\n**Partially addresses:**
https://github.com/elastic/kibana/issues/171520\r\n\r\n##
Summary\r\n\r\nThis PR integrates Prebuilt Rules Customization UI
functionality with\r\nthe
`/internal/detection_engine/prebuilt_rules/upgrade/_perform`\r\nPrebuilt
Rules Customization upgrade API.\r\n\r\n> [!CAUTION]\r\n> This PR
doesn't handle rule type changes. Prebuilt rule updates with\r\nrule
type change consider having a NON SOLVABLE conflict and won't
be\r\nupgraded neither individually nor in bulk. Addressing that task
requires\r\nUI and functional changes and will be addressed in a
separate PR.\r\n\r\n## Details\r\n\r\n\r\n## How to test\r\n\r\n- Clear
Elasticsearch data\r\n- Run Elasticsearch and Kibana locally (do not
open Kibana in a web\r\nbrowser)\r\n- Install an outdated version of the
`security_detection_engine` Fleet\r\npackage\r\n ```bash\r\ncurl -X POST
--user elastic:changeme -H 'Content-Type: application/json'\r\n-H
'kbn-xsrf: 123' -H \"elastic-api-version: 2023-10-31\"
-d\r\n'{\"force\":true}'\r\nhttp://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1\r\n
```\r\n- Install prebuilt rules\r\n ```bash\r\ncurl -X POST --user
elastic:changeme -H 'Content-Type: application/json'\r\n-H 'kbn-xsrf:
123' -H \"elastic-api-version: 1\" -d
'{\"mode\":\"ALL_RULES\"}'\r\nhttp://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform\r\n
```\r\n- Open `Detection Rules (SIEM)` Page -> `Rule Updates`\r\n- [ ]
Check update functionality in a flyout\r\n - Pick a rule\r\n - Click on
rule's name\r\n - Make changes to fields in incoming rule updates
updates\r\n - Save field(s) changes\r\n - Press the `Update` button\r\n-
[ ] Check table row rule update\r\n - Pick a rule\r\n - Click on rule's
name\r\n - Make changes to fields in incoming rule updates updates\r\n -
Save field(s) changes\r\n - Close the flyout\r\n - Press the `Update
rule` button in the rule's table row\r\n- [ ] Check bulk rule update\r\n
- Pick a few rules and for each of them do the next steps\r\n - Click on
rule's name\r\n - Make changes to fields in incoming rule updates
updates\r\n - Save field(s) changes\r\n - Close the flyout\r\n - After
press the `Update All` button on the page\r\n- [ ] Check selected rules
bulk update\r\n - Pick a few rules and for each of them do the next
steps\r\n - Click on rule's name\r\n - Make changes to fields in
incoming rule updates updates\r\n - Save field(s) changes\r\n - Close
the flyout\r\n - After select the modified rule updates\r\n - Press the
`Update N selected rule(s)` button on the page\r\n\r\nCo-authored-by:
Dmitrii Shevchenko
<dmitrii.shevchenko@elastic.co>","sha":"1862b5914786a207238c650465c3d6b3f04ab172"}},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
  • Loading branch information
kibanamachine and maximpn authored Nov 14, 2024
1 parent 072467c commit 63b2b6f
Show file tree
Hide file tree
Showing 2 changed files with 68 additions and 41 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,18 @@
import type { Dispatch, SetStateAction } from 'react';
import React, { createContext, useCallback, useContext, useMemo, useState } from 'react';
import { EuiButton, EuiToolTip } from '@elastic/eui';
import { isNonUpgradeableFieldName } from '../../../../rule_management/model/prebuilt_rule_upgrade/fields';
import type {
RuleFieldsToUpgrade,
RuleUpgradeSpecifier,
RuleUpgradeInfoForReview,
} from '../../../../../../common/api/detection_engine';
import { useIsPrebuiltRulesCustomizationEnabled } from '../../../../rule_management/hooks/use_is_prebuilt_rules_customization_enabled';
import { useAppToasts } from '../../../../../common/hooks/use_app_toasts';
import type { RuleUpgradeInfoForReview } from '../../../../../../common/api/detection_engine';
import type { RulesUpgradeState } from '../../../../rule_management/model/prebuilt_rule_upgrade';
import type {
RuleUpgradeState,
RulesUpgradeState,
} from '../../../../rule_management/model/prebuilt_rule_upgrade';
import { RuleUpgradeConflictsResolverTab } from '../../../../rule_management/components/rule_details/three_way_diff/rule_upgrade_conflicts_resolver_tab';
import { PerFieldRuleDiffTab } from '../../../../rule_management/components/rule_details/per_field_rule_diff_tab';
import { useIsUpgradingSecurityPackages } from '../../../../rule_management/logic/use_upgrade_security_packages';
Expand All @@ -26,13 +34,14 @@ import type { UpgradePrebuiltRulesTableFilterOptions } from './use_filter_prebui
import { useFilterPrebuiltRulesToUpgrade } from './use_filter_prebuilt_rules_to_upgrade';
import { TabContentPadding } from '../../../../rule_management/components/rule_details/rule_details_flyout';
import { RuleDiffTab } from '../../../../rule_management/components/rule_details/rule_diff_tab';
import { FieldUpgradeState } from '../../../../rule_management/model/prebuilt_rule_upgrade/field_upgrade_state';
import { useRulePreviewFlyout } from '../use_rule_preview_flyout';
import { MlJobUpgradeModal } from './modals/ml_job_upgrade_modal';
import { UpgradeConflictsModal } from './modals/upgrade_conflicts_modal';
import * as ruleDetailsI18n from '../../../../rule_management/components/rule_details/translations';
import * as i18n from './translations';
import { usePrebuiltRulesUpgradeState } from './use_prebuilt_rules_upgrade_state';
import { useRulePreviewFlyout } from '../use_rule_preview_flyout';
import { useMlJobUpgradeModal, useUpgradeConflictsModal } from './use_upgrade_modals';
import * as ruleDetailsI18n from '../../../../rule_management/components/rule_details/translations';
import * as i18n from './translations';

export interface UpgradePrebuiltRulesTableState {
/**
Expand Down Expand Up @@ -160,70 +169,64 @@ export const UpgradePrebuiltRulesTableContextProvider = ({
} = useUpgradeConflictsModal();

const shouldConfirmMLJobs = legacyJobsInstalled.length > 0;
const getRulesWithConflicts = useCallback(
(ruleIds?: RuleSignatureId[]) => {
const rulesToUpgrade =
ruleIds?.map((ruleId) => {
const rule = rulesUpgradeState[ruleId];
invariant(rule, `Rule with ID ${ruleId} not found.`);

return rule;
}) ?? [];

return rulesToUpgrade.filter((rule) => rule.diff.num_fields_with_conflicts > 0);
},
[rulesUpgradeState]
);

const { mutateAsync: upgradeSpecificRulesRequest } = usePerformUpgradeSpecificRules({
pickVersion: isPrebuiltRulesCustomizationEnabled ? 'MERGED' : 'TARGET',
});

const upgradeRules = useCallback(
async (ruleIds: RuleSignatureId[]) => {
const rulesToUpgrade = ruleIds.map((ruleId) => ({
rule_id: ruleId,
version:
rulesUpgradeState[ruleId].diff.fields.version?.target_version ??
rulesUpgradeState[ruleId].current_rule.version,
revision: rulesUpgradeState[ruleId].revision,
}));
setLoadingRules((prev) => [...prev, ...rulesToUpgrade.map((r) => r.rule_id)]);
const conflictRuleIdsSet = new Set(
isPrebuiltRulesCustomizationEnabled
? ruleIds.filter(
(ruleId) =>
rulesUpgradeState[ruleId].diff.num_fields_with_conflicts > 0 &&
rulesUpgradeState[ruleId].hasUnresolvedConflicts
)
: []
);
const ruleUpgradeSpecifiers: RuleUpgradeSpecifier[] = ruleIds
.filter((ruleId) => !conflictRuleIdsSet.has(ruleId))
.map((ruleId) => ({
rule_id: ruleId,
version:
rulesUpgradeState[ruleId].diff.fields.version?.target_version ??
rulesUpgradeState[ruleId].current_rule.version,
revision: rulesUpgradeState[ruleId].revision,
fields: isPrebuiltRulesCustomizationEnabled
? constructRuleFieldsToUpgrade(rulesUpgradeState[ruleId])
: undefined,
}));

setLoadingRules((prev) => [...prev, ...ruleUpgradeSpecifiers.map((x) => x.rule_id)]);

try {
// Handle MLJobs modal
if (shouldConfirmMLJobs && !(await confirmLegacyMLJobs())) {
return;
}

// Handle Rule Upgrades modal
const rulesWithConflicts = getRulesWithConflicts(ruleIds);
if (
isPrebuiltRulesCustomizationEnabled &&
rulesWithConflicts.length > 0 &&
conflictRuleIdsSet.size > 0 &&
!(await confirmConflictsUpgrade())
) {
return;
}

// Prepare payload for upgrade with rules with no conflicts
const ruleIdsWithConflicts = new Set(rulesWithConflicts.map((rule) => rule.rule_id));
const rulesToUpgradeWithNoConflicts = isPrebuiltRulesCustomizationEnabled
? rulesToUpgrade.filter((rule) => !ruleIdsWithConflicts.has(rule.rule_id))
: rulesToUpgrade;
await upgradeSpecificRulesRequest(rulesToUpgradeWithNoConflicts);
await upgradeSpecificRulesRequest(ruleUpgradeSpecifiers);
} catch (err) {
addError(err, { title: i18n.UPDATE_ERROR });
} finally {
setLoadingRules((prev) =>
prev.filter((id) => !rulesToUpgrade.some((r) => r.rule_id === id))
);
const upgradedRuleIdsSet = new Set(ruleUpgradeSpecifiers.map((x) => x.rule_id));

setLoadingRules((prev) => prev.filter((id) => !upgradedRuleIdsSet.has(id)));
}
},
[
confirmLegacyMLJobs,
confirmConflictsUpgrade,
shouldConfirmMLJobs,
getRulesWithConflicts,
rulesUpgradeState,
upgradeSpecificRulesRequest,
isPrebuiltRulesCustomizationEnabled,
Expand Down Expand Up @@ -408,3 +411,26 @@ export const useUpgradePrebuiltRulesTableContext = (): UpgradePrebuiltRulesConte

return rulesTableContext;
};

function constructRuleFieldsToUpgrade(ruleUpgradeState: RuleUpgradeState): RuleFieldsToUpgrade {
const finalRule = ruleUpgradeState.finalRule as Record<string, unknown>;
const ruleFieldsToUpgrade: Record<string, unknown> = {};

for (const fieldName of Object.keys(ruleUpgradeState.fieldsUpgradeState)) {
const fieldUpgradeState = ruleUpgradeState.fieldsUpgradeState[fieldName];

if (!isNonUpgradeableFieldName(fieldName) && fieldUpgradeState === FieldUpgradeState.Accepted) {
invariant(
fieldName in finalRule,
`Ready to upgrade field "${fieldName}" is not found in final rule`
);

ruleFieldsToUpgrade[fieldName] = {
pick_version: 'RESOLVED',
resolved_value: finalRule[fieldName],
};
}
}

return ruleFieldsToUpgrade;
}
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,12 @@ import {
type DiffableRule,
type RuleUpgradeInfoForReview,
ThreeWayDiffConflict,
type RuleSignatureId,
} from '../../../../../../common/api/detection_engine';
import { convertRuleToDiffable } from '../../../../../../common/detection_engine/prebuilt_rules/diff/convert_rule_to_diffable';

type RuleResolvedConflicts = Partial<DiffableAllFields>;
type RulesResolvedConflicts = Record<string, RuleResolvedConflicts>;
type RulesResolvedConflicts = Record<RuleSignatureId, RuleResolvedConflicts>;

interface UseRulesUpgradeStateResult {
rulesUpgradeState: RulesUpgradeState;
Expand Down

0 comments on commit 63b2b6f

Please sign in to comment.