[Security Solution] Fix inability to unset optional field values (ela…

…stic#204231)

**Resolves: elastic#203634

## Summary

This PR fixes bugs blocking unsetting optional rule field values in rule
upgrade workflow.

## Details

Changes here cover 3 groups of fields optional, string fields allowing
empty strings and array fields allowing empty arrays. It was verified
that fields in that groups allow to unset the value.

The following issues were fixed
- inability to set an empty string or `setup` and `note` fields
It required adding `stripEmptyFields: false` for rule upgrade fields
edit form.
- inability to unset `timestamp_override` field
  Timestamp override form deserializer was fixed.
- inability to unset `alert_suppression`
Alert Suppression was excluded from special special fields list always
upgrading to the current value. It's expected Alert Suppression won't be
included in Prebuilt Rules delivered in prebuilt rules packages. The
only way to get this setting and have it included in rule upgrade flyout
is editing a prebuilt rule by a user with a sufficient licence.

The following fields were verified and fixed where necessary

### Optional fields

- ✅ `investigation_fields`
- ✅ `rule_name_override`
- ⚠️ `timestamp_override` (field's form deserializer was fixed)
- ✅ `timeline_template`
- ✅ `building_block`
- ⚠️ `alert_suppression` (the field was excluded from special special
fields list always upgrading to the current value)
- ✅ `threat_indicator_path` (empty value resets to default
`threat.indicator`)

### String fields allowing empty strings

- ⚠️ `note` (required adding `stripEmptyFields: false` to the form)
- ⚠️ `setup` (required adding `stripEmptyFields: false` to the form)

### Array fields allowing empty arrays

- ✅ `tags`
- ✅ `references`
- ✅ `false_positives`
- ✅ `threat`
- ✅ `related_integrations`
- ✅ `required_fields`
- ✅ `severity_mapping`
- ✅ `risk_score_mapping`

## Screenshots

![Screenshot 2024-12-17 at 09 15
14](https://github.com/user-attachments/assets/671f5198-55da-4899-ab52-1e93f3c841af)


https://github.com/user-attachments/assets/bd36e5ba-e7fb-4733-a792-ea5435d579e2

## How to test?

- Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled
- Allow internal APIs via adding `server.restrictInternalApis: false` to
`kibana.dev.yaml`
- Clear Elasticsearch data
- Run Elasticsearch and Kibana locally (do not open Kibana in a web
browser)
- Install an outdated version of the `security_detection_engine` Fleet
package
```bash
curl -X POST --user elastic:changeme  -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"force":true}' http://localhost:5601/kbn/api/fleet/epm/packages/security_detection_engine/8.14.1
```

- Install prebuilt rules
```bash
curl -X POST --user elastic:changeme  -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 1" -d '{"mode":"ALL_RULES"}' http://localhost:5601/kbn/internal/detection_engine/prebuilt_rules/installation/_perform
```

- Customize one or more rules (change fields to see them in rule upgrade
workflow)
- Open Rule upgrade for the rule(s)
- Unset field values
- Upgrade rule(s)

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

x-pack/solutions/security/plugins/security_solution/common/api/detection_engine/prebuilt_rules/perform_rule_upgrade/perform_rule_upgrade_route.ts

@@ -26,7 +26,6 @@ export const PickVersionValuesEnum = PickVersionValues.enum;
 export const FIELDS_TO_UPGRADE_TO_CURRENT_VERSION = [
   'enabled',
   'exceptions_list',
-  'alert_suppression',
   'actions',
   'throttle',
   'response_actions',

x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management/components/rule_details/three_way_diff/field_final_side/components/rule_field_edit_form_wrapper.tsx

@@ -100,6 +100,7 @@ export function RuleFieldEditFormWrapper({
     onSubmit: handleSubmit,
     options: {
       warningValidationCodes: VALIDATION_WARNING_CODES,
+      stripEmptyFields: false,
     },
   });
 

x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management/components/rule_details/three_way_diff/final_edit/fields/timestamp_override.tsx

@@ -72,10 +72,11 @@ function TimestampFallbackDisabled() {
   return null;
 }
 
-export function timestampOverrideDeserializer(defaultValue: FormData) {
+export function timestampOverrideDeserializer(_: unknown, finalDiffableRule: DiffableRule) {
   return {
-    timestampOverride: defaultValue.timestamp_override.field_name,
-    timestampOverrideFallbackDisabled: defaultValue.timestamp_override.fallback_disabled ?? false,
+    timestampOverride: finalDiffableRule.timestamp_override?.field_name,
+    timestampOverrideFallbackDisabled:
+      finalDiffableRule.timestamp_override?.fallback_disabled ?? false,
   };
 }