[Security Solution] Implement single-line string diff algorithm

[jpdjere]

Epics: https://github.com/elastic/security-team/issues/1974 (internal), #174168

Summary

Implement an algorithm for diffing and merging changes in single-line string type of fields of detection rules.

Context from the Rule Customization RFC:

• Concrete field diff algorithms by type
• Single-line string fields

To do

• Step 0
  • Make a proposal which fields should use this algorithm. Get feedback and approval from the team.
• Step 1 ([Security Solution] Adds unit tests for simple diff algorithm #184483)
  • Implement the algorithm and cover it with unit tests.
• Step 2 ([Security Solution] Integration tests for simple diff algorithm #184889)
  • Apply the algorithm to a few rule fields. It will become used in the upgrade/_review endpoint.
  • Write a test plan covering rule upgrade scenarios for the fields to which the algorithm has been applied.
    • [Security Solution] Simple diff algorithm test plan #184484
  • Add test coverage according to the test plan (integration tests).

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

[dplumlee]

Here are the proposed fields that would utilize this diff algorithm:

Common fields

• name
• severity (Enum treated as string field for diff logic purposes)
• license

Non-ML rule type fields

• type
• language
• query

EQL fields

• event_category_override
• timestamp_field
• tiebreaker_field

New terms fields

• history_window_start

[banderror]

@dplumlee Thanks, the proposal makes sense to me, except:

• We've already discussed that query fields grouped into a kql_query or eql_query diffable objects will require their own specialized diff algorithms. Internally, these algorithms can use the single-line string diff algo in certain branches of their logic.
• The rule type field might need its own, stricter diff algorithm, that would produce a non-solvable conflict if there's a type update from Elastic. We'll need dedicated UX for that case: [Security Solution] Implement UI for updating prebuilt rule to a new rule type (MVP) #180395

[banderror]

Resolved by:

• [Security Solution] Adds unit tests for simple diff algorithm #184483
• [Security Solution] Simple diff algorithm test plan #184484
• [Security Solution] Integration tests for simple diff algorithm #184889