Failing test: Detection Engine API Integration Tests - Serverless - Rule Execution Logic.x-pack/test/security_solution_api_integration/test_suites/detections_response/default_license/rule_execution_logic/execution_logic/machine_learning·ts - Rule execution logic API Execution logic @ess @serverless Machine learning type rules "before all" hook for "should create 1 alert from ML rule when record meets anomaly_threshold"

[kibanamachine]

A test failed on a tracked branch

AggregateError: 
    Error: Bulk doc failure [operation=index]:
      doc: {"actual":[1],"bucket_span":900,"by_field_name":"process.name","by_field_value":"store","detector_index":0,"function":"rare","function_description":"rare","host.name":["mothra"],"influencers":[{"influencer_field_name":"user.name","influencer_field_values":["root"]},{"influencer_field_name":"process.name","influencer_field_values":["store"]},{"influencer_field_name":"host.name","influencer_field_values":["mothra"]}],"initial_record_score":33.36147565024334,"is_interim":false,"job_id":"v3_linux_anomalous_network_activity","multi_bucket_impact":0,"probability":0.007820139656036713,"process.name":["store"],"record_score":33.36147565024334,"result_type":"record","timestamp":1605567488000,"typical":[0.007820139656036711],"user.name":["root"]}
      error: {"type":"document_parsing_exception","reason":"[1:177] failed to parse field [host] of type [keyword] in document with id 'v3_linux_anomalous_network_activity_record_1586274300000_900_0_-96106189301704594950079884115725560577_5'. Preview of field's value: '{name=[mothra]}'","caused_by":{"type":"illegal_state_exception","reason":"Can't get text on a START_OBJECT at 1:156"}}
        at Array.map (<anonymous>)
        at indexDocs (index_doc_records_stream.ts:62:13)
        at processTicksAndRejections (node:internal/process/task_queues:95:5)
        at Writable.write [as _write] (index_doc_records_stream.ts:76:9)
    at indexDocs (index_doc_records_stream.ts:62:13)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Writable.write [as _write] (index_doc_records_stream.ts:76:9)

First failure: CI Build - main

[kibanamachine]

New failure: CI Build - main

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[kibanamachine]

New failure: CI Build - main

[mistic]

Skipped.

main: 27d2fb9

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[yctercero]

Confirmed passing locally - flake not bug.

[rylnd]

The error in these builds is pretty useful: it looks like the es_archiver call to insert our anomaly records failed, because host.name: ['something'] was causing a conflict with the existing mappings on insert (it appears that host was mapped as a keyword directly):

fail: Rule execution logic API Execution logic @ess @serverless Machine learning type rules "before all" hook for "should create 1 alert from ML rule when record meets anomaly_threshold"
 AggregateError: 
Error: Bulk doc failure [operation=index]:
  doc: {"actual":[1],"bucket_span":900,"by_field_name":"process.name","by_field_value":"store","detector_index":0,"function":"rare","function_description":"rare","host.name":["mothra"],"influencers":[{"influencer_field_name":"user.name","influencer_field_values":["root"]},{"influencer_field_name":"process.name","influencer_field_values":["store"]},{"influencer_field_name":"host.name","influencer_field_values":["mothra"]}],"initial_record_score":33.36147565024334,"is_interim":false,"job_id":"v3_linux_anomalous_network_activity","multi_bucket_impact":0,"probability":0.007820139656036713,"process.name":["store"],"record_score":33.36147565024334,"result_type":"record","timestamp":1605567488000,"typical":[0.007820139656036711],"user.name":["root"]}
 error: {"type":"document_parsing_exception","reason":"[1:177] failed to parse field [host] of type [keyword] in document with id 'v3_linux_anomalous_network_activity_record_1586274300000_900_0_-96106189301704594950079884115725560577_5'. Preview of field's value: '{name=[mothra]}'","caused_by":{"type":"illegal_state_exception","reason":"Can't get text on a START_OBJECT at 1:156"}}

I didn't see any errors leading up to that failure indicating that the archive mappings (which explicitly include host.name) were not applied, nor does the data have any errors, so I'm not exactly sure what went wrong, but the consequence was the above.

[rylnd]

FYI I'm continuing to investigate these failures on this PR

[rylnd]

Closed by #188155.