Malcolm v24.12.0

[mmguero]

Malcolm v24.12.0 contains several improvements to the Malcolm configuration script, the Malcolm user interface, and the Malcolm API, as well as component version updates and bug fixes. This release also corresponds with the release of the malcolm-test (cisagov#486), a Malcolm systems testing framework.

v24.11.0...v24.12.0

• Features and enhancements
  • Creation of a Malcolm systems testing framework (automated testing cisagov/Malcolm#486)
  • Added a number of Zeek packages to detect various CVEs
  • Improvements to the Indices, Ready, and Document Ingest Statistics APIs
  • Use new arkime tag-hiding feature to hide netbox tag from UI (use new arkime tag-hiding feature to hide netbox tag from UI cisagov/Malcolm#495)
  • Provide configuration script options for pulling from threat intel feeds (provide configuration options for pulling from threat intel feeds cisagov/Malcolm#532)
  • Prompt during configuration whether to enable capture statistics (prompt during configuration whether to enable capture statistics cisagov/Malcolm#504)
  • Add additional EVTX fields to index template (evtx fields that need to be added to index template cisagov/Malcolm#525) and minor improvements to normalization
  • Add simple readiness indicator to upload page (add simple readiness indicator to upload page cisagov/Malcolm#528)
  • Add option to upload page to disable NetBox enrichment for the currently-uploaded batch of PCAPs
  • Expose more of the Logstash API passthrough to the Malcolm API
• Component version updates
  • Arkime to v5.5.1
  • capa to v8.0.1
  • elasticearch-dsl Python library to v8.17.0
  • elasticsearch Python library to v8.17.0
  • Fluent Bit to v3.2.2
  • NetBox to v4.1.8 (major update from the v4.0.x series, see bring netbox up-to-date with the current released version cisagov/Malcolm#496)
  • opensearch-py Python library to v2.8.0
  • yq to v4.44.6
  • Zeek to v7.0.5 (security and bugfix release)
• Bug fixes
  • Zeek DNS records don't open correctly in Arkime sessions (Zeek DNS records don't open correctly in Arkime sessions cisagov/Malcolm#509)
    • Fixes to some Zeek dns.log parsing conflicts between ECS's DNS fields and what the Arkime schema is expecting
  • Mandiant threat intel source doesn't get split correctly when using JSON zeek log format (Mandiant threat intel source doesn't get split correctly when using JSON zeek log format cisagov/Malcolm#494)
  • Set indices.query.bool.max_clause_count to 8192 to reflect maximum number of fields
  • Increase Java stack size (-Xss) for Logstash from 1536k to 2048k
  • Minor fixes for parsing Zeek intel.log (some fields not named correctly with Zeek JSON-formatted logs)
  • Fixed setting the Signature event severity tags
• Code and project maintenance
  • Replaced hard-coded Malcolm version number in documentation markdown files with variable-based replacer populated during generation
  • Documentation and screenshot updates