diff --git a/README.md b/README.md index fb4a574d1..f8dde1e6a 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ Malcolm is a powerful network traffic analysis tool suite designed with the following goals in mind: -* **Easy to use** – Malcolm accepts network traffic data in the form of full packet capture (PCAP) files and Zeek (formerly Bro) logs. These artifacts can be uploaded via a simple browser-based interface or captured live and forwarded to Malcolm using lightweight forwarders. In either case, the data is automatically normalized, enriched, and correlated for analysis. +* **Easy to use** – Malcolm accepts network traffic data in the form of full packet capture (PCAP) files and Zeek logs. These artifacts can be uploaded via a simple browser-based interface or captured live and forwarded to Malcolm using lightweight forwarders. In either case, the data is automatically normalized, enriched, and correlated for analysis. * **Powerful traffic analysis** – Visibility into network communications is provided through two intuitive interfaces: OpenSearch Dashboard, a flexible data visualization plugin with dozens of prebuilt dashboards providing an at-a-glance overview of network protocols; and Arkime (formerly Moloch), a powerful tool for finding and identifying the network sessions comprising suspected security incidents. * **Streamlined deployment** – Malcolm operates as a cluster of Docker containers – isolated sandboxes that each serve a dedicated function of the system. This Docker-based deployment model, combined with a few simple scripts for setup and run-time management, makes Malcolm suitable to be deployed quickly across a variety of platforms and use cases; whether it be for long-term deployment on a Linux server in a security operations center (SOC) or for incident response on a Macbook for an individual engagement. * **Secure communications** – All communications with Malcolm, both from the user interface and from remote log forwarders, are secured with industry standard encryption protocols. diff --git a/docs/arkime.md b/docs/arkime.md index 41d9fdc35..557a33193 100644 --- a/docs/arkime.md +++ b/docs/arkime.md @@ -17,7 +17,7 @@ The Arkime interface will be accessible over HTTPS on port 443 at the docker hos ## Zeek log integration -A stock installation of Arkime extracts all its network connection ("session") metadata ("SPI" or "Session Profile Information") from full packet capture artifacts (PCAP files). Zeek (formerly Bro) generates similar session metadata, linking network events to sessions via a connection UID. Malcolm aims to facilitate analysis of Zeek logs by mapping values from Zeek logs to the Arkime session database schema for equivalent fields, and by creating new "native" Arkime database fields for all other Zeek log values for which there is not currently an equivalent in Arkime: +A stock installation of Arkime extracts all its network connection ("session") metadata ("SPI" or "Session Profile Information") from full packet capture artifacts (PCAP files). Zeek generates similar session metadata, linking network events to sessions via a connection UID. Malcolm aims to facilitate analysis of Zeek logs by mapping values from Zeek logs to the Arkime session database schema for equivalent fields, and by creating new "native" Arkime database fields for all other Zeek log values for which there is not currently an equivalent in Arkime: ![Zeek log session record](./images/screenshots/arkime_session_zeek.png) diff --git a/docs/malcolm-hedgehog-e2e-iso-install.md b/docs/malcolm-hedgehog-e2e-iso-install.md index 1c0b2a7a9..7e675bee0 100644 --- a/docs/malcolm-hedgehog-e2e-iso-install.md +++ b/docs/malcolm-hedgehog-e2e-iso-install.md @@ -463,7 +463,7 @@ To specify which files should be extracted, specify the Zeek file carving mode: If unsure what mode to choose, both **mapped (except common plain text files)** (to carve and scan almost all files) and **interesting** (to only carve and scan files with [mime types of common attack vectors]({{ site.github.repository_url }}/blob/{{ site.github.build_revision }}/hedgehog-iso/interface/sensor_ctl/zeek/extractor_override.interesting.zeek)) are probably good choices. -Next, specify which carved files to preserve (saved on the sensor under `/capture/bro/capture/extract_files/quarantine` by default). In order to not consume all the sensor's available storage space, the oldest preserved files will be pruned along with the oldest Zeek logs as described below with **AUTOSTART_PRUNE_ZEEK** in the [autostart services](#HedgehogConfigAutostart) section. +Next, specify which carved files to preserve (saved on the sensor under `/capture/zeek/capture/extract_files/quarantine` by default). In order to not consume all the sensor's available storage space, the oldest preserved files will be pruned along with the oldest Zeek logs as described below with **AUTOSTART_PRUNE_ZEEK** in the [autostart services](#HedgehogConfigAutostart) section. Users will prompted to specify which engine(s) to use to analyze extracted files. Extracted files can be examined through any of three methods: diff --git a/hedgehog-iso/interface/sensor_ctl/control_vars.conf b/hedgehog-iso/interface/sensor_ctl/control_vars.conf index a3b2c76ac..66a9ec266 100644 --- a/hedgehog-iso/interface/sensor_ctl/control_vars.conf +++ b/hedgehog-iso/interface/sensor_ctl/control_vars.conf @@ -32,7 +32,7 @@ export FLUENTBIT_METRICS_INTERVAL=30 export FLUENTBIT_THERMAL_INTERVAL=10 export FLUENTBIT_AIDE_INTERVAL=86400 -export ZEEK_LOG_PATH=/home/sensor/bro_logs +export ZEEK_LOG_PATH=/home/sensor/zeek_logs export ZEEK_MAX_DISK_FILL=90 export ZEEK_PRUNE_CHECK_SECONDS=90 diff --git a/hedgehog-iso/interface/sensor_ctl/filebeat/filebeat.yml b/hedgehog-iso/interface/sensor_ctl/filebeat/filebeat.yml index 7d122e151..adb9ae734 100644 --- a/hedgehog-iso/interface/sensor_ctl/filebeat/filebeat.yml +++ b/hedgehog-iso/interface/sensor_ctl/filebeat/filebeat.yml @@ -5,8 +5,8 @@ logging.metrics.enabled: false filebeat.inputs: - type: log paths: - - ${BEAT_LOG_PATTERN:/home/sensor/bro_logs/*.log} - - ${BEAT_STATIC_LOG_PATTERN:/home/sensor/bro_logs/static/*.log} + - ${BEAT_LOG_PATTERN:/home/sensor/zeek_logs/*.log} + - ${BEAT_STATIC_LOG_PATTERN:/home/sensor/zeek_logs/static/*.log} symlinks: true fields_under_root: true tags: ["_filebeat_zeek_hedgehog_live"] @@ -24,7 +24,7 @@ filebeat.inputs: - type: log paths: - - ${BEAT_SURICATA_LOG_PATTERN:/home/sensor/bro_logs/suricata/eve*.json} + - ${BEAT_SURICATA_LOG_PATTERN:/home/sensor/zeek_logs/suricata/eve*.json} symlinks: true fields_under_root: true tags: ["_filebeat_suricata_hedgehog_live"] diff --git a/hedgehog-iso/interface/sensor_ctl/filebeat/sensor_filebeat_local.sh b/hedgehog-iso/interface/sensor_ctl/filebeat/sensor_filebeat_local.sh index 212a1d1ad..7a6da4307 100755 --- a/hedgehog-iso/interface/sensor_ctl/filebeat/sensor_filebeat_local.sh +++ b/hedgehog-iso/interface/sensor_ctl/filebeat/sensor_filebeat_local.sh @@ -3,10 +3,10 @@ # Copyright (c) 2024 Battelle Energy Alliance, LLC. All rights reserved. if [[ -z "$ZEEK_CAPTURE_PATH" ]]; then - ZEEK_CAPTURE_PATH="$HOME/bro_logs" + ZEEK_CAPTURE_PATH="$HOME/zeek_logs" fi if [[ -z "$SURICATA_CAPTURE_PATH" ]]; then - SURICATA_CAPTURE_PATH="$HOME/bro_logs/suricata" + SURICATA_CAPTURE_PATH="$HOME/zeek_logs/suricata" fi export ZEEK_CAPTURE_PATH export SURICATA_CAPTURE_PATH diff --git a/hedgehog-iso/interface/sensor_interface/static/js/custom.js b/hedgehog-iso/interface/sensor_interface/static/js/custom.js index 9e7373347..347e16927 100644 --- a/hedgehog-iso/interface/sensor_interface/static/js/custom.js +++ b/hedgehog-iso/interface/sensor_interface/static/js/custom.js @@ -28,7 +28,7 @@ function stop_all() { } -function start_bro() { +function start_zeek() { var xhttp = new XMLHttpRequest(); loadingBar('on'); xhttp.onreadystatechange = function () { @@ -41,7 +41,7 @@ function start_bro() { xhttp.send(); } -function stop_bro() { +function stop_zeek() { var xhttp = new XMLHttpRequest(); loadingBar('on'); xhttp.onreadystatechange = function () { diff --git a/hedgehog-iso/interface/sensor_interface/templates/buttons.html b/hedgehog-iso/interface/sensor_interface/templates/buttons.html index 84eae9cdb..652e67dfe 100644 --- a/hedgehog-iso/interface/sensor_interface/templates/buttons.html +++ b/hedgehog-iso/interface/sensor_interface/templates/buttons.html @@ -55,7 +55,7 @@