Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

migrate to open distro for elasticsearch #15

Closed
mmguero opened this issue Sep 9, 2020 · 2 comments
Closed

migrate to open distro for elasticsearch #15

mmguero opened this issue Sep 9, 2020 · 2 comments
Labels
docker Relating to docker and docker-compose as used by Malcolm enhancement New feature or request opensearch Relating to Malcolm's use of OpenSearch

Comments

@mmguero
Copy link
Collaborator

mmguero commented Sep 9, 2020

From Malcolm created by mmguero: cisagov#70

https://opendistro.github.io/for-elasticsearch/

This looks like it might be a good choice for the default ES images used for Malcolm

@mmguero mmguero added docker Relating to docker and docker-compose as used by Malcolm enhancement New feature or request opensearch Relating to Malcolm's use of OpenSearch labels Sep 9, 2020
@mmguero
Copy link
Collaborator Author

mmguero commented Sep 9, 2020

This could potentially cover what ElastAlert does as well. OpenDistro comes with free Alerting plugin for Kibana!

@mmguero mmguero mentioned this issue Nov 11, 2020
16 tasks
@mmguero mmguero changed the title investigate open distro for elasticsearch migrate to open distro for elasticsearch Nov 11, 2020
@mmguero
Copy link
Collaborator Author

mmguero commented Nov 11, 2020

Tracking updating elastic components from 7.6.2 to 7.9.x that open distro tracks.

The unstable/development branch these commits are being done on before integration into the main branch can be found at https://github.com/mmguero-dev/Malcolm/tree/topic/elastic_new

  • elasticsearch updated and runs
  • logstash updated and runs (starts up faster actually)
  • filebeat (Malcolm) updated and runs
  • kibana updated and runs
  • kibana comments replaced with notebooks
  • kibana elastalert plugin replaced with alerting
  • curator replaced with index management
    • needs testing
  • kibana sankey plugin
  • kibana drilldown plugin
  • test arkime
  • protologbeat ("heatbeat") builds now, have not tested on Hedgehog yet
  • metricbeat tested on hedgehog
  • auditbeat tested on hedgehog
  • filebeat tested on hedgehog
  • filebeat (syslog) tested on hedgehog

mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Feb 5, 2021
mmguero added a commit to cisagov/Malcolm that referenced this issue Feb 16, 2021
mmguero added a commit to cisagov/Malcolm that referenced this issue Feb 16, 2021
* bring idaholab/Malcolm up-to-date with cisagov/Malcolm

* Restore malcolm.fyi web page generation

* ISO sha256sum placeholders

* working on idaholab#15, migrating to open distro for elastic

* bump yara to 4.0.5

* update SHA256 sums

* don't show comments from config file in settings appl7

* use mmguero-dev/zeek-community-id so it will build with docker 3.0.12

* somehow the extracted file http server stuff didn't make it into this branch

* expose HTTP file server port

* added filemon virtual_host

* added depends for nginx upstream

* set hostname to file-monitor for filemon

* Allow overriding cloned version of a plugin with a specific branch, and for corelight/bro-xor-exe-plugin use release 1.2 (pre commit corelight/zeek-xor-exe-plugin@a86f595) for Zeek 3.0.x and the latest release for Zeek 4.x

* use corelight/zeek-community-id build 3.0.0 for zeek 3.0.x

* put URLs in quotes for zeek clone

* temporarily remove the 'Top Actions and Results by Service' visualization (of type kbn_sankey) as it is not compatible with the current version of Kibana being used. tracking externally at uniberg/kbn_sankey_vis#15, at which point we can add it back in

* updates for open distro-based images for using non-1000 UIDs

* updates for open distro-based images for using non-1000 UIDs

* tweaks to iso for 3.0.0

* remove unnecessary package

* updated dashboards for new Kibana for Malcolm 3.0.0

* fix wording of prompt for email account inforrmation for alerts

* fix some dashboards where i lost a comma in the JSON

* updates to Arkime startup to try to fix race condition with fields getting created

* update auditbeat.yml rules to ignore firefox caching stuff (reduce noise)

* fix spacing in WISE field definition

* fixed a few dashboards (tag cloud visualizations didn't come over correctly)

* continuation of the idea for commit 045d7e6, for Logstash the STDIN doesn't work if we are dropping privs, which we don't need to do anyway. So differentiate between elastic and logstash for storing keystore values when dropping privileges

* minor tweaks to cloud visualizations

* updated local.zeek for sensor image to include ICSNPP parsers

* Revert "updated local.zeek for sensor image to include ICSNPP parsers"

This reverts commit dc19d83.

* updated local.zeek for sensor image to include ICSNPP parsers

* Updates to documentation and comments in preparation for 3.0.0 release
mmguero added a commit to cisagov/Malcolm that referenced this issue Feb 16, 2021
- Change base for Elasticsearch and Kibana Docker images (version 7.6.2) from Elastic.co to Open Distro for Elastic (based on Elastic 7.10.0); see idaholab#15. This is a major change which **breaks backwards compatibility** for several features (listed below). If you are using these features, you will need to back up the data and/or configuration associated with them and migrate them manually to the new tools. No automatic migration or upgrade of these features is performed. It's recommended that you re-run `install.py --configure` (see [System configuration and tuning](https://github.com/cisagov/Malcolm#ConfigAndTuning)) prior to running Malcolm v3.0.0.
  - Kibana [comments](https://github.com/gwintzer/kibana-comments-app-plugin) replaced with [Notebooks](https://opendistro.github.io/for-elasticsearch-docs/docs/notebooks/)
  - Kibana [elastalert](https://github.com/nsano-rururu/elastalert-kibana-plugin) plugin replaced with [Alerting](https://opendistro.github.io/for-elasticsearch-docs/docs/alerting/) plugin
  - Elasticsearch [curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html) replaced with [Index Management](https://opendistro.github.io/for-elasticsearch-docs/docs/ism/) plugin
  - The third-party [Sankey visualization plugin](https://github.com/mmguero-dev/kbn_sankey_vis] has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see uniberg/kbn_sankey_vis#15)
  - The third-party [Kibana drill-down plugin](https://github.com/mmguero-dev/kibana-plugin-drilldownmenu/) providing Kibana-to-Moloch pivoting has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see goodlabs-studio/kibana-plugin-drilldownmenu#5)
  - In addition to those replacements, the Real Time Anomaly Detection feature is now available:
    - [Real Time Anomaly Detection in Open Distro for Elasticsearch](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/real-time-anomaly-detection-in-open-distro-for-elasticsearch/) blog announcement
     - [Anomaly Detection](https://opendistro.github.io/for-elasticsearch-docs/docs/ad/) documentation and source code for [Elasticsearch](https://github.com/opendistro-for-elasticsearch/anomaly-detection) and [Kibana](https://github.com/opendistro-for-elasticsearch/anomaly-detection-kibana-plugin) components
     - [Random Cut Forests writeup](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/random-cut-forests/)

- Malcolm startup time (especially the Logstash container) has been reduced drastically

- Improvements to Malcolm's prebuilt Kibana dashboards

- Improvements to build scripts

- Minor tweaks and bugfixes for ISO-installed environments for Malcolm and Hedgehog Linux

- Minor other bug fixes and performance improvements

- Version bump
  - Yara [v4.0.5](https://github.com/VirusTotal/yara/releases/tag/v4.0.5)
mmguero added a commit that referenced this issue Feb 16, 2021
- Change base for Elasticsearch and Kibana Docker images (version 7.6.2) from Elastic.co to Open Distro for Elastic (based on Elastic 7.10.0); see #15. This is a major change which **breaks backwards compatibility** for several features (listed below). If you are using these features, you will need to back up the data and/or configuration associated with them and migrate them manually to the new tools. No automatic migration or upgrade of these features is performed. It's recommended that you re-run `install.py --configure` (see [System configuration and tuning](https://github.com/cisagov/Malcolm#ConfigAndTuning)) prior to running Malcolm v3.0.0.
  - Kibana [comments](https://github.com/gwintzer/kibana-comments-app-plugin) replaced with [Notebooks](https://opendistro.github.io/for-elasticsearch-docs/docs/notebooks/)
  - Kibana [elastalert](https://github.com/nsano-rururu/elastalert-kibana-plugin) plugin replaced with [Alerting](https://opendistro.github.io/for-elasticsearch-docs/docs/alerting/) plugin
  - Elasticsearch [curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html) replaced with [Index Management](https://opendistro.github.io/for-elasticsearch-docs/docs/ism/) plugin
  - The third-party [Sankey visualization plugin](https://github.com/mmguero-dev/kbn_sankey_vis] has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see uniberg/kbn_sankey_vis#15)
  - The third-party [Kibana drill-down plugin](https://github.com/mmguero-dev/kibana-plugin-drilldownmenu/) providing Kibana-to-Moloch pivoting has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see goodlabs-studio/kibana-plugin-drilldownmenu#5)
  - In addition to those replacements, the Real Time Anomaly Detection feature is now available:
    - [Real Time Anomaly Detection in Open Distro for Elasticsearch](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/real-time-anomaly-detection-in-open-distro-for-elasticsearch/) blog announcement
     - [Anomaly Detection](https://opendistro.github.io/for-elasticsearch-docs/docs/ad/) documentation and source code for [Elasticsearch](https://github.com/opendistro-for-elasticsearch/anomaly-detection) and [Kibana](https://github.com/opendistro-for-elasticsearch/anomaly-detection-kibana-plugin) components
     - [Random Cut Forests writeup](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/random-cut-forests/)

- Malcolm startup time (especially the Logstash container) has been reduced drastically

- Improvements to Malcolm's prebuilt Kibana dashboards

- Improvements to build scripts

- Minor tweaks and bugfixes for ISO-installed environments for Malcolm and Hedgehog Linux

- Minor other bug fixes and performance improvements

- Version bump
  - Yara [v4.0.5](https://github.com/VirusTotal/yara/releases/tag/v4.0.5)
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Feb 16, 2021
- Change base for Elasticsearch and Kibana Docker images (version 7.6.2) from Elastic.co to Open Distro for Elastic (based on Elastic 7.10.0); see idaholab#15. This is a major change which **breaks backwards compatibility** for several features (listed below). If you are using these features, you will need to back up the data and/or configuration associated with them and migrate them manually to the new tools. No automatic migration or upgrade of these features is performed. It's recommended that you re-run `install.py --configure` (see [System configuration and tuning](https://github.com/cisagov/Malcolm#ConfigAndTuning)) prior to running Malcolm v3.0.0.
  - Kibana [comments](https://github.com/gwintzer/kibana-comments-app-plugin) replaced with [Notebooks](https://opendistro.github.io/for-elasticsearch-docs/docs/notebooks/)
  - Kibana [elastalert](https://github.com/nsano-rururu/elastalert-kibana-plugin) plugin replaced with [Alerting](https://opendistro.github.io/for-elasticsearch-docs/docs/alerting/) plugin
  - Elasticsearch [curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html) replaced with [Index Management](https://opendistro.github.io/for-elasticsearch-docs/docs/ism/) plugin
  - The third-party [Sankey visualization plugin](https://github.com/mmguero-dev/kbn_sankey_vis] has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see uniberg/kbn_sankey_vis#15)
  - The third-party [Kibana drill-down plugin](https://github.com/mmguero-dev/kibana-plugin-drilldownmenu/) providing Kibana-to-Moloch pivoting has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see goodlabs-studio/kibana-plugin-drilldownmenu#5)
  - In addition to those replacements, the Real Time Anomaly Detection feature is now available:
    - [Real Time Anomaly Detection in Open Distro for Elasticsearch](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/real-time-anomaly-detection-in-open-distro-for-elasticsearch/) blog announcement
     - [Anomaly Detection](https://opendistro.github.io/for-elasticsearch-docs/docs/ad/) documentation and source code for [Elasticsearch](https://github.com/opendistro-for-elasticsearch/anomaly-detection) and [Kibana](https://github.com/opendistro-for-elasticsearch/anomaly-detection-kibana-plugin) components
     - [Random Cut Forests writeup](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/random-cut-forests/)

- Malcolm startup time (especially the Logstash container) has been reduced drastically

- Improvements to Malcolm's prebuilt Kibana dashboards

- Improvements to build scripts

- Minor tweaks and bugfixes for ISO-installed environments for Malcolm and Hedgehog Linux

- Minor other bug fixes and performance improvements

- Version bump
  - Yara [v4.0.5](https://github.com/VirusTotal/yara/releases/tag/v4.0.5)
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Feb 16, 2021
- Change base for Elasticsearch and Kibana Docker images (version 7.6.2) from Elastic.co to Open Distro for Elastic (based on Elastic 7.10.0); see idaholab#15. This is a major change which **breaks backwards compatibility** for several features (listed below). If you are using these features, you will need to back up the data and/or configuration associated with them and migrate them manually to the new tools. No automatic migration or upgrade of these features is performed. It's recommended that you re-run `install.py --configure` (see [System configuration and tuning](https://github.com/cisagov/Malcolm#ConfigAndTuning)) prior to running Malcolm v3.0.0.
  - Kibana [comments](https://github.com/gwintzer/kibana-comments-app-plugin) replaced with [Notebooks](https://opendistro.github.io/for-elasticsearch-docs/docs/notebooks/)
  - Kibana [elastalert](https://github.com/nsano-rururu/elastalert-kibana-plugin) plugin replaced with [Alerting](https://opendistro.github.io/for-elasticsearch-docs/docs/alerting/) plugin
  - Elasticsearch [curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html) replaced with [Index Management](https://opendistro.github.io/for-elasticsearch-docs/docs/ism/) plugin
  - The third-party [Sankey visualization plugin](https://github.com/mmguero-dev/kbn_sankey_vis] has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see uniberg/kbn_sankey_vis#15)
  - The third-party [Kibana drill-down plugin](https://github.com/mmguero-dev/kibana-plugin-drilldownmenu/) providing Kibana-to-Moloch pivoting has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see goodlabs-studio/kibana-plugin-drilldownmenu#5)
  - In addition to those replacements, the Real Time Anomaly Detection feature is now available:
    - [Real Time Anomaly Detection in Open Distro for Elasticsearch](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/real-time-anomaly-detection-in-open-distro-for-elasticsearch/) blog announcement
     - [Anomaly Detection](https://opendistro.github.io/for-elasticsearch-docs/docs/ad/) documentation and source code for [Elasticsearch](https://github.com/opendistro-for-elasticsearch/anomaly-detection) and [Kibana](https://github.com/opendistro-for-elasticsearch/anomaly-detection-kibana-plugin) components
     - [Random Cut Forests writeup](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/random-cut-forests/)

- Malcolm startup time (especially the Logstash container) has been reduced drastically

- Improvements to Malcolm's prebuilt Kibana dashboards

- Improvements to build scripts

- Minor tweaks and bugfixes for ISO-installed environments for Malcolm and Hedgehog Linux

- Minor other bug fixes and performance improvements

- Version bump
  - Yara [v4.0.5](https://github.com/VirusTotal/yara/releases/tag/v4.0.5)
mmguero added a commit to mmguero-dev/Malcolm that referenced this issue Feb 16, 2021
- Change base for Elasticsearch and Kibana Docker images (version 7.6.2) from Elastic.co to Open Distro for Elastic (based on Elastic 7.10.0); see idaholab#15. This is a major change which **breaks backwards compatibility** for several features (listed below). If you are using these features, you will need to back up the data and/or configuration associated with them and migrate them manually to the new tools. No automatic migration or upgrade of these features is performed. It's recommended that you re-run `install.py --configure` (see [System configuration and tuning](https://github.com/cisagov/Malcolm#ConfigAndTuning)) prior to running Malcolm v3.0.0.
  - Kibana [comments](https://github.com/gwintzer/kibana-comments-app-plugin) replaced with [Notebooks](https://opendistro.github.io/for-elasticsearch-docs/docs/notebooks/)
  - Kibana [elastalert](https://github.com/nsano-rururu/elastalert-kibana-plugin) plugin replaced with [Alerting](https://opendistro.github.io/for-elasticsearch-docs/docs/alerting/) plugin
  - Elasticsearch [curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html) replaced with [Index Management](https://opendistro.github.io/for-elasticsearch-docs/docs/ism/) plugin
  - The third-party [Sankey visualization plugin](https://github.com/mmguero-dev/kbn_sankey_vis] has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see uniberg/kbn_sankey_vis#15)
  - The third-party [Kibana drill-down plugin](https://github.com/mmguero-dev/kibana-plugin-drilldownmenu/) providing Kibana-to-Moloch pivoting has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see goodlabs-studio/kibana-plugin-drilldownmenu#5)
  - In addition to those replacements, the Real Time Anomaly Detection feature is now available:
    - [Real Time Anomaly Detection in Open Distro for Elasticsearch](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/real-time-anomaly-detection-in-open-distro-for-elasticsearch/) blog announcement
     - [Anomaly Detection](https://opendistro.github.io/for-elasticsearch-docs/docs/ad/) documentation and source code for [Elasticsearch](https://github.com/opendistro-for-elasticsearch/anomaly-detection) and [Kibana](https://github.com/opendistro-for-elasticsearch/anomaly-detection-kibana-plugin) components
     - [Random Cut Forests writeup](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/random-cut-forests/)

- Malcolm startup time (especially the Logstash container) has been reduced drastically

- Improvements to Malcolm's prebuilt Kibana dashboards

- Improvements to build scripts

- Minor tweaks and bugfixes for ISO-installed environments for Malcolm and Hedgehog Linux

- Minor other bug fixes and performance improvements

- Version bump
  - Yara [v4.0.5](https://github.com/VirusTotal/yara/releases/tag/v4.0.5)
@mmguero mmguero closed this as completed Feb 18, 2021
mmguero added a commit that referenced this issue Mar 3, 2021
- Change base for Elasticsearch and Kibana Docker images (version 7.6.2) from Elastic.co to Open Distro for Elastic (based on Elastic 7.10.0); see #15. This is a major change which **breaks backwards compatibility** for several features (listed below). If you are using these features, you will need to back up the data and/or configuration associated with them and migrate them manually to the new tools. No automatic migration or upgrade of these features is performed. It's recommended that you re-run `install.py --configure` (see [System configuration and tuning](https://github.com/cisagov/Malcolm#ConfigAndTuning)) prior to running Malcolm v3.0.0.
  - Kibana [comments](https://github.com/gwintzer/kibana-comments-app-plugin) replaced with [Notebooks](https://opendistro.github.io/for-elasticsearch-docs/docs/notebooks/)
  - Kibana [elastalert](https://github.com/nsano-rururu/elastalert-kibana-plugin) plugin replaced with [Alerting](https://opendistro.github.io/for-elasticsearch-docs/docs/alerting/) plugin
  - Elasticsearch [curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html) replaced with [Index Management](https://opendistro.github.io/for-elasticsearch-docs/docs/ism/) plugin
  - The third-party [Sankey visualization plugin](https://github.com/mmguero-dev/kbn_sankey_vis] has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see uniberg/kbn_sankey_vis#15)
  - The third-party [Kibana drill-down plugin](https://github.com/mmguero-dev/kibana-plugin-drilldownmenu/) providing Kibana-to-Moloch pivoting has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see goodlabs-studio/kibana-plugin-drilldownmenu#5)
  - In addition to those replacements, the Real Time Anomaly Detection feature is now available:
    - [Real Time Anomaly Detection in Open Distro for Elasticsearch](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/real-time-anomaly-detection-in-open-distro-for-elasticsearch/) blog announcement
     - [Anomaly Detection](https://opendistro.github.io/for-elasticsearch-docs/docs/ad/) documentation and source code for [Elasticsearch](https://github.com/opendistro-for-elasticsearch/anomaly-detection) and [Kibana](https://github.com/opendistro-for-elasticsearch/anomaly-detection-kibana-plugin) components
     - [Random Cut Forests writeup](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/random-cut-forests/)

- Malcolm startup time (especially the Logstash container) has been reduced drastically

- Improvements to Malcolm's prebuilt Kibana dashboards

- Improvements to build scripts

- Minor tweaks and bugfixes for ISO-installed environments for Malcolm and Hedgehog Linux

- Minor other bug fixes and performance improvements

- Version bump
  - Yara [v4.0.5](https://github.com/VirusTotal/yara/releases/tag/v4.0.5)
mmguero added a commit that referenced this issue Mar 3, 2021
- Change base for Elasticsearch and Kibana Docker images (version 7.6.2) from Elastic.co to Open Distro for Elastic (based on Elastic 7.10.0); see #15. This is a major change which **breaks backwards compatibility** for several features (listed below). If you are using these features, you will need to back up the data and/or configuration associated with them and migrate them manually to the new tools. No automatic migration or upgrade of these features is performed. It's recommended that you re-run `install.py --configure` (see [System configuration and tuning](https://github.com/cisagov/Malcolm#ConfigAndTuning)) prior to running Malcolm v3.0.0.
  - Kibana [comments](https://github.com/gwintzer/kibana-comments-app-plugin) replaced with [Notebooks](https://opendistro.github.io/for-elasticsearch-docs/docs/notebooks/)
  - Kibana [elastalert](https://github.com/nsano-rururu/elastalert-kibana-plugin) plugin replaced with [Alerting](https://opendistro.github.io/for-elasticsearch-docs/docs/alerting/) plugin
  - Elasticsearch [curator](https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html) replaced with [Index Management](https://opendistro.github.io/for-elasticsearch-docs/docs/ism/) plugin
  - The third-party [Sankey visualization plugin](https://github.com/mmguero-dev/kbn_sankey_vis] has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see uniberg/kbn_sankey_vis#15)
  - The third-party [Kibana drill-down plugin](https://github.com/mmguero-dev/kibana-plugin-drilldownmenu/) providing Kibana-to-Moloch pivoting has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see goodlabs-studio/kibana-plugin-drilldownmenu#5)
  - In addition to those replacements, the Real Time Anomaly Detection feature is now available:
    - [Real Time Anomaly Detection in Open Distro for Elasticsearch](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/real-time-anomaly-detection-in-open-distro-for-elasticsearch/) blog announcement
     - [Anomaly Detection](https://opendistro.github.io/for-elasticsearch-docs/docs/ad/) documentation and source code for [Elasticsearch](https://github.com/opendistro-for-elasticsearch/anomaly-detection) and [Kibana](https://github.com/opendistro-for-elasticsearch/anomaly-detection-kibana-plugin) components
     - [Random Cut Forests writeup](https://opendistro.github.io/for-elasticsearch/blog/odfe-updates/2019/11/random-cut-forests/)

- Malcolm startup time (especially the Logstash container) has been reduced drastically

- Improvements to Malcolm's prebuilt Kibana dashboards

- Improvements to build scripts

- Minor tweaks and bugfixes for ISO-installed environments for Malcolm and Hedgehog Linux

- Minor other bug fixes and performance improvements

- Version bump
  - Yara [v4.0.5](https://github.com/VirusTotal/yara/releases/tag/v4.0.5)
@mmguero mmguero added this to Malcolm Oct 14, 2024
@mmguero mmguero moved this to Released in Malcolm Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docker Relating to docker and docker-compose as used by Malcolm enhancement New feature or request opensearch Relating to Malcolm's use of OpenSearch
Projects
Status: Released
Development

No branches or pull requests

1 participant