Malcolm v3.0.0 release development

[mmguero]

List of changes in Malcolm v3.0.0:

• Change base for Elasticsearch and Kibana Docker images (version 7.6.2) from Elastic.co to Open Distro for Elastic (based on Elastic 7.10.0); see migrate to open distro for elasticsearch idaholab/Malcolm#15. This is a major change which breaks backwards compatibility for several features (listed below). If you are using these features, you will need to back up the data and/or configuration associated with them and migrate them manually to the new tools. No automatic migration or upgrade of these features is performed. It's recommended that you re-run install.py --configure (see System configuration and tuning) prior to running Malcolm v3.0.0.

  • Kibana comments replaced with Notebooks
  • Kibana elastalert plugin replaced with Alerting plugin
  • Elasticsearch curator replaced with Index Management plugin
  • The third-party [Sankey visualization plugin](https://github.com/mmguero-dev/kbn_sankey_vis] has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see visualization doesn't work in Kibana 7.9.x uniberg/kbn_sankey_vis#15)
  • The third-party Kibana drill-down plugin providing Kibana-to-Moloch pivoting has been temporarily removed due to compatibility issues, although it is planned to be reintegrated in a Malcolm point release in the near future (see plugin does not work with the latest version of Kibana goodlabs-studio/kibana-plugin-drilldownmenu#5)
  • In addition to those replacements, the Real Time Anomaly Detection feature is now available:
    • Real Time Anomaly Detection in Open Distro for Elasticsearch blog announcement
    • Anomaly Detection documentation and source code for Elasticsearch and Kibana components
    • Random Cut Forests writeup

• Malcolm startup time (especially the Logstash container) has been reduced drastically

• Improvements to Malcolm's prebuilt Kibana dashboards

• Improvements to build scripts

• Minor tweaks and bugfixes for ISO-installed environments for Malcolm and Hedgehog Linux

• Minor other bug fixes and performance improvements

• Version bump

  • Yara v4.0.5

[lgtm-com]

This pull request introduces 6 alerts and fixes 1 when merging e37eea8 into 83f0627 - view on LGTM.com

new alerts:

• 3 for Unused local variable
• 2 for Variable defined multiple times
• 1 for Unused import

fixed alerts:

• 1 for Variable defined multiple times