Alpha2 (#4)

* bump django version 2.2.8 to 2.2.9 * uncomment mysql requirement * asset and cve history in elasticsearch * vulnerability history * asset integration tests with elasticsearch * tox ci env * elasticsearch travis install package * --force-confnew deb elasticsearch * cwe history * cwe and exploits documents tests * code coverage increase * elastic tests for knowledge_base * added ut cverage, moved port from assets to vulnerability * vulnerability elastic integration test * codacy coverage * fix travis typo * codecov fix * codecov branch * travis ci branch env * travis branch * removed init data * ralph asset factory fix * ralph asset factory added CIA * removed not needed db objects * skip nessus tests * update cve when cwe changed * vulnerability history * nessus fix * include_in_parent * fix ralph factory * update cve only when new exploits appears * update licence * update licence * update readme * update docs link
DSecureMe · Jan 26, 2020 · 29a1ea1 · 29a1ea1
1 parent 7931792
commit 29a1ea1
Show file tree

Hide file tree

Showing 65 changed files with 1,239 additions and 1,652 deletions.
diff --git a/.travis.yml b/.travis.yml
@@ -2,7 +2,18 @@ language: python
 python:
   - "3.6"
   - "3.7"
-install: pip install tox-travis codecov
-script: tox
-after_success:
-  - codecov --token="$CODECOV_TOKEN"
+before_install:
+  - curl -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.3.1-amd64.deb
+  - sudo dpkg -i --force-confnew elasticsearch-7.3.1-amd64.deb
+  - sudo sed -i.old 's/-Xms1g/-Xms128m/' /etc/elasticsearch/jvm.options
+  - sudo sed -i.old 's/-Xmx1g/-Xmx128m/' /etc/elasticsearch/jvm.options
+  - echo -e '-XX:+DisableExplicitGC\n-Djdk.io.permissionsUseCanonicalPath=true\n-Dlog4j.skipJansi=true\n-server\n' | sudo tee -a /etc/elasticsearch/jvm.options
+  - sudo chown -R elasticsearch:elasticsearch /etc/default/elasticsearch
+  - sudo service elasticsearch start
+  - sudo service elasticsearch status
+install:
+  - pip install tox-travis
+before_script:
+  - sleep 30
+script:
+  - tox -e travis
diff --git a/LICENSE.md b/LICENSE.md
@@ -186,7 +186,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright [yyyy] [name of copyright owner]
+   Copyright 2020 DSecure.me
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

diff --git a/README.md b/README.md
@@ -9,23 +9,23 @@
 Thanks to VMC you can focus on the most important assets from environment perspective given information about vulnerabilities and assets.
 
 VMC can present important information in brief and compact way
-![screen3](https://raw.githubusercontent.com/DSecureMe/vmc/master/images/56517124_1081640865372415_2823128148789428224_n.png)
+![screen3](https://raw.githubusercontent.com/DSecureMe/vmc/master/images/kpi_1.png)
 
-![screen4](https://raw.githubusercontent.com/DSecureMe/vmc/master/images/58590369_362012701104738_5283209682779897856_n.png)
+![screen4](https://raw.githubusercontent.com/DSecureMe/vmc/master/images/kpi_2.png)
 
 ![screen5](https://raw.githubusercontent.com/DSecureMe/vmc/master/images/67130855_394264208112259_8200620034528116736_n.png)
 
 VMC also shows live updates regarding you environemnt. It guides your focus to specific threats and assets.
 
-![screen1](https://raw.githubusercontent.com/DSecureMe/vmc/master/images/54525102_322316375302297_4724612730252689408_n.png)
+![screen1](https://raw.githubusercontent.com/DSecureMe/vmc/master/images/hive.png)
 
-![screen2](https://raw.githubusercontent.com/DSecureMe/vmc/master/images/54525980_511427642719427_3614827467690213376_n.png)
+![screen2](https://raw.githubusercontent.com/DSecureMe/vmc/master/images/alerts.png)
 
 You can install VMC either on your machine or use elegant [docker solution](https://github.com/DSecureMe/vmc-docker).
 
 # Details
 ## Documentation
-We’ve made separate repository with guides and documentation available [here](https://github.com/DSecureMe/vmcDocs).
+We’ve made separate repository with guides and documentation available [here](https://github.com/DSecureMe/vmc-docs).
 ## Architecture
 In simple words VMC requires data about detections from your scanner, information about assets and updates about CVE. Thanks to that VMC can live update your focus to the most emerging threats for your assets.
 ## License

diff --git a/VERSION.txt b/VERSION.txt
@@ -1 +1 @@
-1.0.1-alpha
+1.0.1-alpha2
diff --git a/images/54525102_322316375302297_4724612730252689408_n.png b/images/54525102_322316375302297_4724612730252689408_n.png
diff --git a/images/54525980_511427642719427_3614827467690213376_n.png b/images/54525980_511427642719427_3614827467690213376_n.png
diff --git a/images/56517124_1081640865372415_2823128148789428224_n.png b/images/56517124_1081640865372415_2823128148789428224_n.png
diff --git a/images/58590369_362012701104738_5283209682779897856_n.png b/images/58590369_362012701104738_5283209682779897856_n.png
diff --git a/images/alerts.png b/images/alerts.png
diff --git a/images/elastalert.png b/images/elastalert.png
diff --git a/images/hive.png b/images/hive.png
diff --git a/images/kpi_1.png b/images/kpi_1.png
diff --git a/images/kpi_2.png b/images/kpi_2.png
diff --git a/requirements.txt b/requirements.txt
@@ -1,16 +1,16 @@
-django==2.2.8
+django==2.2.9
 psycopg2-binary==2.7.7
 celery==4.3.0
 requests==2.21.0
 beautifulsoup4==4.7.1
-django-elasticsearch-dsl==7.0.0
+django-elasticsearch-dsl==7.1.0
 flower==0.9.3
-django_celery_beat==1.4.0
+django_celery_beat==1.5.0
 django-redis==4.10.0
 lxml==4.3.3
 defusedxml==0.6.0
 django-celery-results==1.0.4
 django-simple-history==2.7.3
 pyyaml==5.1.2
-#mysqlclient==1.4.4
-gunicorn==19.9.0
+mysqlclient==1.4.4
+gunicorn==20.0.4
diff --git a/src/vmc/assets/documents.py b/src/vmc/assets/documents.py
@@ -17,40 +17,39 @@
  * under the License.
  *
 """
+from decimal import Decimal
 
-from django_elasticsearch_dsl.registries import registry
-from django_elasticsearch_dsl import Document, fields
+from elasticsearch_dsl import Date, Keyword, InnerDoc
+from vmc.common.enum import TupleValueEnum
 
-from vmc.assets.models import Asset
+from vmc.common.elastic.documents import Document, TupleValueField
+from vmc.common.elastic.registers import registry
 
 
-@registry.register_document
-class AssetDocument(Document):
-    ip_address = fields.KeywordField()
-    os = fields.KeywordField()
-    confidentiality_requirement = fields.KeywordField()
-    integrity_requirement = fields.KeywordField()
-    availability_requirement = fields.KeywordField()
-    business_owner = fields.KeywordField()
-    technical_owner = fields.KeywordField()
-    hostname = fields.KeywordField()
-    created_date = fields.DateField()
-    modified_date = fields.DateField()
+class Impact(TupleValueEnum):
+    LOW = ('L', Decimal('0.5'))
+    MEDIUM = ('M', Decimal('1.0'))
+    HIGH = ('H', Decimal('1.51'))
+    NOT_DEFINED = ('N', Decimal('1.0'))
 
-    class Index:
-        name = 'asset'
 
-    class Django:
-        model = Asset
+class AssetInnerDoc(InnerDoc):
+    ip_address = Keyword()
+    os = Keyword()
+    cmdb_id = Keyword()
+    confidentiality_requirement = TupleValueField(choice_type=Impact)
+    integrity_requirement = TupleValueField(choice_type=Impact)
+    availability_requirement = TupleValueField(choice_type=Impact)
+    business_owner = Keyword()
+    technical_owner = Keyword()
+    hostname = Keyword()
+    created_date = Date()
+    modified_date = Date()
+    change_reason = Keyword()
 
-    @staticmethod
-    def prepare_confidentiality_requirement(instance) -> str:
-        return instance.get_confidentiality_requirement_display()
 
-    @staticmethod
-    def prepare_integrity_requirement(instance) -> str:
-        return instance.get_integrity_requirement_display()
+@registry.register_document
+class AssetDocument(AssetInnerDoc, Document):
+    class Index:
+        name = 'asset'
 
-    @staticmethod
-    def prepare_availability_requirement(instance) -> str:
-        return instance.get_availability_requirement_display()
diff --git a/src/vmc/assets/fixtures/assets.json b/src/vmc/assets/fixtures/assets.json
diff --git a/src/vmc/assets/migrations/0001_initial.py b/src/vmc/assets/migrations/0001_initial.py
diff --git a/src/vmc/assets/models.py b/src/vmc/assets/models.py