Skip to content

Commit

Permalink
Merge pull request #275 from oasis-open/acs-conversion
Browse files Browse the repository at this point in the history 
acs conversion
  • Loading branch information
@rpiazza
rpiazza authored Oct 8, 2021
2 parents 85ed052 + 08dc11f commit 3bc461a
Show file tree
Hide file tree
Showing 45 changed files with 3,652 additions and 60 deletions.
12 changes: 12 additions & 0 deletions CHANGELOG
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,18 @@
CHANGELOG
=========

4.1.0 - 2021-10-8

This release optionally handles ACS data markings (see install doc)

* Other changes

- added --acs oiption for ACS data markings
- Fix hash_constant processing to handle defaults
- Handle literals in missing policy code
- Added time properties to incident extension
- Handle malware aliases

4.0.2 - 2021-09-10

* Changes
Expand Down
5 changes: 5 additions & 0 deletions docs/command-line.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ STIX 1.x content to STIX 2.x content:
[--missing-policy {use-extensions,use-custom-properties,add-to-description,ignore}]
[--custom-property-prefix CUSTOM_PROPERTY_PREFIX]
[--infrastructure]
[--acs]
[--incidents]
[--package-created-by-id PACKAGE_CREATED_BY_ID]
[--default-timestamp DEFAULT_TIMESTAMP]
Expand Down Expand Up @@ -58,6 +59,10 @@ optional arguments:
Incidents will be included in the conversion.
Default for version 2.1 is true.
--acs
Process ACS data markings
Default is false.
--package-created-by-id PACKAGE_CREATED_BY_ID
Use provided identifier for "created_by_ref"
properties.
Expand Down
4 changes: 3 additions & 1 deletion docs/conf.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@
# import sys
# sys.path.insert(0, os.path.abspath('.'))

# Standard Library
import datetime

# -- General configuration ------------------------------------------------

Expand Down Expand Up @@ -50,7 +52,7 @@

# General information about the project.
project = 'stix2-elevator'
copyright = '2017, OASIS Open'
copyright = '{}, OASIS Open'.format(datetime.date.today().year)
author = 'OASIS Open'

# The version info for the project you're documenting, acts as replacement for
Expand Down
29 changes: 29 additions & 0 deletions docs/install.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,32 @@ You can also install the stix2-elevator from GitHub to get the latest (unstable)
.. code-block:: bash
$ pip install git+https://github.com/oasis-open/cti-stix-elevator.git
Installation Steps for ACS Data Marking Support
-----------------------------------------------

ACS data markings correspond to the common marking scheme used by the U.S. government (e.g., U, C, S, TS).
To elevate STIX 1.x content that contains ACS data markings, it is necessary to install an additional python package
called 'stix_edh'.

Install with pip

.. code-block:: bash
$ pip install stix2-elevator[acs]
Installation Steps for Ignoring Data Markings Not Defined in the STIX Specification
-----------------------------------------------------------------------------------

The elevator uses the -m option to declare data marking python classes that support data markings not defined within the
STIX specification. See the Command Line Interface section for an example.

However, the elevator must import those class definitions. The suggested way is to create a small python wrapper script
that imports the needed package.

.. code-block:: python
import <data marking package>
from stix2elevator import elevate
elevate(...)
6 changes: 6 additions & 0 deletions docs/warnings.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ custom_property_prefix is provided, but the missing policy is not 'use-custom-pr
*type* option was not given, but it defaults to true for version 2.1" 214 info
Custom properties/objects/extensions are deprecated in version 2.1. Suggest using 'use-extensions' instead 215 info
The missing policy option of 'use-extensions' cannot be used with version 2.0. 'use-custom-properies' is suggested 216 error
ACS data markings cannot be supported in version 2.0. --acs option is ignored. 217 warn
================================================================================================================== ==== =====


Expand All @@ -54,6 +55,8 @@ Property *property_name* of *id* is ignored, because it can't be represented in
New extension-definition id *id* was generated for *type*. *id* 315 warn
Custom Content *property_name* of *id* is ignored 316 warn
Used *object_path* for extension property for *property_name* 317 warn
Token in control set not recognized: *token* 318 warn
Used extensions for ACS data markings. See *id* 319 warn
============================================================================================================================== ==== =====


Expand Down Expand Up @@ -98,6 +101,7 @@ The confidence value *value* cannot be converted
Location with free text address in *id* not handled yet 433 warn
Observed Data objects cannot refer to other external objects: *property name* in *type*" 434 warn
CIQ Address information in *id* is not representable in 2.0 435 warn
ACS data markings only supported when --acs option is used. See *id* 436 warn
============================================================================================================================================== ==== =====

Multiple values are not supported in STIX 2.x
Expand Down Expand Up @@ -165,6 +169,8 @@ Address direction in *id* is not provided, using 'src'
cisa-proprietary is only permitted when ais-consent is everyone, so it has been dropped. See *id* 637 warn
Indicator *id* does not contain the information necessary to generate a pattern 638 warn
This observable *id* already is associated with cyber observables 639 warn
Unable to determine the hash type for *hash value* 640 warn
Required property *property* is not provided for ACS data marking 641 warn
=========================================================================================================================================== ==== =====

STIX Elevator conversion based on assumptions
Expand Down
100 changes: 100 additions & 0 deletions idioms-json-2.0-custom/taxii-regression-test-11-not-all.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
{
"id": "bundle--8ba8d091-2a49-48bc-86e4-bced167fc9df",
"objects": [
{
"created": "9999-12-31T00:00:00.000Z",
"description": "This is the description of a CISA defensive measure",
"id": "course-of-action--709db951-3ba8-4691-9357-d4cd7b088b82",
"modified": "9999-12-31T00:00:00.000Z",
"name": "This is the title of the CISA defensive measure",
"type": "course-of-action"
},
{
"created": "9999-12-31T00:00:00.000Z",
"description": "This is the description of a new Vulnerability.",
"external_references": [
{
"external_id": "CVE-1234-9999",
"source_name": "cve"
},
{
"external_id": "1",
"source_name": "osvdb"
}
],
"id": "vulnerability--1176c2f1-ed68-4dc5-8a01-4bf48a350f64",
"modified": "9999-12-31T00:00:00.000Z",
"name": "Title of a vulnerability containing an ID that needs to be sanitized. Regression_Test-et-11. And now another one that doesn't exist elsewhere isa:guide.19001.123.456 and one with a different namespace Another-01",
"type": "vulnerability"
},
{
"created": "9999-12-31T00:00:00.000Z",
"external_references": [
{
"external_id": "CVE-1234-9999",
"source_name": "cve"
},
{
"external_id": "1",
"source_name": "osvdb"
}
],
"id": "vulnerability--2da4bfd6-94ca-41e1-8245-7fd0d74adfd9",
"modified": "9999-12-31T00:00:00.000Z",
"name": "Doesn't have one",
"type": "vulnerability"
},
{
"count": 1,
"created": "9999-12-31T00:00:00.000Z",
"id": "sighting--5f8e6b75-de1e-4f23-aa37-507ed33bb91b",
"modified": "9999-12-31T00:00:00.000Z",
"sighting_of_ref": "indicator--5c7f3088-9ec9-49d5-8df1-04fe828ccfc1",
"summary": false,
"type": "sighting"
},
{
"created": "9999-12-31T00:00:00.000Z",
"description": "This is a sample indicator description",
"id": "indicator--4a91a2ed-fc24-4113-9a28-fecd37d3c81c",
"labels": [
"ip-watchlist"
],
"modified": "9999-12-31T00:00:00.000Z",
"pattern": "[ipv4-addr:value = '1.1.10.1'] AND [ipv4-addr:value = '1.1.1.1']",
"type": "indicator",
"valid_from": "9999-12-31T00:00:00.000000Z",
"valid_until": "9999-12-31T00:01:00.000000Z",
"x_elevator_confidence": "Medium"
},
{
"created": "9999-12-31T00:00:00.000Z",
"id": "relationship--c6c3ac54-c343-4f6e-b9e1-31793b417ab9",
"modified": "9999-12-31T00:00:00.000Z",
"relationship_type": "mitigates",
"source_ref": "course-of-action--709db951-3ba8-4691-9357-d4cd7b088b82",
"target_ref": "vulnerability--1176c2f1-ed68-4dc5-8a01-4bf48a350f64",
"type": "relationship"
},
{
"created": "9999-12-31T00:00:00.000Z",
"id": "relationship--2e62b247-bbc0-43c8-9be7-750dffa42391",
"modified": "9999-12-31T00:00:00.000Z",
"relationship_type": "mitigates",
"source_ref": "course-of-action--709db951-3ba8-4691-9357-d4cd7b088b82",
"target_ref": "vulnerability--2da4bfd6-94ca-41e1-8245-7fd0d74adfd9",
"type": "relationship"
},
{
"created": "9999-12-31T00:00:00.000Z",
"id": "relationship--bf13f0c4-2666-4a56-96d1-e6e44cdd11e4",
"modified": "9999-12-31T00:00:00.000Z",
"relationship_type": "investigates",
"source_ref": "course-of-action--709db951-3ba8-4691-9357-d4cd7b088b82",
"target_ref": "indicator--5c7f3088-9ec9-49d5-8df1-04fe828ccfc1",
"type": "relationship"
}
],
"spec_version": "2.0",
"type": "bundle"
}
100 changes: 100 additions & 0 deletions idioms-json-2.0-custom/taxii-regression-test-11.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
{
"id": "bundle--8ba8d091-2a49-48bc-86e4-bced167fc9df",
"objects": [
{
"created": "9999-12-31T00:00:00.000Z",
"description": "This is the description of a CISA defensive measure",
"id": "course-of-action--709db951-3ba8-4691-9357-d4cd7b088b82",
"modified": "9999-12-31T00:00:00.000Z",
"name": "This is the title of the CISA defensive measure",
"type": "course-of-action"
},
{
"created": "9999-12-31T00:00:00.000Z",
"description": "This is the description of a new Vulnerability.",
"external_references": [
{
"external_id": "CVE-1234-9999",
"source_name": "cve"
},
{
"external_id": "1",
"source_name": "osvdb"
}
],
"id": "vulnerability--1176c2f1-ed68-4dc5-8a01-4bf48a350f64",
"modified": "9999-12-31T00:00:00.000Z",
"name": "Title of a vulnerability containing an ID that needs to be sanitized. Regression_Test-et-11. And now another one that doesn't exist elsewhere isa:guide.19001.123.456 and one with a different namespace Another-01",
"type": "vulnerability"
},
{
"created": "9999-12-31T00:00:00.000Z",
"external_references": [
{
"external_id": "CVE-1234-9999",
"source_name": "cve"
},
{
"external_id": "1",
"source_name": "osvdb"
}
],
"id": "vulnerability--2da4bfd6-94ca-41e1-8245-7fd0d74adfd9",
"modified": "9999-12-31T00:00:00.000Z",
"name": "Doesn't have one",
"type": "vulnerability"
},
{
"count": 1,
"created": "9999-12-31T00:00:00.000Z",
"id": "sighting--5f8e6b75-de1e-4f23-aa37-507ed33bb91b",
"modified": "9999-12-31T00:00:00.000Z",
"sighting_of_ref": "indicator--5c7f3088-9ec9-49d5-8df1-04fe828ccfc1",
"summary": false,
"type": "sighting"
},
{
"created": "9999-12-31T00:00:00.000Z",
"description": "This is a sample indicator description",
"id": "indicator--4a91a2ed-fc24-4113-9a28-fecd37d3c81c",
"labels": [
"ip-watchlist"
],
"modified": "9999-12-31T00:00:00.000Z",
"pattern": "[ipv4-addr:value = '1.1.10.1'] AND [ipv4-addr:value = '1.1.1.1']",
"type": "indicator",
"valid_from": "9999-12-31T00:00:00.000000Z",
"valid_until": "9999-12-31T00:01:00.000000Z",
"x_elevator_confidence": "Medium"
},
{
"created": "9999-12-31T00:00:00.000Z",
"id": "relationship--c6c3ac54-c343-4f6e-b9e1-31793b417ab9",
"modified": "9999-12-31T00:00:00.000Z",
"relationship_type": "mitigates",
"source_ref": "course-of-action--709db951-3ba8-4691-9357-d4cd7b088b82",
"target_ref": "vulnerability--1176c2f1-ed68-4dc5-8a01-4bf48a350f64",
"type": "relationship"
},
{
"created": "9999-12-31T00:00:00.000Z",
"id": "relationship--2e62b247-bbc0-43c8-9be7-750dffa42391",
"modified": "9999-12-31T00:00:00.000Z",
"relationship_type": "mitigates",
"source_ref": "course-of-action--709db951-3ba8-4691-9357-d4cd7b088b82",
"target_ref": "vulnerability--2da4bfd6-94ca-41e1-8245-7fd0d74adfd9",
"type": "relationship"
},
{
"created": "9999-12-31T00:00:00.000Z",
"id": "relationship--bf13f0c4-2666-4a56-96d1-e6e44cdd11e4",
"modified": "9999-12-31T00:00:00.000Z",
"relationship_type": "investigates",
"source_ref": "course-of-action--709db951-3ba8-4691-9357-d4cd7b088b82",
"target_ref": "indicator--5c7f3088-9ec9-49d5-8df1-04fe828ccfc1",
"type": "relationship"
}
],
"spec_version": "2.0",
"type": "bundle"
}
30 changes: 30 additions & 0 deletions idioms-json-2.0-custom/taxii-regression-test-13.json
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"id": "bundle--97aa3df0-a291-4fc5-bb9b-c042d81af64e",
"objects": [
{
"count": 1,
"created": "9999-12-31T00:00:00.000Z",
"id": "sighting--b4ae2b11-5030-45be-a9a7-e41d6f1c4187",
"modified": "9999-12-31T00:00:00.000Z",
"sighting_of_ref": "indicator--bceaed2d-a467-47a4-8844-45d5316798df",
"summary": false,
"type": "sighting"
},
{
"created": "9999-12-31T00:00:00.000Z",
"description": "This is a sample indicator description",
"id": "indicator--b950f4f0-286f-42a3-8980-5e0d711bfb74",
"labels": [
"ip-watchlist"
],
"modified": "9999-12-31T00:00:00.000Z",
"pattern": "[ipv4-addr:value = '1.1.10.12'] AND [ipv4-addr:value = '1.1.1.12']",
"type": "indicator",
"valid_from": "9999-12-31T00:00:00.000000Z",
"valid_until": "9999-12-31T00:01:00.000000Z",
"x_elevator_confidence": "Medium"
}
],
"spec_version": "2.0",
"type": "bundle"
}
Loading

0 comments on commit 3bc461a

Please sign in to comment.