Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Amazon Security Lake] - OCSF v1.1 update with major refactor & adding support for dynamic template and mappings & system tests #10405

Open
wants to merge 34 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
66c9372
added support for new user inventory info event class and updated inc…
ShourieG Jun 7, 2024
fb78670
trying to make a working system test
ShourieG Jun 13, 2024
3902a02
merged with upstream
ShourieG Jun 17, 2024
6bec44b
initial working system tests added pending elastic-package changes to…
ShourieG Jun 21, 2024
64f285b
merged with upstream/main
ShourieG Jul 2, 2024
118b2d2
test commit to be reverted
ShourieG Jul 10, 2024
185e2f9
initial working test for dynamic template
ShourieG Jul 12, 2024
f784e75
updated root org templates
ShourieG Jul 12, 2024
4282225
reworked 'org' object mapping as tynamic template for all data streams
ShourieG Jul 12, 2024
e2f8457
Merge branch 'main' into security_lake/ocsf_1.1
ShourieG Jul 23, 2024
d4788f4
Merge remote-tracking branch 'upstream/main' into security_lake/ocsf_1.1
ShourieG Jul 30, 2024
32ed102
segregated process fields in 'findings', added 'actor' fields for new…
ShourieG Jul 30, 2024
78c1ea2
added fulnerability findings support and segregated 'resource' group …
ShourieG Jul 30, 2024
0656284
Merge remote-tracking branch 'upstream/main' into security_lake/ocsf_1.1
ShourieG Jul 30, 2024
8f7122d
added ntp activity event class, deprecated proxy event class, aded pr…
ShourieG Aug 1, 2024
5352aac
added os patch state event class, segregated device fields across all…
ShourieG Aug 2, 2024
ac66e6e
added datastore activity event class, segregated actor, user & metada…
ShourieG Aug 6, 2024
73b7be8
added support for detection finding event class, segregated and mappe…
ShourieG Aug 6, 2024
1236584
added support of compliance finding event class, segregated and updat…
ShourieG Aug 7, 2024
03b5099
segregated and expanded api object across all data streams, added sup…
ShourieG Aug 7, 2024
e99119c
added support for Device Config State Change event class, updated sch…
ShourieG Aug 8, 2024
7e5f687
added support for scan activity event class
ShourieG Aug 8, 2024
516b63b
segregated file fields across required data streams, added support fo…
ShourieG Aug 8, 2024
bf779a5
added cwe & epss objects as flattened to cve object
ShourieG Aug 8, 2024
97459f5
converted feature object to follow dynamic mapping rules across all d…
ShourieG Aug 8, 2024
bb88d57
added firewall rule object to respective event categories
ShourieG Aug 8, 2024
f0fdc32
added some missing fields after locally running system tests for disc…
ShourieG Aug 9, 2024
0b356dc
reworked terrform deployer to support multi-bucket based system tests
ShourieG Aug 9, 2024
19ffbf7
updated docs and changelog
ShourieG Aug 9, 2024
dd90df2
fixed timestamp issues across all data streams, added all system test…
ShourieG Aug 13, 2024
360c3d8
resolved merge conflicts
ShourieG Aug 14, 2024
2b1250d
resolved merge conflicts
ShourieG Aug 19, 2024
2261431
removed system test configs until respective elastic-package changes …
ShourieG Aug 19, 2024
5794401
updated with main, resolved merge conflicts
ShourieG Aug 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -615,6 +615,9 @@
- name: group
type: group
fields:
- name: domain
type: keyword
description: The domain where the group is defined. For example, the LDAP or Active Directory domain.
- name: desc
type: keyword
description: The group description.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,39 @@
- name: category_uid
type: keyword
description: The category unique identifier of the event.3 Identity & Access ManagementIdentity & Access Management (IAM) events relate to the supervision of the system's authentication and access control model. Examples of such events are the success or failure of authentication, granting of authority, password change, entity change, privileged use etc.
- name: connection_info
type: group
fields:
- name: boundary
type: keyword
description: The boundary of the connection, normalized to the caption of 'boundary_id'. In the case of 'Other', it is defined by the event source.For cloud connections, this translates to the traffic-boundary(same VPC, through IGW, etc.). For traditional networks, this is described as Local, Internal, or External.
- name: boundary_id
type: keyword
description: The normalized identifier of the boundary of the connection. For cloud connections, this translates to the traffic-boundary (same VPC, through IGW, etc.). For traditional networks, this is described as Local, Internal, or External.
- name: direction
type: keyword
description: The direction of the initiated connection, traffic, or email, normalized to the caption of the direction_id value. In the case of 'Other', it is defined by the event source.
- name: direction_id
type: keyword
description: The normalized identifier of the direction of the initiated connection, traffic, or email.
- name: protocol_name
type: keyword
description: 'The TCP/IP protocol name in lowercase, as defined by the Internet Assigned Numbers Authority (IANA). See Protocol Numbers. For example: tcp or udp.'
- name: protocol_num
type: keyword
description: 'The TCP/IP protocol number, as defined by the Internet Assigned Numbers Authority (IANA). Use -1 if the protocol is not defined by IANA. See Protocol Numbers. For example: 6 for TCP and 17 for UDP.'
- name: protocol_ver
type: keyword
description: The Internet Protocol version.
- name: protocol_ver_id
type: keyword
description: The Internet Protocol version identifier.
- name: tcp_flags
type: long
description: The network connection TCP header flags (i.e., control bits).
- name: uid
type: keyword
description: The unique identifier of the connection.
- name: class_name
type: keyword
description: 'The event class name, as defined by class_uid value: Security Finding.'
Expand Down Expand Up @@ -244,6 +277,12 @@
- name: value
type: keyword
description: The value of the attribute to which the enriched data pertains.
- name: expiration_time
type: date
description: The share expiration time.
- name: expiration_time_dt
type: date
description: The share expiration time (date).
- name: http_request
type: group
fields:
Expand Down Expand Up @@ -334,33 +373,9 @@
- name: message
type: keyword
description: The description of the event, as defined by the event source.
- name: num_detections
type: integer
description: The number of detections.
- name: num_files
type: integer
description: The number of files scanned.
- name: num_folders
type: integer
description: The number of folders scanned.
- name: num_network_items
type: integer
description: The number of network items scanned.
- name: num_processes
type: integer
description: The number of processes scanned.
- name: num_registry_items
type: integer
description: The number of registry items scanned.
- name: num_resolutions
type: integer
description: The number of items that were resolved.
- name: num_skipped_items
type: integer
description: The number of items that were skipped.
- name: num_trusted_items
- name: num_*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work? til

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup this works

type: integer
description: The number of trusted items.
description: The number fields for counting various item scan results.
- name: observables
type: group
fields:
Expand Down
Loading