Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project plan for anomaly detection features #43

Merged
merged 3 commits into from
Jul 25, 2022
Merged

Project plan for anomaly detection features #43

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7bcde6c
Project plan for anomaly detection features
peternied Jul 8, 2022
c7a5a47
Apply suggestions from code review
peternied Jul 12, 2022
0e0ac34
Remaining PR feedback
peternied Jul 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 58 additions & 2 deletions SECURITY.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,25 @@
- [Extensions Security Guidelines](#extensions-security-guidelines)
- [Document Phases](#document-phases)
- [Terms](#terms)
- [Areas](#areas)
- [Host security](#host-security)
- [Communications security (COMSEC)](#communications-security-comsec)
- [Data Security](#data-security)
- [Access Control](#access-control)
- [Auditing](#auditing)
- [Installation](#installation)
- [Versioning](#versioning)
- [Configuration](#configuration)
- [Reliability](#reliability)
- [Projects](#projects)
- [Anomaly Detection Plugin](#anomaly-detection-plugin)
- [User identity OpenSearch#3846 :negative_squared_cross_mark:](#user-identity-opensearch3846-negative_squared_cross_mark)
- [Aquiring User objects sdk#37](#aquiring-user-objects-sdk37)
- [Resource user/role checks sdk#40](#resource-userrole-checks-sdk40)
- [Delegate Authority OpenSearch#3850 :negative_squared_cross_mark:](#delegate-authority--opensearch3850-negative_squared_cross_mark)
- [Extension identity sdk#41](#extension-identity-sdk41)
- [Delayed action API sdk#42](#delayed-action-api-sdk42)

# Extensions Security Guidelines

OpenSearch's support for extensions allows for taking already powerful use cases and expanding on them. With this increased functionality comes a larger surface area for misuse, vulnerabilities, and malicious interactions.
Expand Down Expand Up @@ -30,10 +52,12 @@ Extensions are sandboxed from the host system by operating via APIs. This secur

## Communications security (COMSEC)

Plugins are loaded into the same java virtual machine instance allowing communicate to OpenSearch through in process java APIs. Plugins can issue API requests to the OpenSearch hosts reusing the standard node-to-node communications, internally called the transport client.
Plugins are loaded into the same java virtual machine instance allowing communicate to OpenSearch through in-process java APIs. Plugins can issue API requests to the OpenSearch hosts reusing the standard node-to-node communications, internally called the transport client.
peternied marked this conversation as resolved.
Show resolved Hide resolved

Extensions of OpenSearch communicate via https requests between the nodes on the cluster and the extensions endpoint(s). This is a bi-direction communication also allows extensions to contact the OpenSearch cluster through its available APIs.
peternied marked this conversation as resolved.
Show resolved Hide resolved

* :warning: The communication protocal has not been locked-in, following up with [Extensions to OpenSearch communication #34](https://github.com/opensearch-project/opensearch-sdk/issues/34).
peternied marked this conversation as resolved.
Show resolved Hide resolved

## Data Security

OpenSearch stores data in memory and local file system storage. This data is stored unencrypted.
Expand All @@ -46,8 +70,13 @@ With the security plugin installed, role based access control (RBAC) is availabl

For resource that are managed by plugins, access control is governed within individual plugin. By examining [user](https://github.com/opensearch-project/common-utils/blob/main/src/main/java/org/opensearch/commons/authuser/User.java) object from OpenSearch's thread context permissions are available for approval/denial. An example from anomaly detection is [checkUserPermissions](https://github.com/opensearch-project/anomaly-detection/blob/875b03c1c7596cb34d74fea285c28d949cfb0d19/src/main/java/org/opensearch/ad/util/ParseUtils.java#L568). Uniform resource controls and models are needed to protect from misconfiguration and code defects.

* :building_construction: Adding a uniform resource permission check is being worked on in [sdk#40](https://github.com/opensearch-project/opensearch-sdk/issues/40).


As Extensions do not have access OpenSearch's thread context, identity and its associated privileges must be communicated through APIs.

* :building_construction: User identity is being worked on in [sdk#37](https://github.com/opensearch-project/opensearch-sdk/issues/37).

## Auditing

With the security plugin installed, when actions are performed on the OpenSearch cluster they are recorded if filtering criteria are meet to configurable audit log sinks.
Expand Down Expand Up @@ -76,4 +105,31 @@ Extensions configuration setup is tied to OpenSearch settings, extensions config

## Reliability

OpenSearch plugins can create cluster or node instability if incorrectly configured or by software defects.
OpenSearch plugins can create cluster or node instability if incorrectly configured or by software defects.

# Projects
To stretch out the design process while fulfilling scenarios some security efforts will be tracked as longer running projects. There will be tracking issues in github tied to the work, documentation here is justification for a project and how it ties into the security space. Tasks will be denotated as incomplete with :negative_squared_cross_mark: `:negative_squared_cross_mark:` or completed with :white_check_mark:
`:white_check_mark:`.

## Anomaly Detection Plugin
Overall project is tracked with [[FEATURE] Migrate Anomaly Detector plugin to work as an Extension](https://github.com/opensearch-project/opensearch-sdk/issues/24). By migrating this plugin it will exercise the general extensions and security specific scenarios.
peternied marked this conversation as resolved.
Show resolved Hide resolved

Additional background avaliable from [Security#1895](https://github.com/opensearch-project/security/issues/1895)

### User identity [OpenSearch#3846](https://github.com/opensearch-project/OpenSearch/issues/3846) :negative_squared_cross_mark:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AD exposes a bunch of actions.
For example: cluster:admin/opensearch/ad/detector/delete[1] is a Transport Action API exposed by AD.
How do we plan to support authorization of user for extensions?

[1] https://github.com/opensearch-project/anomaly-detection/blob/4baf75ca46da02c4e70016d34982f2c5a34d0acd/src/main/java/org/opensearch/ad/transport/DeleteAnomalyDetectorAction.java#L20

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the existing security plugin in place, no additional action is needed as these are already filtered. The assumption is that the security plugins hooks will still execute before the extensions hooks are triggered so the API calls can be intercepted as is already implemented.

If/When we rewrite the permissions evaluation system I think part of the backwards compatibility story will need to account for scenarios like this where no/minimal action is needed by plugin authors.

Replace [commons.authuser.User](https://github.com/opensearch-project/common-utils/blob/main/src/main/java/org/opensearch/commons/authuser/User.java) the common identity object used by plugins. The new object should be obtainable by the extension through other means than [InjectSecurity](https://github.com/opensearch-project/common-utils/blob/main/src/main/java/org/opensearch/commons/InjectSecurity.java#L69) depends on the thread context.
peternied marked this conversation as resolved.
Show resolved Hide resolved

#### Aquiring User objects [sdk#37](https://github.com/opensearch-project/opensearch-sdk/issues/37)
When OpenSearch sents a request to an extension the identiy should be avaliable, more discussion see [Handling identity in extensions](https://github.com/opensearch-project/opensearch-sdk/issues/14).
peternied marked this conversation as resolved.
Show resolved Hide resolved

#### Resource user/role checks [sdk#40](https://github.com/opensearch-project/opensearch-sdk/issues/40)
anomaly Detection has detectors that analyzer data and store its results so it can be inspected or alerted on, [more details](https://opensearch.org/docs/latest/monitoring-plugins/ad/index/). OpenSearch should be responsible for inspecting the user, roles, resources to ensure standard practices are used. A permissions check API should be designed and implemented to offload this work from extensions creators.
peternied marked this conversation as resolved.
Show resolved Hide resolved

### Delegate Authority [OpenSearch#3850](https://github.com/opensearch-project/OpenSearch/issues/3850) :negative_squared_cross_mark:
anomaly Detection runs background jobs to scan for anamolies and alerts that trigger if conditions are detected. Background tasks should be tied to an idenity and a delegated identity so permissions can be verified. The underlying systems depends on the [Job Scheduler](https://github.com/opensearch-project/job-scheduler/blob/main/src/main/java/org/opensearch/jobscheduler/scheduler/JobScheduler.java) plugin to execute these requests.

#### Extension identity [sdk#41](https://github.com/opensearch-project/opensearch-sdk/issues/41)
There should be different levels of permissions granularity interactive allowing for disgushing a user actions or user action through an extension. Extensions should have an identity and there should be a way that the identity of action is layered with all the parties that have triggered it.

#### Delayed action API [sdk#42](https://github.com/opensearch-project/opensearch-sdk/issues/42)
When actions are triggered without an interactive user session OpenSearch will need to permit the action to occur or not. Create an API for these background tasks to get an identity associated with the session.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably want to mention Principal identities here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about moving this detail into the associated issue? Its a great requirement that should be part of use cases and acceptance criteria.