From ef21e06559296717d7cfbe43336af6be3f69b721 Mon Sep 17 00:00:00 2001 From: AnaisUrlichs Date: Mon, 26 Feb 2024 11:13:40 +0000 Subject: [PATCH 01/11] update docs contributing guidelines for custom checks Signed-off-by: AnaisUrlichs --- docs/community/contribute/checks/overview.md | 126 ++++++++++++++++++ .../contribute/checks/provider-support.md | 49 +++++++ .../contribute/checks/service-support.md | 49 +++++++ 3 files changed, 224 insertions(+) create mode 100644 docs/community/contribute/checks/overview.md create mode 100644 docs/community/contribute/checks/provider-support.md create mode 100644 docs/community/contribute/checks/service-support.md diff --git a/docs/community/contribute/checks/overview.md b/docs/community/contribute/checks/overview.md new file mode 100644 index 000000000000..8db8e068cdc5 --- /dev/null +++ b/docs/community/contribute/checks/overview.md @@ -0,0 +1,126 @@ +# Contribute Rego Checks + +The following guide provides an overview of contributing checks to the default checks in Trivy. + +All of the checks in Trivy can be found in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/tree/main) repository on GitHub. + +If anything is unclear, please [start a discussion](https://github.com/aquasecurity/trivy/discussions/new) and we'll do our best to help. + +## Check structure + +Checks are written in Rego and follow a particular structure in Trivy. Below is an example check for AWS: + +```rego +# METADATA +# title: "RDS IAM Database Authentication Disabled" +# description: "Ensure IAM Database Authentication is enabled for RDS database instances to manage database access" +# scope: package +# schemas: +# - input: schema["aws"] +# related_resources: +# - https://docs.aws.amazon.com/neptune/latest/userguide/iam-auth.html +# custom: +# id: AVD-AWS-0176 +# avd_id: AVD-AWS-0176 +# provider: aws +# service: rds +# severity: MEDIUM +# short_code: enable-iam-auth +# recommended_action: "Modify the PostgreSQL and MySQL type RDS instances to enable IAM database authentication." +# input: +# selector: +# - type: cloud +# subtypes: +# - service: rds +# provider: aws + +package builtin.aws.rds.aws0176 + +deny[res] { + instance := input.aws.rds.instances[_] + instance.engine.value == ["postgres", "mysql"][_] + not instance.iamauthenabled.value + res := result.new("Instance does not have IAM Authentication enabled", instance.iamauthenabled) +} +``` + +## Adding a new provider + +Every check references a provider. The list of providers are found in the [defsec](https://github.com/aquasecurity/defsec) repository. + +Before writing a new check, you need to verify if the provider your check targets is supported by Trivy. If it's not, you'll need to add support for it. In case the provider that you want to target exists, you need to check whether the service your policy will target is supported. + +### Add Support for a New Cloud Provider + +[Please reference the documentation on adding Support for a New Cloud Provider](./provider-support.md). + +### Add Support for a New Service in an existing Provider + +[Please reference the documentation on adding Support for a New Service](./service-support.md). + +### Service Properties + +Beefore you beging, you need to check if the properties that you want to target in a service are supported, and if not, add support for them. The guide on [Adding Support for a New Service](https://github.com/aquasecurity/defsec/blob/master/CONTRIBUTING.md#adding-support-for-a-new-service) covers adding new properties. + +## Create a new Check + +To create a new check, + +## Check Metadata + +The metadata is the top section that starts with `# METADATA`, and has to be placed on top of the check. You can copy and paste from another rule as a starting point. This format is effectively _yaml_ within a Rego comment, and is [defined as part of Rego itself](https://www.openpolicyagent.org/docs/latest/policy-language/#metadata). + +The metadata consists of the following fields: + +- `title` is a title for the rule. The title should clearly and succinctly state the problem which is being detected. +- `description` is a description of the problem which is being detected. The description should be a little more verbose than the title, and should describe what the rule is trying to achieve. Imagine it completing a sentence starting with `You should...`. +- `scope` is used to define the scope of the policy which is always `package`. In this case, we are defining a check that applies to the entire package. At the moment, Trivy only supports using package scope for metadata, so this should always be set to `package`. +- `schemas` tells Rego that it should use the `AWS` schema to validate the use of the input data in the policy. We currently support [the following shemas](https://github.com/aquasecurity/defsec/tree/9b3cc255faff5dc57de5ff77ed0ce0009c80a4bb/pkg/rego/schemas). Please choose a schema that is applicable to your check. Using a schema can help you validate your policy faster for syntax issues. +- `custom` is used to define custom fields that can be used by Trivy to provide additional context to the policy and any related detections. This can contain the following: + - `avd_id` can be used to link the checl to the AVD entry In the example check above, the `avd_id` `AVD-AWS-0176` is the ID of the checl in the [AWS Vulnerability Database](https://avd.aquasec.com/). If there is no AVD_ID available, you need to generate an ID to use for this field using `make id`. + - `provider` is the name of the [provider](https://github.com/aquasecurity/defsec/tree/master/pkg/providers) the check targets. This should be the same as the provider name in the `pkg/providers` directory, e.g. `aws`. + - `service` is the name of the service by the provider that the check targets. This should be the same as the service name in the `pkg/providers` directory, e.g. `rds`. + - `severity` is the severity of the check. This should be one of `LOW`, `MEDIUM`, `HIGH`, or `CRITICAL`. + - `short_code` is a short code for the check. This should be a short, descriptive name for the check, separating words with hyphens. You should omit provider/service from this. + - `recommended_action` is a recommended remediation action for the check. This should be a short, descriptive sentence describing what the user should do to resolve the issue. + - `input` tells _defsec_ what inputs this check should be applied to. Cloud provider checks should always use the `selector` input, and should always use the `type` selector with `cloud`. Check targeting Kubernetes yaml can use `kubenetes`, RBAC can use `rbac`, and so on. + - `subtypes` aid the engine to determine if it should load this policy or not for scanning. This can aid with the performance of scanning, especially if you have a lot of checks but not all apply to the IaC that you are trying to scan. + +### Generating an ID + +Running `make id` will provide you with the next available _ID_ for your rule. The ID is used in the Rego check to identify it. + + + + + + +At last, it's time to write your rule code! Rules are defined using _OPA Rego_. You can find a number of examples in the `checks/cloud` directory. The [OPA documentation](https://www.openpolicyagent.org/docs/latest/policy-language/) is a great place to start learning Rego. You can also check out the [Rego Playground](https://play.openpolicyagent.org/) to experiment with Rego, and [join the OPA Slack](https://slack.openpolicyagent.org/). + +Create a new file in `checks/cloud` with the name of your rule. You should nest it in the existing directory structure as applicable. The package name should be in the format `builtin.PROVIDER.SERVICE.ID`, e.g. `builtin.aws.rds.aws0176`. + + + + + + +Now you'll need to write the rule logic. This is the code that will be executed to detect the issue. You should define a rule named `deny` and place your code inside this. + +```rego +deny[res] { + instance := input.aws.rds.instances[_] + instance.engine.value == ["postgres", "mysql"][_] + not instance.iamauthenabled.value + res := result.new("Instance does not have IAM Authentication enabled", instance.iamauthenabled) +} +``` + +The rule should return a result, which can be created using `result.new` (this function does not need to be imported, it is defined internally and provided at runtime). The first argument is the message to display, and the second argument is the resource that the issue was detected on. + +In the example above, you'll notice properties are being accessed from the `input.aws` object. The full set of schemas containing all of these properties is [available here](https://github.com/aquasecurity/defsec/tree/master/pkg/rego/schemas). You can match the schema name to the type of input you want to scan. + +You should also write a test for your rule(s). There are many examples of these in the `checks/cloud` directory. + +Finally, you'll want to generate documentation for your newly added rule. Please run `make docs` to generate the documentation for your new policy and submit a PR for us to take a look at. + +You can see a full example PR for a new rule being added here: [https://github.com/aquasecurity/defsec/pull/1000](https://github.com/aquasecurity/defsec/pull/1000). diff --git a/docs/community/contribute/checks/provider-support.md b/docs/community/contribute/checks/provider-support.md new file mode 100644 index 000000000000..dfadcd94ffb2 --- /dev/null +++ b/docs/community/contribute/checks/provider-support.md @@ -0,0 +1,49 @@ +# Add Provider Support + +A provider refers to a cloud provier such as AWS or resource type such as Docker. + +Before adding a new provider, check whether the provider you would like to add does not already exist in the [trivy repository](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers). Next, create a [discussion](https://github.com/aquasecurity/trivy/discussions/categories/ideas) in the Trivy repository to get input on your plan of adding a new provider. + +We highly welcome new contributions! + +## Adding a new Provider + +First, add a new subdirectory to the [pkg/iac/providers](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) directory in the Trivy repository, named after your provider. Inside this, create a Go file with the same name as your provider and create a Go struct to hold information about all of the services supported by your provider. + +For example, adding support for a new provider called foo would look like this: + +pkg/iac/providers/foo/foo.go: + +``` +package foo + +type Foo struct { + // Add services here later... +} +``` + +Next, you should add a reference to your provider struct in [pkg/state/state.go](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/state/state.go): + +``` +type State struct { + // ... + Foo foo.Foo + // ... +} +``` + +Once done, add one or more adapters to [internal/adapters](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/adapters). An adapter takes an input and populates your provider struct. For example, if you want to scan a Terraform plan, you will need to add an adapter that takes the Terraform plan and populates your provider struct. The AWS provider support in Trivy uses multiple adapters - it can adapt CloudFormation, Terraform, and live AWS accounts. Each of these has an adapter in this directory. + +To support Terraform as an input, your adapter should look similar to the following: + +``` +func Adapt(modules terraform.Modules) (foo.Foo, error) { + return foo.Foo{ + // ... + }, nil +} +``` + +Additionally, it needs to be called in [iac/adapters/terraform/adapt.go](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/adapters/terraform/adapt.go) in the case of Terraform. + +We recommend browsing the existing adapters to see how they work, as there is a lot of common code that can be reused. \ No newline at end of file diff --git a/docs/community/contribute/checks/service-support.md b/docs/community/contribute/checks/service-support.md new file mode 100644 index 000000000000..e267593c221a --- /dev/null +++ b/docs/community/contribute/checks/service-support.md @@ -0,0 +1,49 @@ +# Add Service Support + +A service refers to a service by a cloud provider. This section details how to add a new service to an existing provider. All contributions need to be made to the [defsec repository](https://github.com/aquasecurity/defsec/). + +Before you begin, verify that the provider does not already have the service that you plan to add. + +Adding a new service involves two steps. The service will need a data structure to store information about the required resources, and then one or more adapters to convert input(s) into the aforementioned data structure. + +## Adding a new service to an existing provider + +To add a new service named `Bar` to a provider named `foo`, you'll need to add a new file at `pkg/providers/foo/bar/bar.go`: + +``` +type Bar struct { + // ... +} +``` + +Let's say the `Bar` service manages resources called `Baz`. You'll need to add a new struct to the `Bar` struct to hold information about this resource: + +``` +type Bar struct { + // ... + Baz []Baz + // ... +} + +type Baz struct { + types.Metadata + Name types.StringValue + Encrypted types.BoolValue +} +``` + +A `Baz` can have a name, and can optionally be encrypted. Instead of using raw string and bool types respectively, we use the defsec types `types.StringValue` and `types.BoolValue`. These types wrap the raw values and provide additional metadata about the value, such as whether it was set by the user or not, and the file and line number where the resource was defined. The `types.Metadata` struct is embedded in all of the defsec types, and provides a common set of metadata for all resources. This includes the file and line number where the resource was defined, and the name of the resource. + +Next you'll need to add a reference to your new service struct in the provider struct at `pkg/providers/foo/foo.go`: + +``` +type Foo struct { + // ... + Bar bar.Bar + // ... +} +``` + +Now you'll need to update all of the adapters which populate the Foo provider struct. For example, if you want to support Terraform, you'll need to update `internal/adapters/terraform/foo/bar/adapt.go`. + +Finally, make sure you run make schema to generate the schema for your new service. \ No newline at end of file From d7b8fb61fdebb2b2fce2c88ca08df4fae121c280 Mon Sep 17 00:00:00 2001 From: AnaisUrlichs Date: Mon, 26 Feb 2024 12:19:16 +0000 Subject: [PATCH 02/11] docs: update contributing custom checks Signed-off-by: AnaisUrlichs --- docs/community/contribute/checks/overview.md | 44 +++++++++---------- .../custom/contribute-checks.md | 9 ++++ mkdocs.yml | 5 +++ 3 files changed, 36 insertions(+), 22 deletions(-) create mode 100644 docs/docs/scanner/misconfiguration/custom/contribute-checks.md diff --git a/docs/community/contribute/checks/overview.md b/docs/community/contribute/checks/overview.md index 8db8e068cdc5..6ccfc20bb6d1 100644 --- a/docs/community/contribute/checks/overview.md +++ b/docs/community/contribute/checks/overview.md @@ -2,7 +2,11 @@ The following guide provides an overview of contributing checks to the default checks in Trivy. -All of the checks in Trivy can be found in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/tree/main) repository on GitHub. +All of the checks in Trivy can be found in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/tree/main) repository on GitHub. Before you beging writing a check, ensure: + +1. The check does not already exist as part of the default checks in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/tree/main) repository. +2. The pull requests in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/pulls) repository to see whether somone else is already contributing the check that you wanted to add. +3. The [issues in Trivy](https://github.com/aquasecurity/trivy/issues) to see whether any specific checks are missing in Trivy. If anything is unclear, please [start a discussion](https://github.com/aquasecurity/trivy/discussions/new) and we'll do our best to help. @@ -46,9 +50,9 @@ deny[res] { ## Adding a new provider -Every check references a provider. The list of providers are found in the [defsec](https://github.com/aquasecurity/defsec) repository. +Every check references a provider. The list of providers are found in the [trivy](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) repository. -Before writing a new check, you need to verify if the provider your check targets is supported by Trivy. If it's not, you'll need to add support for it. In case the provider that you want to target exists, you need to check whether the service your policy will target is supported. +Before writing a new check, you need to verify if the provider your check targets is supported by Trivy. If it's not, you'll need to add support for it. Additionally, if the provider that you want to target exists, you need to check whether the service your policy will target is supported. ### Add Support for a New Cloud Provider @@ -58,9 +62,7 @@ Before writing a new check, you need to verify if the provider your check target [Please reference the documentation on adding Support for a New Service](./service-support.md). -### Service Properties - -Beefore you beging, you need to check if the properties that you want to target in a service are supported, and if not, add support for them. The guide on [Adding Support for a New Service](https://github.com/aquasecurity/defsec/blob/master/CONTRIBUTING.md#adding-support-for-a-new-service) covers adding new properties. +This guide also showcases how to add new properties for an existing Service. ## Create a new Check @@ -68,16 +70,16 @@ To create a new check, ## Check Metadata -The metadata is the top section that starts with `# METADATA`, and has to be placed on top of the check. You can copy and paste from another rule as a starting point. This format is effectively _yaml_ within a Rego comment, and is [defined as part of Rego itself](https://www.openpolicyagent.org/docs/latest/policy-language/#metadata). +The metadata is the top section that starts with `# METADATA`, and has to be placed on top of the check. You can copy and paste from another check as a starting point. This format is effectively _yaml_ within a Rego comment, and is [defined as part of Rego itself](https://www.openpolicyagent.org/docs/latest/policy-language/#metadata). The metadata consists of the following fields: - `title` is a title for the rule. The title should clearly and succinctly state the problem which is being detected. - `description` is a description of the problem which is being detected. The description should be a little more verbose than the title, and should describe what the rule is trying to achieve. Imagine it completing a sentence starting with `You should...`. - `scope` is used to define the scope of the policy which is always `package`. In this case, we are defining a check that applies to the entire package. At the moment, Trivy only supports using package scope for metadata, so this should always be set to `package`. -- `schemas` tells Rego that it should use the `AWS` schema to validate the use of the input data in the policy. We currently support [the following shemas](https://github.com/aquasecurity/defsec/tree/9b3cc255faff5dc57de5ff77ed0ce0009c80a4bb/pkg/rego/schemas). Please choose a schema that is applicable to your check. Using a schema can help you validate your policy faster for syntax issues. +- `schemas` tells Rego that it should use the `AWS` schema to validate the use of the input data in the policy. We currently support [the following shemas](https://github.com/aquasecurity/defsec/tree/master/pkg/rego/schemas). Please choose a schema that is applicable to your check. Using a schema can help you validate your policy faster for syntax issues. - `custom` is used to define custom fields that can be used by Trivy to provide additional context to the policy and any related detections. This can contain the following: - - `avd_id` can be used to link the checl to the AVD entry In the example check above, the `avd_id` `AVD-AWS-0176` is the ID of the checl in the [AWS Vulnerability Database](https://avd.aquasec.com/). If there is no AVD_ID available, you need to generate an ID to use for this field using `make id`. + - `avd_id` can be used to link the check to the AVD entry. In the example check above, the `avd_id` `AVD-AWS-0176` is the ID of the check in the [AWS Vulnerability Database](https://avd.aquasec.com/). If there is no AVD_ID available, you need to generate an ID to use for this field using `make id`. - `provider` is the name of the [provider](https://github.com/aquasecurity/defsec/tree/master/pkg/providers) the check targets. This should be the same as the provider name in the `pkg/providers` directory, e.g. `aws`. - `service` is the name of the service by the provider that the check targets. This should be the same as the service name in the `pkg/providers` directory, e.g. `rds`. - `severity` is the severity of the check. This should be one of `LOW`, `MEDIUM`, `HIGH`, or `CRITICAL`. @@ -88,23 +90,17 @@ The metadata consists of the following fields: ### Generating an ID -Running `make id` will provide you with the next available _ID_ for your rule. The ID is used in the Rego check to identify it. - - - - +If you plan to contribue your check back into the [trivy-policies](https://github.com/aquasecurity/trivy-policies) repository, it will require a valid ID. +Running `make id` in the trivy-policies repository will provide you with the next available _ID_ for your rule. The ID is used in the Rego check to identify it. -At last, it's time to write your rule code! Rules are defined using _OPA Rego_. You can find a number of examples in the `checks/cloud` directory. The [OPA documentation](https://www.openpolicyagent.org/docs/latest/policy-language/) is a great place to start learning Rego. You can also check out the [Rego Playground](https://play.openpolicyagent.org/) to experiment with Rego, and [join the OPA Slack](https://slack.openpolicyagent.org/). +## Writing Rego Rules -Create a new file in `checks/cloud` with the name of your rule. You should nest it in the existing directory structure as applicable. The package name should be in the format `builtin.PROVIDER.SERVICE.ID`, e.g. `builtin.aws.rds.aws0176`. +At last, it's time to write your rule code! Rules are defined using _OPA Rego_. You can find a number of examples in the `checks/cloud` directory ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks/cloud)). The [OPA documentation](https://www.openpolicyagent.org/docs/latest/policy-language/) is a great place to start learning Rego. You can also check out the [Rego Playground](https://play.openpolicyagent.org/) to experiment with Rego, and [join the OPA Slack](https://slack.openpolicyagent.org/). +Create a new file in `checks/cloud` ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks/cloud)) with the name of your rule. You should nest it in the existing directory structure as applicable. The package name should be in the format `builtin.PROVIDER.SERVICE.ID`, e.g. `builtin.aws.rds.aws0176`. - - - - -Now you'll need to write the rule logic. This is the code that will be executed to detect the issue. You should define a rule named `deny` and place your code inside this. +Now you'll need to write the rule logic. This is the code that will be executed to detect the issue. You should define a rule named `deny` and place your code inside this. Every check in Trivy needs to have a `deny` rule. ```rego deny[res] { @@ -119,8 +115,12 @@ The rule should return a result, which can be created using `result.new` (this f In the example above, you'll notice properties are being accessed from the `input.aws` object. The full set of schemas containing all of these properties is [available here](https://github.com/aquasecurity/defsec/tree/master/pkg/rego/schemas). You can match the schema name to the type of input you want to scan. -You should also write a test for your rule(s). There are many examples of these in the `checks/cloud` directory. +You should also write a test for your rule(s). There are many examples of these in the `checks/cloud` directory ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks/cloud)). More information on how to write tests for Rego checks is provided in the [custom misconfiguration](../../../docs/scanner/misconfiguration/custom/testing.md) section of the docs. + +## Generate docs Finally, you'll want to generate documentation for your newly added rule. Please run `make docs` to generate the documentation for your new policy and submit a PR for us to take a look at. +## Example PR + You can see a full example PR for a new rule being added here: [https://github.com/aquasecurity/defsec/pull/1000](https://github.com/aquasecurity/defsec/pull/1000). diff --git a/docs/docs/scanner/misconfiguration/custom/contribute-checks.md b/docs/docs/scanner/misconfiguration/custom/contribute-checks.md new file mode 100644 index 000000000000..b39e9c767454 --- /dev/null +++ b/docs/docs/scanner/misconfiguration/custom/contribute-checks.md @@ -0,0 +1,9 @@ +## Contribute Rego Checks + +The contributing section provides detailed information on + +* how to contribute custom checks to the [trivy-policies repository](../../../../community/contribute/checks/overview.md/). +* add [new services](../../../../community/contribute/checks/service-support.md) to existing cloud providers +* add a [new provider](../../../../community/contribute/checks/provider-support.md/) + +This way, they become accessible as default [policies.](https://github.com/aquasecurity/trivy-policies) \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index f85fd7a8f209..76f072ebb6ee 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -65,6 +65,7 @@ nav: - Schemas: docs/scanner/misconfiguration/custom/schema.md - Testing: docs/scanner/misconfiguration/custom/testing.md - Debugging Policies: docs/scanner/misconfiguration/custom/debug.md + - Contribute Checks: docs/scanner/misconfiguration/custom/contribute-checks.md - Secret: docs/scanner/secret.md - License: docs/scanner/license.md - Coverage: @@ -184,6 +185,10 @@ nav: - Issues: community/contribute/issue.md - Discussions: community/contribute/discussion.md - Pull Requests: community/contribute/pr.md + - Contribute Rego Checks: + - Overview: community/contribute/checks/overview.md + - Add Provider Support: community/contribute/checks/provider-support.md + - Add Service Supprot: community/contribute/checks/service-support.md - Maintainer: - Help Wanted: community/maintainer/help-wanted.md - Triage: community/maintainer/triage.md From 2a8a9601c3499474e814443fe7a284956e638e82 Mon Sep 17 00:00:00 2001 From: AnaisUrlichs Date: Thu, 29 Feb 2024 13:21:43 +0000 Subject: [PATCH 03/11] docs: fix small errors in contributing checks docs Signed-off-by: AnaisUrlichs --- docs/community/contribute/checks/overview.md | 16 ++++++---------- .../contribute/checks/service-support.md | 4 ++-- .../misconfiguration/custom/contribute-checks.md | 2 +- 3 files changed, 9 insertions(+), 13 deletions(-) diff --git a/docs/community/contribute/checks/overview.md b/docs/community/contribute/checks/overview.md index 6ccfc20bb6d1..e43377abed97 100644 --- a/docs/community/contribute/checks/overview.md +++ b/docs/community/contribute/checks/overview.md @@ -5,7 +5,7 @@ The following guide provides an overview of contributing checks to the default c All of the checks in Trivy can be found in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/tree/main) repository on GitHub. Before you beging writing a check, ensure: 1. The check does not already exist as part of the default checks in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/tree/main) repository. -2. The pull requests in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/pulls) repository to see whether somone else is already contributing the check that you wanted to add. +2. The pull requests in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/pulls) repository to see whether someone else is already contributing the check that you wanted to add. 3. The [issues in Trivy](https://github.com/aquasecurity/trivy/issues) to see whether any specific checks are missing in Trivy. If anything is unclear, please [start a discussion](https://github.com/aquasecurity/trivy/discussions/new) and we'll do our best to help. @@ -48,7 +48,7 @@ deny[res] { } ``` -## Adding a new provider +## Adding a new provider or service Every check references a provider. The list of providers are found in the [trivy](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) repository. @@ -64,10 +64,6 @@ Before writing a new check, you need to verify if the provider your check target This guide also showcases how to add new properties for an existing Service. -## Create a new Check - -To create a new check, - ## Check Metadata The metadata is the top section that starts with `# METADATA`, and has to be placed on top of the check. You can copy and paste from another check as a starting point. This format is effectively _yaml_ within a Rego comment, and is [defined as part of Rego itself](https://www.openpolicyagent.org/docs/latest/policy-language/#metadata). @@ -76,8 +72,8 @@ The metadata consists of the following fields: - `title` is a title for the rule. The title should clearly and succinctly state the problem which is being detected. - `description` is a description of the problem which is being detected. The description should be a little more verbose than the title, and should describe what the rule is trying to achieve. Imagine it completing a sentence starting with `You should...`. -- `scope` is used to define the scope of the policy which is always `package`. In this case, we are defining a check that applies to the entire package. At the moment, Trivy only supports using package scope for metadata, so this should always be set to `package`. -- `schemas` tells Rego that it should use the `AWS` schema to validate the use of the input data in the policy. We currently support [the following shemas](https://github.com/aquasecurity/defsec/tree/master/pkg/rego/schemas). Please choose a schema that is applicable to your check. Using a schema can help you validate your policy faster for syntax issues. +- `scope` is used to define the scope of the policy. In this case, we are defining a check that applies to the entire package. At the moment, Trivy only supports using package scope for metadata, so this should always be set to `package`. +- `schemas` tells Rego that it should use the `AWS` schema to validate the use of the input data in the policy. We currently support [the following shemas](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas). Please choose a schema that is applicable to your check. Using a schema can help you validate your policy faster for syntax issues. - `custom` is used to define custom fields that can be used by Trivy to provide additional context to the policy and any related detections. This can contain the following: - `avd_id` can be used to link the check to the AVD entry. In the example check above, the `avd_id` `AVD-AWS-0176` is the ID of the check in the [AWS Vulnerability Database](https://avd.aquasec.com/). If there is no AVD_ID available, you need to generate an ID to use for this field using `make id`. - `provider` is the name of the [provider](https://github.com/aquasecurity/defsec/tree/master/pkg/providers) the check targets. This should be the same as the provider name in the `pkg/providers` directory, e.g. `aws`. @@ -85,7 +81,7 @@ The metadata consists of the following fields: - `severity` is the severity of the check. This should be one of `LOW`, `MEDIUM`, `HIGH`, or `CRITICAL`. - `short_code` is a short code for the check. This should be a short, descriptive name for the check, separating words with hyphens. You should omit provider/service from this. - `recommended_action` is a recommended remediation action for the check. This should be a short, descriptive sentence describing what the user should do to resolve the issue. - - `input` tells _defsec_ what inputs this check should be applied to. Cloud provider checks should always use the `selector` input, and should always use the `type` selector with `cloud`. Check targeting Kubernetes yaml can use `kubenetes`, RBAC can use `rbac`, and so on. + - `input` tells trivy what inputs this check should be applied to. Cloud provider checks should always use the `selector` input, and should always use the `type` selector with `cloud`. Check targeting Kubernetes yaml can use `kubenetes`, RBAC can use `rbac`, and so on. - `subtypes` aid the engine to determine if it should load this policy or not for scanning. This can aid with the performance of scanning, especially if you have a lot of checks but not all apply to the IaC that you are trying to scan. ### Generating an ID @@ -113,7 +109,7 @@ deny[res] { The rule should return a result, which can be created using `result.new` (this function does not need to be imported, it is defined internally and provided at runtime). The first argument is the message to display, and the second argument is the resource that the issue was detected on. -In the example above, you'll notice properties are being accessed from the `input.aws` object. The full set of schemas containing all of these properties is [available here](https://github.com/aquasecurity/defsec/tree/master/pkg/rego/schemas). You can match the schema name to the type of input you want to scan. +In the example above, you'll notice properties are being accessed from the `input.aws` object. The full set of schemas containing all of these properties is [available here](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas). You can match the schema name to the type of input you want to scan. You should also write a test for your rule(s). There are many examples of these in the `checks/cloud` directory ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks/cloud)). More information on how to write tests for Rego checks is provided in the [custom misconfiguration](../../../docs/scanner/misconfiguration/custom/testing.md) section of the docs. diff --git a/docs/community/contribute/checks/service-support.md b/docs/community/contribute/checks/service-support.md index e267593c221a..99a7c387f776 100644 --- a/docs/community/contribute/checks/service-support.md +++ b/docs/community/contribute/checks/service-support.md @@ -1,6 +1,6 @@ # Add Service Support -A service refers to a service by a cloud provider. This section details how to add a new service to an existing provider. All contributions need to be made to the [defsec repository](https://github.com/aquasecurity/defsec/). +A service refers to a service by a cloud provider. This section details how to add a new service to an existing provider. All contributions need to be made to the [trivy repository](https://github.com/aquasecurity/trivy/). Before you begin, verify that the provider does not already have the service that you plan to add. @@ -32,7 +32,7 @@ type Baz struct { } ``` -A `Baz` can have a name, and can optionally be encrypted. Instead of using raw string and bool types respectively, we use the defsec types `types.StringValue` and `types.BoolValue`. These types wrap the raw values and provide additional metadata about the value, such as whether it was set by the user or not, and the file and line number where the resource was defined. The `types.Metadata` struct is embedded in all of the defsec types, and provides a common set of metadata for all resources. This includes the file and line number where the resource was defined, and the name of the resource. +A `Baz` can have a name, and can optionally be encrypted. Instead of using raw string and bool types respectively, we use the trivy types `types.StringValue` and `types.BoolValue`. These types wrap the raw values and provide additional metadata about the value, such as whether it was set by the user or not, and the file and line number where the resource was defined. The `types.Metadata` struct is embedded in all of the Trivy types, and provides a common set of metadata for all resources. This includes the file and line number where the resource was defined, and the name of the resource. Next you'll need to add a reference to your new service struct in the provider struct at `pkg/providers/foo/foo.go`: diff --git a/docs/docs/scanner/misconfiguration/custom/contribute-checks.md b/docs/docs/scanner/misconfiguration/custom/contribute-checks.md index b39e9c767454..b38d201e8295 100644 --- a/docs/docs/scanner/misconfiguration/custom/contribute-checks.md +++ b/docs/docs/scanner/misconfiguration/custom/contribute-checks.md @@ -2,7 +2,7 @@ The contributing section provides detailed information on -* how to contribute custom checks to the [trivy-policies repository](../../../../community/contribute/checks/overview.md/). +* how to contribute custom checks to the [trivy-policies repository](../../../../community/contribute/checks/overview.md/) * add [new services](../../../../community/contribute/checks/service-support.md) to existing cloud providers * add a [new provider](../../../../community/contribute/checks/provider-support.md/) From 03e1d3c1bc9d01c42195a2f0ef21fef785e6f338 Mon Sep 17 00:00:00 2001 From: AnaisUrlichs Date: Mon, 4 Mar 2024 15:40:20 +0000 Subject: [PATCH 04/11] docs: update contributing custom checks based on PR feedback, first revision Signed-off-by: AnaisUrlichs --- docs/community/contribute/checks/overview.md | 71 ++++++++++--------- .../contribute/checks/provider-support.md | 49 ------------- .../contribute/checks/service-support.md | 63 ++++++++++------ .../scanner/misconfiguration/custom/index.md | 20 +++++- .../misconfiguration/custom-checks.md | 2 +- mkdocs.yml | 1 - 6 files changed, 98 insertions(+), 108 deletions(-) delete mode 100644 docs/community/contribute/checks/provider-support.md diff --git a/docs/community/contribute/checks/overview.md b/docs/community/contribute/checks/overview.md index e43377abed97..1e0c27f41f6e 100644 --- a/docs/community/contribute/checks/overview.md +++ b/docs/community/contribute/checks/overview.md @@ -2,11 +2,11 @@ The following guide provides an overview of contributing checks to the default checks in Trivy. -All of the checks in Trivy can be found in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/tree/main) repository on GitHub. Before you beging writing a check, ensure: +All of the checks in Trivy can be found in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/tree/main) repository on GitHub. Before you begin writing a check, ensure: 1. The check does not already exist as part of the default checks in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/tree/main) repository. 2. The pull requests in the [trivy-policies](https://github.com/aquasecurity/trivy-policies/pulls) repository to see whether someone else is already contributing the check that you wanted to add. -3. The [issues in Trivy](https://github.com/aquasecurity/trivy/issues) to see whether any specific checks are missing in Trivy. +3. The [issues in Trivy](https://github.com/aquasecurity/trivy/issues) to see whether any specific checks are missing in Trivy that you can contribute. If anything is unclear, please [start a discussion](https://github.com/aquasecurity/trivy/discussions/new) and we'll do our best to help. @@ -48,15 +48,11 @@ deny[res] { } ``` -## Adding a new provider or service +## Verify the provider and service exists Every check references a provider. The list of providers are found in the [trivy](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) repository. -Before writing a new check, you need to verify if the provider your check targets is supported by Trivy. If it's not, you'll need to add support for it. Additionally, if the provider that you want to target exists, you need to check whether the service your policy will target is supported. - -### Add Support for a New Cloud Provider - -[Please reference the documentation on adding Support for a New Cloud Provider](./provider-support.md). +Before writing a new check, you need to verify if the provider your check targets is supported by Trivy. If the provider that you want to target exists, you need to check whether the service your policy will target is supported. If the service is not support, please refer to the documentation on adding support for a new service in an existing provider. ### Add Support for a New Service in an existing Provider @@ -64,39 +60,45 @@ Before writing a new check, you need to verify if the provider your check target This guide also showcases how to add new properties for an existing Service. -## Check Metadata +## Create a new .rego file -The metadata is the top section that starts with `# METADATA`, and has to be placed on top of the check. You can copy and paste from another check as a starting point. This format is effectively _yaml_ within a Rego comment, and is [defined as part of Rego itself](https://www.openpolicyagent.org/docs/latest/policy-language/#metadata). +The following directory in the trivy-policies repository contains all of our custom checks. Depending on what type of check you want to create, you will need to nest a new `.rego` file in either of the [subdirectories](https://github.com/aquasecurity/trivy-policies/tree/main/checks): -The metadata consists of the following fields: +* cloud: All checks related to cloud providers and their services +* docker: Docker specific checks +* kubernetes: Kubernetes specific checks -- `title` is a title for the rule. The title should clearly and succinctly state the problem which is being detected. -- `description` is a description of the problem which is being detected. The description should be a little more verbose than the title, and should describe what the rule is trying to achieve. Imagine it completing a sentence starting with `You should...`. -- `scope` is used to define the scope of the policy. In this case, we are defining a check that applies to the entire package. At the moment, Trivy only supports using package scope for metadata, so this should always be set to `package`. -- `schemas` tells Rego that it should use the `AWS` schema to validate the use of the input data in the policy. We currently support [the following shemas](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas). Please choose a schema that is applicable to your check. Using a schema can help you validate your policy faster for syntax issues. -- `custom` is used to define custom fields that can be used by Trivy to provide additional context to the policy and any related detections. This can contain the following: - - `avd_id` can be used to link the check to the AVD entry. In the example check above, the `avd_id` `AVD-AWS-0176` is the ID of the check in the [AWS Vulnerability Database](https://avd.aquasec.com/). If there is no AVD_ID available, you need to generate an ID to use for this field using `make id`. - - `provider` is the name of the [provider](https://github.com/aquasecurity/defsec/tree/master/pkg/providers) the check targets. This should be the same as the provider name in the `pkg/providers` directory, e.g. `aws`. - - `service` is the name of the service by the provider that the check targets. This should be the same as the service name in the `pkg/providers` directory, e.g. `rds`. - - `severity` is the severity of the check. This should be one of `LOW`, `MEDIUM`, `HIGH`, or `CRITICAL`. - - `short_code` is a short code for the check. This should be a short, descriptive name for the check, separating words with hyphens. You should omit provider/service from this. - - `recommended_action` is a recommended remediation action for the check. This should be a short, descriptive sentence describing what the user should do to resolve the issue. - - `input` tells trivy what inputs this check should be applied to. Cloud provider checks should always use the `selector` input, and should always use the `type` selector with `cloud`. Check targeting Kubernetes yaml can use `kubenetes`, RBAC can use `rbac`, and so on. - - `subtypes` aid the engine to determine if it should load this policy or not for scanning. This can aid with the performance of scanning, especially if you have a lot of checks but not all apply to the IaC that you are trying to scan. +## Check Package name -### Generating an ID +Have a look at the existing package names in the [built in checks](https://github.com/aquasecurity/trivy-policies/tree/main/checks). -If you plan to contribue your check back into the [trivy-policies](https://github.com/aquasecurity/trivy-policies) repository, it will require a valid ID. +The package name should be in the format `builtin.PROVIDER.SERVICE.ID`, e.g. `builtin.aws.rds.aws0176`. -Running `make id` in the trivy-policies repository will provide you with the next available _ID_ for your rule. The ID is used in the Rego check to identify it. +## Generating an ID -## Writing Rego Rules +Every check has a custom ID that is referenced throughout the metadata of the check to uniquely identify the check. If you plan to contribue your check back into the [trivy-policies](https://github.com/aquasecurity/trivy-policies) repository, it will require a valid ID. -At last, it's time to write your rule code! Rules are defined using _OPA Rego_. You can find a number of examples in the `checks/cloud` directory ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks/cloud)). The [OPA documentation](https://www.openpolicyagent.org/docs/latest/policy-language/) is a great place to start learning Rego. You can also check out the [Rego Playground](https://play.openpolicyagent.org/) to experiment with Rego, and [join the OPA Slack](https://slack.openpolicyagent.org/). +Running `make id` in the root of the trivy-policies repository will provide you with the next available _ID_ for your rule. -Create a new file in `checks/cloud` ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks/cloud)) with the name of your rule. You should nest it in the existing directory structure as applicable. The package name should be in the format `builtin.PROVIDER.SERVICE.ID`, e.g. `builtin.aws.rds.aws0176`. +## Check Schemas + +Rego Checks for Trivy can utilise Schams to map the input to specific objects. The schemas available are listed [here.](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas). + +More information on using the builtin schemas is provided in the [main documentation.](../../../docs/scanner/misconfiguration/custom/schema.md) + +## Check Metadata + +The metadata is the top section that starts with `# METADATA`, and has to be placed on top of the check. You can copy and paste from another check as a starting point. This format is effectively _yaml_ within a Rego comment, and is [defined as part of Rego itself](https://www.openpolicyagent.org/docs/latest/policy-language/#metadata). + +For detailed information on each component of the Check Metadata, please refer to the [main documentation.](../../../docs/scanner/misconfiguration/custom/index.md) + +Note that while the Metadata is optional in your own custom checks for Trivy, if you are contributing your check to the Trivy builtin checks, the Metadata section will be required. + + +## Writing Rego Rules + +Rules are defined using _OPA Rego_. You can find a number of examples in the `checks` directory ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks)). The [OPA documentation](https://www.openpolicyagent.org/docs/latest/policy-language/) is a great place to start learning Rego. You can also check out the [Rego Playground](https://play.openpolicyagent.org/) to experiment with Rego, and [join the OPA Slack](https://slack.openpolicyagent.org/). -Now you'll need to write the rule logic. This is the code that will be executed to detect the issue. You should define a rule named `deny` and place your code inside this. Every check in Trivy needs to have a `deny` rule. ```rego deny[res] { @@ -109,14 +111,17 @@ deny[res] { The rule should return a result, which can be created using `result.new` (this function does not need to be imported, it is defined internally and provided at runtime). The first argument is the message to display, and the second argument is the resource that the issue was detected on. -In the example above, you'll notice properties are being accessed from the `input.aws` object. The full set of schemas containing all of these properties is [available here](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas). You can match the schema name to the type of input you want to scan. +It is possible to pass any rego variable that references a field of the input document. -You should also write a test for your rule(s). There are many examples of these in the `checks/cloud` directory ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks/cloud)). More information on how to write tests for Rego checks is provided in the [custom misconfiguration](../../../docs/scanner/misconfiguration/custom/testing.md) section of the docs. ## Generate docs Finally, you'll want to generate documentation for your newly added rule. Please run `make docs` to generate the documentation for your new policy and submit a PR for us to take a look at. +## Adding Tests + +There are many examples of these in the `checks` directory for each check ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks)). More information on how to write tests for Rego checks is provided in the [custom misconfiguration](../../../docs/scanner/misconfiguration/custom/testing.md) section of the docs. + ## Example PR You can see a full example PR for a new rule being added here: [https://github.com/aquasecurity/defsec/pull/1000](https://github.com/aquasecurity/defsec/pull/1000). diff --git a/docs/community/contribute/checks/provider-support.md b/docs/community/contribute/checks/provider-support.md deleted file mode 100644 index dfadcd94ffb2..000000000000 --- a/docs/community/contribute/checks/provider-support.md +++ /dev/null @@ -1,49 +0,0 @@ -# Add Provider Support - -A provider refers to a cloud provier such as AWS or resource type such as Docker. - -Before adding a new provider, check whether the provider you would like to add does not already exist in the [trivy repository](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers). Next, create a [discussion](https://github.com/aquasecurity/trivy/discussions/categories/ideas) in the Trivy repository to get input on your plan of adding a new provider. - -We highly welcome new contributions! - -## Adding a new Provider - -First, add a new subdirectory to the [pkg/iac/providers](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) directory in the Trivy repository, named after your provider. Inside this, create a Go file with the same name as your provider and create a Go struct to hold information about all of the services supported by your provider. - -For example, adding support for a new provider called foo would look like this: - -pkg/iac/providers/foo/foo.go: - -``` -package foo - -type Foo struct { - // Add services here later... -} -``` - -Next, you should add a reference to your provider struct in [pkg/state/state.go](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/state/state.go): - -``` -type State struct { - // ... - Foo foo.Foo - // ... -} -``` - -Once done, add one or more adapters to [internal/adapters](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/adapters). An adapter takes an input and populates your provider struct. For example, if you want to scan a Terraform plan, you will need to add an adapter that takes the Terraform plan and populates your provider struct. The AWS provider support in Trivy uses multiple adapters - it can adapt CloudFormation, Terraform, and live AWS accounts. Each of these has an adapter in this directory. - -To support Terraform as an input, your adapter should look similar to the following: - -``` -func Adapt(modules terraform.Modules) (foo.Foo, error) { - return foo.Foo{ - // ... - }, nil -} -``` - -Additionally, it needs to be called in [iac/adapters/terraform/adapt.go](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/adapters/terraform/adapt.go) in the case of Terraform. - -We recommend browsing the existing adapters to see how they work, as there is a lot of common code that can be reused. \ No newline at end of file diff --git a/docs/community/contribute/checks/service-support.md b/docs/community/contribute/checks/service-support.md index 99a7c387f776..9b925764dde7 100644 --- a/docs/community/contribute/checks/service-support.md +++ b/docs/community/contribute/checks/service-support.md @@ -2,48 +2,67 @@ A service refers to a service by a cloud provider. This section details how to add a new service to an existing provider. All contributions need to be made to the [trivy repository](https://github.com/aquasecurity/trivy/). -Before you begin, verify that the provider does not already have the service that you plan to add. +## Prerequisites -Adding a new service involves two steps. The service will need a data structure to store information about the required resources, and then one or more adapters to convert input(s) into the aforementioned data structure. +Before you begin, verify that the [provider](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) does not already have the service that you plan to add. ## Adding a new service to an existing provider -To add a new service named `Bar` to a provider named `foo`, you'll need to add a new file at `pkg/providers/foo/bar/bar.go`: +Adding a new service involves two steps. The service will need a data structure to store information about the required resources that will be scanned. Additionally, the service will require one or more adapters to convert the scan targetes as input(s) into the aforementioned data structure. + + +### Create a new file in the provider directory + +In this example, we are adding the CodeBuild service to the AWS provider. + +First create a new directory and file for your new service under the provider directory: e.g. [aws/codebuild/codebuild.go](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/providers/aws/codebuild/codebuild.go) + +The CodeBuild service will require a structure `struct` to hold the information on input that is scanned. The input is the CodeBuild resource that a user configured and wants to scan for misconfiguration. + ``` -type Bar struct { - // ... +type CodeBuild struct { + Projects []Project } ``` -Let's say the `Bar` service manages resources called `Baz`. You'll need to add a new struct to the `Bar` struct to hold information about this resource: +The CodeBuild service manages `Project` resources. The `Project` struct has been added to hold information about each Project resources; `Project` Resources in turn manage `ArtifactSettings`: ``` -type Bar struct { - // ... - Baz []Baz - // ... +type Project struct { + Metadata iacTypes.Metadata + ArtifactSettings ArtifactSettings + SecondaryArtifactSettings []ArtifactSettings } -type Baz struct { - types.Metadata - Name types.StringValue - Encrypted types.BoolValue +type ArtifactSettings struct { + Metadata iacTypes.Metadata + EncryptionEnabled iacTypes.BoolValue } ``` -A `Baz` can have a name, and can optionally be encrypted. Instead of using raw string and bool types respectively, we use the trivy types `types.StringValue` and `types.BoolValue`. These types wrap the raw values and provide additional metadata about the value, such as whether it was set by the user or not, and the file and line number where the resource was defined. The `types.Metadata` struct is embedded in all of the Trivy types, and provides a common set of metadata for all resources. This includes the file and line number where the resource was defined, and the name of the resource. +The `iacTypes.Metadata` struct is embedded in all of the Trivy types, and provides a common set of metadata for all resources. This includes the file and line number where the resource was defined, and the name of the resource. -Next you'll need to add a reference to your new service struct in the provider struct at `pkg/providers/foo/foo.go`: +A resource, in this example `Project`, can have a name, and can optionally be encrypted. Instead of using raw string and bool types respectively, we use the trivy types `iacTypes.Metadata` and `iacTypes.BoolValue`. These types wrap the raw values and provide additional metadata about the value, such as whether it was set by the user or not, and the file and line number where the resource was defined. + +Have a look at the other providers and services in the [`iac/providers`](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) directory in Trivy. + +Next you'll need to add a reference to your new service struct in the [provider struct](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/providers/aws/aws.go) at `pkg/iac/providers/aws/aws.go`: ``` -type Foo struct { - // ... - Bar bar.Bar - // ... +type AWS struct { + ... + CodeBuild codebuild.CodeBuild + ... } ``` -Now you'll need to update all of the adapters which populate the Foo provider struct. For example, if you want to support Terraform, you'll need to update `internal/adapters/terraform/foo/bar/adapt.go`. +Now you'll need to update all of the [adapters](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/adapters) which populate the AWS provider struct. For example, if you want to support Terraform, you'll need to update [`trivy/pkg/iac/adapters/terraform/aws/codebuild/adapt.go`](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/adapters/terraform/aws/codebuild/adapt.go). + +TODO HOW? + +## Create a new Schema for your provider + +Once the new service has been added to the provider, you need to create a the schema for the service as part of the provider schema. In the Trivy root directory run `make schema` to generate the schema for your new service. -Finally, make sure you run make schema to generate the schema for your new service. \ No newline at end of file +TODO \ No newline at end of file diff --git a/docs/docs/scanner/misconfiguration/custom/index.md b/docs/docs/scanner/misconfiguration/custom/index.md index b1c16219e8f8..cdfa4b18a660 100644 --- a/docs/docs/scanner/misconfiguration/custom/index.md +++ b/docs/docs/scanner/misconfiguration/custom/index.md @@ -63,7 +63,20 @@ If you add a new custom policy, it must be defined under a new package like `use `# METADATA` (optional) : - SHOULD be defined for clarity since these values will be displayed in the scan results - - `custom.input` SHOULD be set to indicate the input type the policy should be applied to. See [list of available types](https://github.com/aquasecurity/defsec/blob/418759b4dc97af25f30f32e0bd365be7984003a1/pkg/types/sources.go) + - `title` is a title for the rule. The title should clearly and succinctly state the problem which is being detected. + - `description` is a description of the problem which is being detected. The description should be a little more verbose than the title, and should describe what the rule is trying to achieve. Imagine it completing a sentence starting with `You should...`. + - `scope` is used to define the scope of the policy. In this case, we are defining a check that applies to the entire package. At the moment, Trivy only supports using package scope for metadata, so this should always be set to `package`. + - `schemas` tells Rego that it should use the `AWS` schema to validate the use of the input data in the policy. We currently support [the following schemas](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas). Please choose a schema that is applicable to your check. Using a schema can help you validate your policy faster for syntax issues. + - `custom` is used to define custom fields that can be used by Trivy to provide additional context to the policy and any related detections. This can contain the following: + - `avd_id` can be used to link the check to the AVD entry. In the example check above, the `avd_id` `AVD-AWS-0176` is the ID of the check in the [AWS Vulnerability Database](https://avd.aquasec.com/). If there is no AVD_ID available, you can generate an ID to use for this field using `make id`. + - `provider` is the name of the [provider](https://github.com/aquasecurity/defsec/tree/master/pkg/providers) the check targets. This should be the same as the provider name in the `pkg/providers` directory, e.g. `aws`. + - `service` is the name of the service by the provider that the check targets. This should be the same as the service name in the `pkg/providers` directory, e.g. `rds`. + - `severity` is the severity of the check. This should be one of `LOW`, `MEDIUM`, `HIGH`, or `CRITICAL`. + - `short_code` for the check. This should be a short, descriptive name for the check, separating words with hyphens. You should omit provider/service from this. + - `recommended_action` is a recommended remediation action for the check. This should be a short, descriptive sentence describing what the user should do to resolve the issue. + - `input` tells trivy what inputs this check should be applied to. Cloud provider checks should always use the `selector` input, and should always use the `type` selector with `cloud`. Check targeting Kubernetes yaml can use `kubenetes`, RBAC can use `rbac`, and so on. + - `subtypes` Please refer to the section below for subtypes + - `custom.input` SHOULD be set to indicate the input type the policy should be applied to. See [list of available types](https://github.com/aquasecurity/defsec/blob/418759b4dc97af25f30f32e0bd365be7984003a1/pkg/types/sources.go) `package` (required) : - MUST follow the Rego's [specification][package] @@ -80,7 +93,10 @@ If you add a new custom policy, it must be defined under a new package like `use - A `string` denoting the detected issue - Although `object` with `msg` field is accepted, other fields are dropped and `string` is recommended if `result.new()` is not utilised. - e.g. `{"msg": "deny message", "details": "something"}` - + +### Subtypes in the custom data + +TODO ### Package A package name must be unique per policy. diff --git a/docs/tutorials/misconfiguration/custom-checks.md b/docs/tutorials/misconfiguration/custom-checks.md index ecf8f5af1b5a..5d39e34263f2 100644 --- a/docs/tutorials/misconfiguration/custom-checks.md +++ b/docs/tutorials/misconfiguration/custom-checks.md @@ -8,7 +8,7 @@ When you are writing a check, it's important to understand the input to the chec Since Rego is primarily tailored to query JSON objects, all incoming configuration files needs to be first converted to structured objects, which is available to the Rego code as the input variable. This is nothing that users have to do manually in Trivy. Instead, Rego makes it possible to pass in custom Schemas that detail how files are converted. Once Rego has access to a custom Schema, it will know in which format to access configuration files such as a Dockerfile. -[Here you can find the schemas](https://github.com/aquasecurity/defsec/tree/master/pkg/rego/schemas) that define how different configuration files are converted to JSON by Trivy. +[Here you can find the schemas](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas) that define how different configuration files are converted to JSON by Trivy. This tutorial will make use of the [dockerfile.json schema](https://github.com/aquasecurity/defsec/tree/master/pkg/rego/schemas). The schema will need to be parsed into your custom check. Users can also use the [Schema Explorer](https://aquasecurity.github.io/trivy-schemas/) to view the structure of the data provided to Rego. diff --git a/mkdocs.yml b/mkdocs.yml index 76f072ebb6ee..b319cf2af9d1 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -187,7 +187,6 @@ nav: - Pull Requests: community/contribute/pr.md - Contribute Rego Checks: - Overview: community/contribute/checks/overview.md - - Add Provider Support: community/contribute/checks/provider-support.md - Add Service Supprot: community/contribute/checks/service-support.md - Maintainer: - Help Wanted: community/maintainer/help-wanted.md From 2da6cfe600eefb31c809b2770ba3a86aa1b9fb35 Mon Sep 17 00:00:00 2001 From: AnaisUrlichs Date: Mon, 4 Mar 2024 15:42:30 +0000 Subject: [PATCH 05/11] docs: fix schema link from defsec to trivy Signed-off-by: AnaisUrlichs --- docs/docs/scanner/misconfiguration/custom/schema.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/scanner/misconfiguration/custom/schema.md b/docs/docs/scanner/misconfiguration/custom/schema.md index 8791d1a22752..93c8f433f2a0 100644 --- a/docs/docs/scanner/misconfiguration/custom/schema.md +++ b/docs/docs/scanner/misconfiguration/custom/schema.md @@ -4,7 +4,7 @@ Policies can be defined with custom schemas that allow inputs to be verified against them. Adding a policy schema enables Trivy to show more detailed error messages when an invalid input is encountered. -In Trivy we have been able to define a schema for a [Dockerfile](https://github.com/aquasecurity/trivy-iac/blob/main/pkg/rego/schemas/dockerfile.json) +In Trivy we have been able to define a schema for a [Dockerfile](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas) Without input schemas, a policy would be as follows: !!! example From 00ee07dcf42849b385e41cc40df118238f2b9854 Mon Sep 17 00:00:00 2001 From: Anais Urlichs <33576047+AnaisUrlichs@users.noreply.github.com> Date: Tue, 5 Mar 2024 09:46:04 +0000 Subject: [PATCH 06/11] Update docs/community/contribute/checks/overview.md Co-authored-by: simar7 <1254783+simar7@users.noreply.github.com> --- docs/community/contribute/checks/overview.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/docs/community/contribute/checks/overview.md b/docs/community/contribute/checks/overview.md index 1e0c27f41f6e..a3c59577fa29 100644 --- a/docs/community/contribute/checks/overview.md +++ b/docs/community/contribute/checks/overview.md @@ -50,7 +50,12 @@ deny[res] { ## Verify the provider and service exists -Every check references a provider. The list of providers are found in the [trivy](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) repository. +Every check for a cloud service references a cloud provider. The list of providers are found in the [trivy](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) repository. + +Before writing a new check for a cloud provider, you need to verify if the cloud provider your check targets is supported by Trivy. If it's not, you'll need to add support for it. Additionally, if the provider that you want to target exists, you need to check whether the service your policy will target is supported. As a reference you can take a look at the AWS provider [here](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/providers/aws/aws.go). + +???+ note + New Kubernetes and Dockerfile checks do not require any additional provider definitions. You can find an example of a Dockerfile check [here](https://github.com/aquasecurity/trivy-policies/blob/main/checks/docker/add_instead_of_copy.rego) and a Kubernetes check [here](https://github.com/aquasecurity/trivy-policies/blob/main/checks/kubernetes/general/CPU_not_limited.rego). Before writing a new check, you need to verify if the provider your check targets is supported by Trivy. If the provider that you want to target exists, you need to check whether the service your policy will target is supported. If the service is not support, please refer to the documentation on adding support for a new service in an existing provider. From d9a2ee3349c3751005080d9503cf71ba07e62b71 Mon Sep 17 00:00:00 2001 From: AnaisUrlichs Date: Tue, 5 Mar 2024 11:27:03 +0000 Subject: [PATCH 07/11] docs: fix contributing checks based on PR comments Signed-off-by: AnaisUrlichs --- docs/community/contribute/checks/overview.md | 11 ++- .../contribute/checks/service-support.md | 19 ++--- .../custom/contribute-checks.md | 2 +- .../scanner/misconfiguration/custom/index.md | 72 +++++++++++-------- 4 files changed, 58 insertions(+), 46 deletions(-) diff --git a/docs/community/contribute/checks/overview.md b/docs/community/contribute/checks/overview.md index a3c59577fa29..faf8be9d33c2 100644 --- a/docs/community/contribute/checks/overview.md +++ b/docs/community/contribute/checks/overview.md @@ -52,7 +52,7 @@ deny[res] { Every check for a cloud service references a cloud provider. The list of providers are found in the [trivy](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) repository. -Before writing a new check for a cloud provider, you need to verify if the cloud provider your check targets is supported by Trivy. If it's not, you'll need to add support for it. Additionally, if the provider that you want to target exists, you need to check whether the service your policy will target is supported. As a reference you can take a look at the AWS provider [here](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/providers/aws/aws.go). +Before writing a new check for a cloud provider, you need to verify if the cloud provider or resource type that your check targets is supported by Trivy. If it's not, you'll need to add support for it. Additionally, if the provider that you want to target exists, you need to check whether the service your policy will target is supported. As a reference you can take a look at the AWS provider [here](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/providers/aws/aws.go). ???+ note New Kubernetes and Dockerfile checks do not require any additional provider definitions. You can find an example of a Dockerfile check [here](https://github.com/aquasecurity/trivy-policies/blob/main/checks/docker/add_instead_of_copy.rego) and a Kubernetes check [here](https://github.com/aquasecurity/trivy-policies/blob/main/checks/kubernetes/general/CPU_not_limited.rego). @@ -87,7 +87,7 @@ Running `make id` in the root of the trivy-policies repository will provide you ## Check Schemas -Rego Checks for Trivy can utilise Schams to map the input to specific objects. The schemas available are listed [here.](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas). +Rego Checks for Trivy can utilise Schemas to map the input to specific objects. The schemas available are listed [here.](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas). More information on using the builtin schemas is provided in the [main documentation.](../../../docs/scanner/misconfiguration/custom/schema.md) @@ -114,18 +114,17 @@ deny[res] { } ``` -The rule should return a result, which can be created using `result.new` (this function does not need to be imported, it is defined internally and provided at runtime). The first argument is the message to display, and the second argument is the resource that the issue was detected on. +The rule should return a result, which can be created using `result.new`. This function does not need to be imported, it is defined internally and provided at runtime. The first argument is the message to display and the second argument is the resource that the issue was detected on. It is possible to pass any rego variable that references a field of the input document. - ## Generate docs -Finally, you'll want to generate documentation for your newly added rule. Please run `make docs` to generate the documentation for your new policy and submit a PR for us to take a look at. +Finally, you'll want to generate documentation for your newly added rule. Please run `make docs` in the [trivy-policies](https://github.com/aquasecurity/trivy-policies) directory to generate the documentation for your new policy and submit a PR for us to take a look at. ## Adding Tests -There are many examples of these in the `checks` directory for each check ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks)). More information on how to write tests for Rego checks is provided in the [custom misconfiguration](../../../docs/scanner/misconfiguration/custom/testing.md) section of the docs. +All Rego checks need to have tests. There are many examples of these in the `checks` directory for each check ([Link](https://github.com/aquasecurity/trivy-policies/tree/main/checks)). More information on how to write tests for Rego checks is provided in the [custom misconfiguration](../../../docs/scanner/misconfiguration/custom/testing.md) section of the docs. ## Example PR diff --git a/docs/community/contribute/checks/service-support.md b/docs/community/contribute/checks/service-support.md index 9b925764dde7..8f750115af08 100644 --- a/docs/community/contribute/checks/service-support.md +++ b/docs/community/contribute/checks/service-support.md @@ -10,15 +10,13 @@ Before you begin, verify that the [provider](https://github.com/aquasecurity/tri Adding a new service involves two steps. The service will need a data structure to store information about the required resources that will be scanned. Additionally, the service will require one or more adapters to convert the scan targetes as input(s) into the aforementioned data structure. - ### Create a new file in the provider directory In this example, we are adding the CodeBuild service to the AWS provider. -First create a new directory and file for your new service under the provider directory: e.g. [aws/codebuild/codebuild.go](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/providers/aws/codebuild/codebuild.go) - -The CodeBuild service will require a structure `struct` to hold the information on input that is scanned. The input is the CodeBuild resource that a user configured and wants to scan for misconfiguration. +First, create a new directory and file for your new service under the provider directory: e.g. [aws/codebuild/codebuild.go](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/providers/aws/codebuild/codebuild.go) +The CodeBuild service will require a structure `struct` to hold the information on the input that is scanned. The input is the CodeBuild resource that a user configured and wants to scan for misconfiguration. ``` type CodeBuild struct { @@ -41,7 +39,7 @@ type ArtifactSettings struct { } ``` -The `iacTypes.Metadata` struct is embedded in all of the Trivy types, and provides a common set of metadata for all resources. This includes the file and line number where the resource was defined, and the name of the resource. +The `iacTypes.Metadata` struct is embedded in all of the Trivy types and provides a common set of metadata for all resources. This includes the file and line number where the resource was defined and the name of the resource. A resource, in this example `Project`, can have a name, and can optionally be encrypted. Instead of using raw string and bool types respectively, we use the trivy types `iacTypes.Metadata` and `iacTypes.BoolValue`. These types wrap the raw values and provide additional metadata about the value, such as whether it was set by the user or not, and the file and line number where the resource was defined. @@ -57,12 +55,15 @@ type AWS struct { } ``` -Now you'll need to update all of the [adapters](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/adapters) which populate the AWS provider struct. For example, if you want to support Terraform, you'll need to update [`trivy/pkg/iac/adapters/terraform/aws/codebuild/adapt.go`](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/adapters/terraform/aws/codebuild/adapt.go). +### Update Adapters + +Now you'll need to update all of the [adapters](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/adapters) which populate the struct of the provider that you have been using. Following the example above, if you want to add support for CodeBuild in Terraform, you'll need to update the Terraform AWS adatper as shown here: [`trivy/pkg/iac/adapters/terraform/aws/codebuild/adapt.go`](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/adapters/terraform/aws/codebuild/adapt.go). + +Another example for updating the adapters is provided in the [following PR.](https://github.com/aquasecurity/defsec/pull/1000/files) Additionally, please refer to the respective Terraform documentation on the provider to which you are adding the service. For instance, the Terraform documentation for AWS CodeBuild is provided [here.](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codebuild_project) -TODO HOW? ## Create a new Schema for your provider -Once the new service has been added to the provider, you need to create a the schema for the service as part of the provider schema. In the Trivy root directory run `make schema` to generate the schema for your new service. +Once the new service has been added to the provider, you need to create the schema for the service as part of the provider schema. -TODO \ No newline at end of file +This process has been automated with mage commands. In the Trivy root directory run `mage schema:generate` to generate the schema for your new service and `mage schema:verify`. diff --git a/docs/docs/scanner/misconfiguration/custom/contribute-checks.md b/docs/docs/scanner/misconfiguration/custom/contribute-checks.md index b38d201e8295..037973fe4adc 100644 --- a/docs/docs/scanner/misconfiguration/custom/contribute-checks.md +++ b/docs/docs/scanner/misconfiguration/custom/contribute-checks.md @@ -4,6 +4,6 @@ The contributing section provides detailed information on * how to contribute custom checks to the [trivy-policies repository](../../../../community/contribute/checks/overview.md/) * add [new services](../../../../community/contribute/checks/service-support.md) to existing cloud providers -* add a [new provider](../../../../community/contribute/checks/provider-support.md/) +* add [new fields](../../../../community/contribute/checks/service-support.md) to an existing Service This way, they become accessible as default [policies.](https://github.com/aquasecurity/trivy-policies) \ No newline at end of file diff --git a/docs/docs/scanner/misconfiguration/custom/index.md b/docs/docs/scanner/misconfiguration/custom/index.md index cdfa4b18a660..7eb862da287d 100644 --- a/docs/docs/scanner/misconfiguration/custom/index.md +++ b/docs/docs/scanner/misconfiguration/custom/index.md @@ -61,22 +61,9 @@ If you add a new custom policy, it must be defined under a new package like `use ### Policy structure -`# METADATA` (optional) +`# METADATA` (optional unless the check will be contributed into Trivy) : - SHOULD be defined for clarity since these values will be displayed in the scan results - - `title` is a title for the rule. The title should clearly and succinctly state the problem which is being detected. - - `description` is a description of the problem which is being detected. The description should be a little more verbose than the title, and should describe what the rule is trying to achieve. Imagine it completing a sentence starting with `You should...`. - - `scope` is used to define the scope of the policy. In this case, we are defining a check that applies to the entire package. At the moment, Trivy only supports using package scope for metadata, so this should always be set to `package`. - - `schemas` tells Rego that it should use the `AWS` schema to validate the use of the input data in the policy. We currently support [the following schemas](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/rego/schemas). Please choose a schema that is applicable to your check. Using a schema can help you validate your policy faster for syntax issues. - - `custom` is used to define custom fields that can be used by Trivy to provide additional context to the policy and any related detections. This can contain the following: - - `avd_id` can be used to link the check to the AVD entry. In the example check above, the `avd_id` `AVD-AWS-0176` is the ID of the check in the [AWS Vulnerability Database](https://avd.aquasec.com/). If there is no AVD_ID available, you can generate an ID to use for this field using `make id`. - - `provider` is the name of the [provider](https://github.com/aquasecurity/defsec/tree/master/pkg/providers) the check targets. This should be the same as the provider name in the `pkg/providers` directory, e.g. `aws`. - - `service` is the name of the service by the provider that the check targets. This should be the same as the service name in the `pkg/providers` directory, e.g. `rds`. - - `severity` is the severity of the check. This should be one of `LOW`, `MEDIUM`, `HIGH`, or `CRITICAL`. - - `short_code` for the check. This should be a short, descriptive name for the check, separating words with hyphens. You should omit provider/service from this. - - `recommended_action` is a recommended remediation action for the check. This should be a short, descriptive sentence describing what the user should do to resolve the issue. - - `input` tells trivy what inputs this check should be applied to. Cloud provider checks should always use the `selector` input, and should always use the `type` selector with `cloud`. Check targeting Kubernetes yaml can use `kubenetes`, RBAC can use `rbac`, and so on. - - `subtypes` Please refer to the section below for subtypes - - `custom.input` SHOULD be set to indicate the input type the policy should be applied to. See [list of available types](https://github.com/aquasecurity/defsec/blob/418759b4dc97af25f30f32e0bd365be7984003a1/pkg/types/sources.go) + - `custom.input` SHOULD be set to indicate the input type the policy should be applied to. See [list of available types](https://github.com/aquasecurity/defsec/blob/418759b4dc97af25f30f32e0bd365be7984003a1/pkg/types/sources.go) `package` (required) : - MUST follow the Rego's [specification][package] @@ -94,10 +81,6 @@ If you add a new custom policy, it must be defined under a new package like `use - Although `object` with `msg` field is accepted, other fields are dropped and `string` is recommended if `result.new()` is not utilised. - e.g. `{"msg": "deny message", "details": "something"}` -### Subtypes in the custom data - -TODO - ### Package A package name must be unique per policy. @@ -136,21 +119,50 @@ Trivy supports extra fields in the `custom` section as described below. # - type: kubernetes ``` -All fields are optional. The `schemas` field should be used to enable policy validation using a built-in schema. The +If you are creating checks for your Trivy misconfiguration scans, all fields are optional. The `schemas` field should be used to enable policy validation using a built-in schema. The schema that will be used is based on the input document type. It is recommended to use this to ensure your policies are correct and do not reference incorrect properties/values. -| Field name | Allowed values | Default value | In table | In JSON | -|----------------------------|-------------------------------------------------------------------|:----------------------------:|:----------------:|:----------------:| -| title | Any characters | N/A | :material-check: | :material-check: | -| description | Any characters | | :material-close: | :material-check: | -| schemas.input | `schema["kubernetes"]`, `schema["dockerfile"]`, `schema["cloud"]` | (applied to all input types) | :material-close: | :material-close: | -| custom.id | Any characters | N/A | :material-check: | :material-check: | -| custom.severity | `LOW`, `MEDIUM`, `HIGH`, `CRITICAL` | UNKNOWN | :material-check: | :material-check: | -| custom.recommended_actions | Any characters | | :material-close: | :material-check: | -| custom.input.selector.type | Any item(s) in [this list][source-types] | | :material-close: | :material-check: | -| url | Any characters | | :material-close: | :material-check: | +| Field name | Allowed values | Description | Default value | In table | In JSON | Required in Trivy Check | +|----------------------------|-------------------------------------------------------------------|:-------------------------------------------------------:|:-----------------------------:|:----------------:|:----------------:| :-----------------------------: | +| title | Any characters | Name of the policy | N/A | :material-check: | :material-check: | :material-check: | +| description | Any characters | Description of the problem | N/A | :material-close: | :material-check: | :material-check: | +| schemas.input | `schema["kubernetes"]`, `schema["dockerfile"]`, `schema["cloud"]` | Used to validate the policy for syntax errors | (applied to all input types) | :material-close: | :material-close: | :material-check: | +| scope | `package` | Defines the policy scope | (applied to all input types) | :material-close: | :material-check: | :material-check: | +| custom.id | Any characters | More information provided below | N/A | :material-close: | :material-check: | :material-check: | +| custom.avd_id | Any characters | More information provided below | N/A | :material-close: | :material-check: | :material-check: | +| custom.severity | `LOW`, `MEDIUM`, `HIGH`, `CRITICAL` | The severity of the misconfiguration | `MEDIUM` | :material-close: | :material-check: | :material-check: | +| custom.provider | Any provider available in Trivy | The resource provider this check relates to | N/A | :material-close: | :material-check: | :material-close: | +| custom.service | The service name of the provider | One of the services available in the provider | N/A | :material-close: | :material-check: | :material-close: | +| custom.short_code | Any characters | Descriptive name for the check | N/A | :material-close: | :material-check: | :material-check: | +| custom.subtypes | Cloud Resource Subtypes | Refer to the section on subtypes below | N/A | :material-close: | :material-check: | :material-close: | +| custom.recommended_actions | Any characters | Describing what the user should do to resolve the issue | N/A | :material-close: | :material-check: | :material-check: | +| custom.input.selector.type | Any item(s) in [this list][source-types] | More information provided below | N/A | :material-close: | :material-check: | :material-check: | +| related_resource | Any characters | URL to related resources | N/A | :material-close: | :material-check: | :material-check: | + +#### custom.avd_id and custom.id + +The AVD_ID can be used to link the check to the Aqua Vulnerability Database (AVD) entry. For example, the `avd_id` `AVD-AWS-0176` is the ID of the check in the [AWS Vulnerability Database](https://avd.aquasec.com/). If you are [contributing your check to trivy-policies](../../../../community/contribute/checks/overview.md), you need to generate an ID using `make id` in the [trivy-policies](https://github.com/aquasecurity/trivy-policies) repository. The output of the command will provide you the next free IDs for the different providers in Trivy. + +The ID is based on the AVD_ID. For instanc,e if the `avd_id` is `AVD-AWS-0176`, the ID is `ID0176`. + +#### custom.provider + +The `provider` field references the [provider](https://github.com/aquasecurity/defsec/tree/master/pkg/providers) available in Trivy. This should be the same as the provider name in the `pkg/iac/providers` directory, e.g. `aws`. + +#### custom.service + +The `service` is the name of the service by the provider that the check targets. This should be the same as the service name in the `pkg/iac/providers` directory ([Link](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers)), e.g. `rds`. + +#### custom.input + +The `input` tells Trivy what inputs this check should be applied to. Cloud provider checks should always use the `selector` input, and should always use the `type` selector with `cloud`. Check targeting Kubernetes yaml can use `kubenetes`, RBAC can use `rbac`, and so on. + +#### Subtypes in the custom data + +Subtypes currently only need to be defined for cloud providers [as detailed in the documentation.](./selectors.md/#enabling-selectors-and-subtypes) +#### Scan Result Some fields are displayed in scan results. From 92630d04c6b6d6bcbd69a2e78bfcdea1a9fe68da Mon Sep 17 00:00:00 2001 From: Anais Urlichs <33576047+AnaisUrlichs@users.noreply.github.com> Date: Wed, 6 Mar 2024 09:20:22 +0000 Subject: [PATCH 08/11] Update docs/docs/scanner/misconfiguration/custom/index.md Co-authored-by: simar7 <1254783+simar7@users.noreply.github.com> --- docs/docs/scanner/misconfiguration/custom/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/scanner/misconfiguration/custom/index.md b/docs/docs/scanner/misconfiguration/custom/index.md index 7eb862da287d..a3540af235ed 100644 --- a/docs/docs/scanner/misconfiguration/custom/index.md +++ b/docs/docs/scanner/misconfiguration/custom/index.md @@ -119,7 +119,7 @@ Trivy supports extra fields in the `custom` section as described below. # - type: kubernetes ``` -If you are creating checks for your Trivy misconfiguration scans, all fields are optional. The `schemas` field should be used to enable policy validation using a built-in schema. The +If you are creating checks for your Trivy misconfiguration scans, some fields are optional as referenced in the table below. The `schemas` field should be used to enable policy validation using a built-in schema. The schema that will be used is based on the input document type. It is recommended to use this to ensure your policies are correct and do not reference incorrect properties/values. From d7526397346e1d68794bea9e5fbf3b0455ce8794 Mon Sep 17 00:00:00 2001 From: Anais Urlichs <33576047+AnaisUrlichs@users.noreply.github.com> Date: Wed, 6 Mar 2024 09:20:31 +0000 Subject: [PATCH 09/11] Update docs/community/contribute/checks/service-support.md Co-authored-by: simar7 <1254783+simar7@users.noreply.github.com> --- docs/community/contribute/checks/service-support.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/community/contribute/checks/service-support.md b/docs/community/contribute/checks/service-support.md index 8f750115af08..91bcefe14da9 100644 --- a/docs/community/contribute/checks/service-support.md +++ b/docs/community/contribute/checks/service-support.md @@ -41,7 +41,7 @@ type ArtifactSettings struct { The `iacTypes.Metadata` struct is embedded in all of the Trivy types and provides a common set of metadata for all resources. This includes the file and line number where the resource was defined and the name of the resource. -A resource, in this example `Project`, can have a name, and can optionally be encrypted. Instead of using raw string and bool types respectively, we use the trivy types `iacTypes.Metadata` and `iacTypes.BoolValue`. These types wrap the raw values and provide additional metadata about the value, such as whether it was set by the user or not, and the file and line number where the resource was defined. +A resource in this example `Project` can have a name and can optionally be encrypted. Instead of using raw string and bool types respectively, we use the trivy types `iacTypes.Metadata` and `iacTypes.BoolValue`. These types wrap the raw values and provide additional metadata about the value. For instance, whether it was set by the user and the file and line number where the resource was defined. Have a look at the other providers and services in the [`iac/providers`](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) directory in Trivy. From c89c92c486f8990504f09e00dbc0ef998db7cd06 Mon Sep 17 00:00:00 2001 From: Anais Urlichs <33576047+AnaisUrlichs@users.noreply.github.com> Date: Wed, 6 Mar 2024 09:21:09 +0000 Subject: [PATCH 10/11] Update docs/docs/scanner/misconfiguration/custom/index.md Co-authored-by: simar7 <1254783+simar7@users.noreply.github.com> --- docs/docs/scanner/misconfiguration/custom/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/scanner/misconfiguration/custom/index.md b/docs/docs/scanner/misconfiguration/custom/index.md index a3540af235ed..7ddaabc5b726 100644 --- a/docs/docs/scanner/misconfiguration/custom/index.md +++ b/docs/docs/scanner/misconfiguration/custom/index.md @@ -144,7 +144,7 @@ correct and do not reference incorrect properties/values. The AVD_ID can be used to link the check to the Aqua Vulnerability Database (AVD) entry. For example, the `avd_id` `AVD-AWS-0176` is the ID of the check in the [AWS Vulnerability Database](https://avd.aquasec.com/). If you are [contributing your check to trivy-policies](../../../../community/contribute/checks/overview.md), you need to generate an ID using `make id` in the [trivy-policies](https://github.com/aquasecurity/trivy-policies) repository. The output of the command will provide you the next free IDs for the different providers in Trivy. -The ID is based on the AVD_ID. For instanc,e if the `avd_id` is `AVD-AWS-0176`, the ID is `ID0176`. +The ID is based on the AVD_ID. For instance if the `avd_id` is `AVD-AWS-0176`, the ID is `ID0176`. #### custom.provider From c9255cd05435ada83fa3f453918e6ead53126c11 Mon Sep 17 00:00:00 2001 From: AnaisUrlichs Date: Thu, 7 Mar 2024 13:13:12 +0000 Subject: [PATCH 11/11] docs: revert changes Signed-off-by: AnaisUrlichs --- docs/community/contribute/checks/overview.md | 2 +- .../scanner/misconfiguration/custom/index.md | 46 ++++++++++--------- 2 files changed, 26 insertions(+), 22 deletions(-) diff --git a/docs/community/contribute/checks/overview.md b/docs/community/contribute/checks/overview.md index faf8be9d33c2..5706c5a02a9c 100644 --- a/docs/community/contribute/checks/overview.md +++ b/docs/community/contribute/checks/overview.md @@ -50,7 +50,7 @@ deny[res] { ## Verify the provider and service exists -Every check for a cloud service references a cloud provider. The list of providers are found in the [trivy](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) repository. +Every check for a cloud service references a cloud provider. The list of providers are found in the [Trivy](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) repository. Before writing a new check for a cloud provider, you need to verify if the cloud provider or resource type that your check targets is supported by Trivy. If it's not, you'll need to add support for it. Additionally, if the provider that you want to target exists, you need to check whether the service your policy will target is supported. As a reference you can take a look at the AWS provider [here](https://github.com/aquasecurity/trivy/blob/main/pkg/iac/providers/aws/aws.go). diff --git a/docs/docs/scanner/misconfiguration/custom/index.md b/docs/docs/scanner/misconfiguration/custom/index.md index 7ddaabc5b726..1d8236ae8094 100644 --- a/docs/docs/scanner/misconfiguration/custom/index.md +++ b/docs/docs/scanner/misconfiguration/custom/index.md @@ -63,7 +63,7 @@ If you add a new custom policy, it must be defined under a new package like `use `# METADATA` (optional unless the check will be contributed into Trivy) : - SHOULD be defined for clarity since these values will be displayed in the scan results - - `custom.input` SHOULD be set to indicate the input type the policy should be applied to. See [list of available types](https://github.com/aquasecurity/defsec/blob/418759b4dc97af25f30f32e0bd365be7984003a1/pkg/types/sources.go) + - `custom.input` SHOULD be set to indicate the input type the policy should be applied to. See [list of available types][source-types] `package` (required) : - MUST follow the Rego's [specification][package] @@ -123,22 +123,26 @@ If you are creating checks for your Trivy misconfiguration scans, some fields ar schema that will be used is based on the input document type. It is recommended to use this to ensure your policies are correct and do not reference incorrect properties/values. -| Field name | Allowed values | Description | Default value | In table | In JSON | Required in Trivy Check | -|----------------------------|-------------------------------------------------------------------|:-------------------------------------------------------:|:-----------------------------:|:----------------:|:----------------:| :-----------------------------: | -| title | Any characters | Name of the policy | N/A | :material-check: | :material-check: | :material-check: | -| description | Any characters | Description of the problem | N/A | :material-close: | :material-check: | :material-check: | -| schemas.input | `schema["kubernetes"]`, `schema["dockerfile"]`, `schema["cloud"]` | Used to validate the policy for syntax errors | (applied to all input types) | :material-close: | :material-close: | :material-check: | -| scope | `package` | Defines the policy scope | (applied to all input types) | :material-close: | :material-check: | :material-check: | -| custom.id | Any characters | More information provided below | N/A | :material-close: | :material-check: | :material-check: | -| custom.avd_id | Any characters | More information provided below | N/A | :material-close: | :material-check: | :material-check: | -| custom.severity | `LOW`, `MEDIUM`, `HIGH`, `CRITICAL` | The severity of the misconfiguration | `MEDIUM` | :material-close: | :material-check: | :material-check: | -| custom.provider | Any provider available in Trivy | The resource provider this check relates to | N/A | :material-close: | :material-check: | :material-close: | -| custom.service | The service name of the provider | One of the services available in the provider | N/A | :material-close: | :material-check: | :material-close: | -| custom.short_code | Any characters | Descriptive name for the check | N/A | :material-close: | :material-check: | :material-check: | -| custom.subtypes | Cloud Resource Subtypes | Refer to the section on subtypes below | N/A | :material-close: | :material-check: | :material-close: | -| custom.recommended_actions | Any characters | Describing what the user should do to resolve the issue | N/A | :material-close: | :material-check: | :material-check: | -| custom.input.selector.type | Any item(s) in [this list][source-types] | More information provided below | N/A | :material-close: | :material-check: | :material-check: | -| related_resource | Any characters | URL to related resources | N/A | :material-close: | :material-check: | :material-check: | +| Field name | Allowed values | Description | Required in Trivy Check | Default value | In table | In JSON | +|----------------------------|-------------------------------------------------------------------|:-------------------------------------------------------:|:------------------------------:|:-----------------------------:|:----------------:|:----------------:| +| title | Any characters | Name of the policy | :material-check: | N/A | :material-check: | :material-check: | +| description | Any characters | Description of the problem | :material-check: | N/A | :material-close: | :material-check: | +| schemas.input | `schema["One of the available schemas"]` | Used to validate the policy for syntax errors | :material-close: | (applied to all input types) | :material-close: | :material-close: | +| scope | `package` | Defines the policy scope | :material-check: | (applied to all input types) | :material-close: | :material-check: | +| custom.id | Any characters | More information provided below | :material-check: | N/A | :material-close: | :material-check: | +| custom.avd_id | Any characters | More information provided below | :material-check: | N/A | :material-close: | :material-check: | +| custom.severity | `LOW`, `MEDIUM`, `HIGH`, `CRITICAL` | The severity of the misconfiguration | :material-check: | `MEDIUM` | :material-close: | :material-check: | +| custom.provider | Any provider available in Trivy | The resource provider this check relates to | :material-close: | N/A | :material-close: | :material-check: | +| custom.service | The service name of the provider | One of the services available in the provider | :material-close: | N/A | :material-close: | :material-check: | +| custom.short_code | Any characters | Descriptive name for the check | :material-check: | N/A | :material-close: | :material-check: | +| custom.subtypes | Cloud Resource Subtypes | Refer to the section on subtypes below | :material-close: | N/A | :material-close: | :material-check: | +| custom.recommended_actions | Any characters | Describing what the user should do to resolve the issue | :material-check: | N/A | :material-close: | :material-check: | +| custom.input.selector.type | Any item(s) in [this list][source-types] | More information provided below | :material-check: | N/A | :material-close: | :material-check: | +| related_resource | Any characters | URL to related resources | :material-check: | N/A | :material-close: | :material-check: | + +#### schemas.input + +The input schema is required for validating the checks for functional correctness (syntax). Any of the [schemas](./schema.md) can be referenced as the input. #### custom.avd_id and custom.id @@ -148,11 +152,11 @@ The ID is based on the AVD_ID. For instance if the `avd_id` is `AVD-AWS-0176`, t #### custom.provider -The `provider` field references the [provider](https://github.com/aquasecurity/defsec/tree/master/pkg/providers) available in Trivy. This should be the same as the provider name in the `pkg/iac/providers` directory, e.g. `aws`. +The `provider` field references the [provider](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers) available in Trivy. This should be the same as the provider name in the `pkg/iac/providers` directory, e.g. `aws`. #### custom.service -The `service` is the name of the service by the provider that the check targets. This should be the same as the service name in the `pkg/iac/providers` directory ([Link](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers)), e.g. `rds`. +Services are defined within a provider. For instance, RDS is a service and AWS is a provider. This should be the same as the service name in one of the provider directories. ([Link](https://github.com/aquasecurity/trivy/tree/main/pkg/iac/providers)), e.g. `aws/rds`. #### custom.input @@ -210,7 +214,7 @@ You can specify input format via the `custom.input` annotation. - `dockerfile` (Dockerfile) - `kubernetes` (Kubernetes YAML/JSON) - `rbac` (Kubernetes RBAC YAML/JSON) - - `cloud` (Cloud format, as defined by defsec - this is used for Terraform, CloudFormation, and Cloud/AWS scanning) + - `cloud` (Cloud format, as defined by Trivy - this is used for Terraform, CloudFormation, and Cloud/AWS scanning) - `yaml` (Generic YAML) - `json` (Generic JSON) - `toml` (Generic TOML) @@ -229,4 +233,4 @@ See [here](schema.md) for the detail. [rego]: https://www.openpolicyagent.org/docs/latest/policy-language/ [package]: https://www.openpolicyagent.org/docs/latest/policy-language/#packages -[source-types]: https://github.com/aquasecurity/defsec/blob/418759b4dc97af25f30f32e0bd365be7984003a1/pkg/types/sources.go +[source-types]: https://github.com/aquasecurity/trivy/blob/9361cdb7e28fd304d6fd2a1091feac64a6786672/pkg/iac/types/sources.go#L4