diff --git a/README.md b/README.md index 02974d3..a36080c 100644 --- a/README.md +++ b/README.md @@ -87,6 +87,8 @@ You can Test: You can test all this locally without worrying about AWS Credentials. +A number of these tests can be configured in a common way to apply to all templates through the use of the [hooks](./examples/unit/hooks/README.md) functionality. + ### Functional Testing This project is a wrapper around Taskcat. Taskcat is a great tool for ensuring your Cloudformation Template can be deployed in multiple AWS Regions. Cloud-Radar enhances Taskcat by making it easier to write more complete functional tests. @@ -170,7 +172,7 @@ The default values for pseudo parameters: | **StackId** | "" | | **StackName** | "" | | **URLSuffix** | "amazonaws.com" | -_Note: Bold variables are not fully impletmented yet see the [Roadmap](#roadmap)_ +_Note: Bold variables are not fully implemented yet see the [Roadmap](#roadmap)_ At the point of creating the `Template` instance additional configuration is required to be provided if you are using certain approaches to resolving values. diff --git a/examples/unit/hooks/README.md b/examples/unit/hooks/README.md new file mode 100644 index 0000000..8106809 --- /dev/null +++ b/examples/unit/hooks/README.md @@ -0,0 +1,99 @@ +# What do these examples cover? + +The two subdirectories here, `resources` and `template`, show the two types of "hooks" that are available for defining common checks. + +The "hooks" functionality allows you to configure standardised tests once (for example through your `pytest` `conftest.py` file, possibly even using functions from an external library), and then each test stack you create will have your tests applied to them at the point the stack is rendered. In future it is planned to support a plugin system to discover these hooks through installed packages to simplify this configuration. + +Hooks can be suppressed using CloudFormation MetaData (in a similar fashion to other tools like cfn-lint), at either the template or resource level. The expected format is as follows, where the items in the list are the function names of the configured hooks. + +``` + Metadata: + Cloud-Radar: + ignore-hooks: + - my_s3_encryption_hook +``` + +# Template Hooks + +Template hooks are evaluated at the point that the CloudFormation template is loaded in to Cloud-Radar by calling `Template.from_yaml` function. These are designed for performing template level checks that do not depend on the template having any processing performed on it for parameter/condition resolution etc. + +The types of scenarios this could be used for include: +* ensuring that all templates have some common parameters that are expected to be used for naming +* ensuring that all parameters have input validation configured + +A basic example is included in the [template/test_hooks_template.py](./template/test_hooks_template.py) file. + +## Defining a Hook + +Template hooks are functions that take in a single parameter, `template`. These are expected to raise an error if their check does not pass. + +```python +# Example hook that checks that the cloudformation template +# name for all parameters starts with a "p". +def my_parameter_prefix_checks(template: Template) -> None: + # Get all the parameters + parameters = template.template.get("Parameters", {}) + + # Check them + _object_prefix_check(parameters, "p") + +def _object_prefix_check(items: List[str], expected_prefix: str): + # Iterate through each parameter checking them + for item in items: + if not item.startswith(expected_prefix): + raise ValueError( + f"{item} does not follow the convention of starting with '{expected_prefix}'" + ) +``` + +The name of your function is used as the hook name in assertion messages and for the purposes of suppressions, so you should try to keep them unique within your code. + +## Configuring a Hook + +This type of hooks are set as a list of functions on the Template object. + +``` +Template.Hooks.template = [ my_parameter_prefix_checks ] +``` + +# Resource Hooks + +Resource hooks are evaluated at the point of the stack being rendered by calling `template.create_stack`. These hooks are designed to be able to check aspects of the template *after* items like parameter substitution and conditions have been applied. + +These are intended for much more in depth checks, specific to the type of a resource. The initial inception of this feature was to support naming convention checks, but equally can be expanded into the sorts of compliance tests that [CloudFormation Guard](https://docs.aws.amazon.com/cfn-guard/latest/ug/what-is-guard.html) is used for if you prefer to code rules in Python as opposed to the Guard DSL. + +A basic example is included in the [resources/test_hooks_resources.py](./resources/test_hooks_resources.py) file. + +## Defining a Hook + +Resource hooks are functions that take in a single `context` parameter. This is a `ResourceHookContext` object that contains the following: + +``` + logical_id: str + resource_definition: Resource + stack: Stack + template: "Template" +``` + +These type of hooks are expected to raise an error if their check does not pass. + +As a basic compliance type example, this hook verifies that the encryption property is present for an S3 resource. + +```python +# Example hook that verifies that all S3 bucket definitions +# have the "BucketEncryption" property set +def my_s3_encryption_hook(context: ResourceHookContext) -> None: + # Use one of the built in functions to confirm the property exists + context.resource_definition.assert_has_property("BucketEncryption") +``` + +This is configured as a hook against the S3 resource type by + +```python +Template.Hooks.resources = { + "AWS::S3::Bucket": [my_s3_naming_hook, my_s3_encryption_hook] +} + +``` + +In this setup a dict is set with the AWS Resource type as the key, and a list of functions for the value. diff --git a/examples/unit/hooks/resources/naming_resources.yaml b/examples/unit/hooks/resources/naming_resources.yaml new file mode 100644 index 0000000..cfac955 --- /dev/null +++ b/examples/unit/hooks/resources/naming_resources.yaml @@ -0,0 +1,52 @@ +AWSTemplateFormatVersion: 2010-09-09 +Description: 'Basic template containing test resources' +Parameters: + pName: + Type: String + Description: 'A string included in resource names' +Resources: + rS3Bucket: + Type: 'AWS::S3::Bucket' + DeletionPolicy: Retain + UpdateReplacePolicy: Retain + Properties: + BucketName: !Sub "${pName}-${AWS::Region}-bucket" + BucketEncryption: + ServerSideEncryptionConfiguration: + - ServerSideEncryptionByDefault: + SSEAlgorithm: AES256 + + + rSampleBucketPolicy: + Type: AWS::S3::BucketPolicy + Properties: + Bucket: !Ref rS3Bucket + PolicyDocument: + Version: 2012-10-17 + Statement: + - Action: + - 's3:GetObject' + Effect: Allow + Resource: !Join + - '' + - - 'arn:aws:s3:::' + - !Ref rS3Bucket + - /* + Principal: '*' + Condition: + StringLike: + 'aws:Referer': + - 'http://www.example.com/*' + - 'http://example.net/*' + + rFileSystem: + Type: "AWS::EFS::FileSystem" + Properties: + PerformanceMode: "generalPurpose" + FileSystemTags: + - Key: "Name" + Value: !Sub "my-${pName}-${AWS::Region}-vol" +Outputs: + oBucket: + Description: S3 Bucket Name + Value: !Ref rS3Bucket diff --git a/examples/unit/hooks/resources/naming_resources_no_encryption.yaml b/examples/unit/hooks/resources/naming_resources_no_encryption.yaml new file mode 100644 index 0000000..d655164 --- /dev/null +++ b/examples/unit/hooks/resources/naming_resources_no_encryption.yaml @@ -0,0 +1,46 @@ +AWSTemplateFormatVersion: 2010-09-09 +Description: 'Basic template containing test resources' +Parameters: + pName: + Type: String + Description: 'A string included in resource names' +Resources: + rS3Bucket: + Type: 'AWS::S3::Bucket' + Properties: + BucketName: !Sub "${pName}-${AWS::Region}-bucket" + DeletionPolicy: Retain + + rSampleBucketPolicy: + Type: AWS::S3::BucketPolicy + Properties: + Bucket: !Ref rS3Bucket + PolicyDocument: + Version: 2012-10-17 + Statement: + - Action: + - 's3:GetObject' + Effect: Allow + Resource: !Join + - '' + - - 'arn:aws:s3:::' + - !Ref rS3Bucket + - /* + Principal: '*' + Condition: + StringLike: + 'aws:Referer': + - 'http://www.example.com/*' + - 'http://example.net/*' + + rFileSystem: + Type: "AWS::EFS::FileSystem" + Properties: + PerformanceMode: "generalPurpose" + FileSystemTags: + - Key: "Name" + Value: !Sub "my-${pName}-${AWS::Region}-vol" +Outputs: + oBucket: + Description: S3 Bucket Name + Value: !Ref rS3Bucket diff --git a/examples/unit/hooks/resources/naming_resources_no_encryption_resource_suppression.yaml b/examples/unit/hooks/resources/naming_resources_no_encryption_resource_suppression.yaml new file mode 100644 index 0000000..0c9c6ad --- /dev/null +++ b/examples/unit/hooks/resources/naming_resources_no_encryption_resource_suppression.yaml @@ -0,0 +1,51 @@ +AWSTemplateFormatVersion: 2010-09-09 +Description: 'Basic template containing test resources' +Parameters: + pName: + Type: String + Description: 'A string included in resource names' +Resources: + rS3Bucket: + Type: 'AWS::S3::Bucket' + DeletionPolicy: Retain + UpdateReplacePolicy: Retain + Metadata: + Cloud-Radar: + ignore-hooks: + - my_s3_encryption_hook + Properties: + BucketName: !Sub "${pName}-${AWS::Region}-bucket" + + rSampleBucketPolicy: + Type: AWS::S3::BucketPolicy + Properties: + Bucket: !Ref rS3Bucket + PolicyDocument: + Version: 2012-10-17 + Statement: + - Action: + - 's3:GetObject' + Effect: Allow + Resource: !Join + - '' + - - 'arn:aws:s3:::' + - !Ref rS3Bucket + - /* + Principal: '*' + Condition: + StringLike: + 'aws:Referer': + - 'http://www.example.com/*' + - 'http://example.net/*' + + rFileSystem: + Type: "AWS::EFS::FileSystem" + Properties: + PerformanceMode: "generalPurpose" + FileSystemTags: + - Key: "Name" + Value: !Sub "my-${pName}-${AWS::Region}-vol" +Outputs: + oBucket: + Description: S3 Bucket Name + Value: !Ref rS3Bucket diff --git a/examples/unit/hooks/resources/naming_resources_no_encryption_resource_suppression_diff_rule.yaml b/examples/unit/hooks/resources/naming_resources_no_encryption_resource_suppression_diff_rule.yaml new file mode 100644 index 0000000..9a140f4 --- /dev/null +++ b/examples/unit/hooks/resources/naming_resources_no_encryption_resource_suppression_diff_rule.yaml @@ -0,0 +1,51 @@ +AWSTemplateFormatVersion: 2010-09-09 +Description: 'Basic template containing test resources' +Parameters: + pName: + Type: String + Description: 'A string included in resource names' +Resources: + rS3Bucket: + Type: 'AWS::S3::Bucket' + DeletionPolicy: Retain + UpdateReplacePolicy: Retain + Metadata: + Cloud-Radar: + ignore-hooks: + - a_different_check + Properties: + BucketName: !Sub "${pName}-${AWS::Region}-bucket" + + rSampleBucketPolicy: + Type: AWS::S3::BucketPolicy + Properties: + Bucket: !Ref rS3Bucket + PolicyDocument: + Version: 2012-10-17 + Statement: + - Action: + - 's3:GetObject' + Effect: Allow + Resource: !Join + - '' + - - 'arn:aws:s3:::' + - !Ref rS3Bucket + - /* + Principal: '*' + Condition: + StringLike: + 'aws:Referer': + - 'http://www.example.com/*' + - 'http://example.net/*' + + rFileSystem: + Type: "AWS::EFS::FileSystem" + Properties: + PerformanceMode: "generalPurpose" + FileSystemTags: + - Key: "Name" + Value: !Sub "my-${pName}-${AWS::Region}-vol" +Outputs: + oBucket: + Description: S3 Bucket Name + Value: !Ref rS3Bucket diff --git a/examples/unit/hooks/resources/naming_resources_no_encryption_template_suppression.yaml b/examples/unit/hooks/resources/naming_resources_no_encryption_template_suppression.yaml new file mode 100644 index 0000000..6c90964 --- /dev/null +++ b/examples/unit/hooks/resources/naming_resources_no_encryption_template_suppression.yaml @@ -0,0 +1,51 @@ +AWSTemplateFormatVersion: 2010-09-09 +Description: 'Basic template containing test resources' +Metadata: + Cloud-Radar: + ignore-hooks: + - my_s3_encryption_hook +Parameters: + pName: + Type: String + Description: 'A string included in resource names' +Resources: + rS3Bucket: + Type: 'AWS::S3::Bucket' + DeletionPolicy: Retain + UpdateReplacePolicy: Retain + Properties: + BucketName: !Sub "${pName}-${AWS::Region}-bucket" + + rSampleBucketPolicy: + Type: AWS::S3::BucketPolicy + Properties: + Bucket: !Ref rS3Bucket + PolicyDocument: + Version: 2012-10-17 + Statement: + - Action: + - 's3:GetObject' + Effect: Allow + Resource: !Join + - '' + - - 'arn:aws:s3:::' + - !Ref rS3Bucket + - /* + Principal: '*' + Condition: + StringLike: + 'aws:Referer': + - 'http://www.example.com/*' + - 'http://example.net/*' + + rFileSystem: + Type: "AWS::EFS::FileSystem" + Properties: + PerformanceMode: "generalPurpose" + FileSystemTags: + - Key: "Name" + Value: !Sub "my-${pName}-${AWS::Region}-vol" +Outputs: + oBucket: + Description: S3 Bucket Name + Value: !Ref rS3Bucket diff --git a/examples/unit/hooks/resources/naming_resources_no_encryption_template_suppression_diff_rule.yaml b/examples/unit/hooks/resources/naming_resources_no_encryption_template_suppression_diff_rule.yaml new file mode 100644 index 0000000..93743ac --- /dev/null +++ b/examples/unit/hooks/resources/naming_resources_no_encryption_template_suppression_diff_rule.yaml @@ -0,0 +1,51 @@ +AWSTemplateFormatVersion: 2010-09-09 +Description: 'Basic template containing test resources' +Metadata: + Cloud-Radar: + ignore-hooks: + - a_different_check +Parameters: + pName: + Type: String + Description: 'A string included in resource names' +Resources: + rS3Bucket: + Type: 'AWS::S3::Bucket' + DeletionPolicy: Retain + UpdateReplacePolicy: Retain + Properties: + BucketName: !Sub "${pName}-${AWS::Region}-bucket" + + rSampleBucketPolicy: + Type: AWS::S3::BucketPolicy + Properties: + Bucket: !Ref rS3Bucket + PolicyDocument: + Version: 2012-10-17 + Statement: + - Action: + - 's3:GetObject' + Effect: Allow + Resource: !Join + - '' + - - 'arn:aws:s3:::' + - !Ref rS3Bucket + - /* + Principal: '*' + Condition: + StringLike: + 'aws:Referer': + - 'http://www.example.com/*' + - 'http://example.net/*' + + rFileSystem: + Type: "AWS::EFS::FileSystem" + Properties: + PerformanceMode: "generalPurpose" + FileSystemTags: + - Key: "Name" + Value: !Sub "my-${pName}-${AWS::Region}-vol" +Outputs: + oBucket: + Description: S3 Bucket Name + Value: !Ref rS3Bucket diff --git a/examples/unit/hooks/resources/test_hooks_resources.py b/examples/unit/hooks/resources/test_hooks_resources.py new file mode 100644 index 0000000..8459ae0 --- /dev/null +++ b/examples/unit/hooks/resources/test_hooks_resources.py @@ -0,0 +1,112 @@ +from pathlib import Path + +import pytest + +from cloud_radar.cf.unit import ResourceHookContext, Template + + +@pytest.fixture() +def configure_hooks(): + # Add in locally defined hooks + Template.Hooks.resources = { + "AWS::S3::Bucket": [my_s3_naming_hook, my_s3_encryption_hook] + } + + yield + + # Clear the hooks after + Template.Hooks.resources = {} + + +# Example hook that verifies that the rendered bucket name (after +# substitutions) includes the region the template is being deployed to. +def my_s3_naming_hook(context: ResourceHookContext) -> None: + name: str = context.resource_definition.get_property_value("BucketName") + + if context.template.Region not in name: + raise ValueError( + ( + f"{context.logical_id} - All buckets are " + "expected to include the region in their name" + ) + ) + + +# Example hook that verifies that all S3 bucket definitions +# have the "BucketEncryption" property set +def my_s3_encryption_hook(context: ResourceHookContext) -> None: + # Use one of the built in functions to confirm the property exists + context.resource_definition.assert_has_property("BucketEncryption") + + +# Helper method to load a template file relative to this test file +def load_template(filename: str): + template_path = Path(__file__).parent / filename + template = Template.from_yaml(template_path) + + return template + + +# Test case showing that when all hooks pass, no errors are raised. +@pytest.mark.usefixtures("configure_hooks") +def test_basic_all_success(): + template = load_template("naming_resources.yaml") + + # In these cases I'll usually use a non-existant region to ensure a real region + # is not hard coded + + # Render the stack, this will execute the resource level hooks + template.create_stack(params={"pName": "test"}, region="xx-west-3") + + +# Test case showing that when a resource hook causes an error, +# we see it at the point of creating the rendered stack +@pytest.mark.usefixtures("configure_hooks") +def test_basic_resource_failure(): + template = load_template("naming_resources_no_encryption.yaml") + + # Render the stack, this will execute the resource level hooks + with pytest.raises( + AssertionError, match="Resource 'rS3Bucket' has no property BucketEncryption." + ): + template.create_stack(params={"pName": "test"}, region="xx-west-3") + + +# The contents of this test template are mostly the same as the one we used for +# test_basic_resource_failure, but this time we have the appropriate Metadata +# at the Template level to ignore the resource hook that was failing +@pytest.mark.usefixtures("configure_hooks") +def test_template_suppression_success(): + template = load_template("naming_resources_no_encryption_template_suppression.yaml") + + template.create_stack(params={"pName": "test"}, region="xx-west-3") + + +# This test is similar to test_template_suppression_success, +# but shows that if the hook being ignored is different from the +# failing one that we still see the failure. +@pytest.mark.usefixtures("configure_hooks") +def test_template_suppression_diff_hook(): + template = load_template( + "naming_resources_no_encryption_template_suppression_diff_rule.yaml" + ) + + with pytest.raises( + AssertionError, match="Resource 'rS3Bucket' has no property BucketEncryption." + ): + template.create_stack(params={"pName": "test"}, region="xx-west-3") + + +# The contents of this test template are mostly the same as the one we used for +# test_basic_resource_failure, but this time we have the appropriate Metadata +# at the Resource level to ignore the resource hook that was failing +@pytest.mark.usefixtures("configure_hooks") +def test_resource_suppression_success(): + template = load_template( + "naming_resources_no_encryption_resource_suppression_diff_rule.yaml" + ) + + with pytest.raises( + AssertionError, match="Resource 'rS3Bucket' has no property BucketEncryption." + ): + template.create_stack(params={"pName": "test"}, region="xx-west-3") diff --git a/examples/unit/hooks/template/naming_resources.yaml b/examples/unit/hooks/template/naming_resources.yaml new file mode 100644 index 0000000..0fe73ef --- /dev/null +++ b/examples/unit/hooks/template/naming_resources.yaml @@ -0,0 +1,29 @@ +AWSTemplateFormatVersion: 2010-09-09 +Description: 'Basic template containing test resources' +Parameters: + pName: + Type: String + Description: 'A string included in resource names' +Resources: + rS3Bucket: + Type: 'AWS::S3::Bucket' + DeletionPolicy: Retain + UpdateReplacePolicy: Retain + Properties: + BucketName: !Sub "${pName}-${AWS::Region}-bucket" + BucketEncryption: + ServerSideEncryptionConfiguration: + - ServerSideEncryptionByDefault: + SSEAlgorithm: AES256 + + rFileSystem: + Type: "AWS::EFS::FileSystem" + Properties: + PerformanceMode: "generalPurpose" + FileSystemTags: + - Key: "Name" + Value: !Sub "my-${pName}-${AWS::Region}-vol" +Outputs: + oBucket: + Description: S3 Bucket Name + Value: !Ref rS3Bucket diff --git a/examples/unit/hooks/template/naming_resources_failure.yaml b/examples/unit/hooks/template/naming_resources_failure.yaml new file mode 100644 index 0000000..ec707f6 --- /dev/null +++ b/examples/unit/hooks/template/naming_resources_failure.yaml @@ -0,0 +1,29 @@ +AWSTemplateFormatVersion: 2010-09-09 +Description: 'Basic template containing test resources' +Parameters: + Name: + Type: String + Description: 'A string included in resource names' +Resources: + rS3Bucket: + Type: 'AWS::S3::Bucket' + DeletionPolicy: Retain + UpdateReplacePolicy: Retain + Properties: + BucketName: !Sub "${Name}-${AWS::Region}-bucket" + BucketEncryption: + ServerSideEncryptionConfiguration: + - ServerSideEncryptionByDefault: + SSEAlgorithm: AES256 + + rFileSystem: + Type: "AWS::EFS::FileSystem" + Properties: + PerformanceMode: "generalPurpose" + FileSystemTags: + - Key: "Name" + Value: !Sub "my-${Name}-${AWS::Region}-vol" +Outputs: + oBucket: + Description: S3 Bucket Name + Value: !Ref rS3Bucket diff --git a/examples/unit/hooks/template/test_hooks_template.py b/examples/unit/hooks/template/test_hooks_template.py new file mode 100644 index 0000000..5d402e3 --- /dev/null +++ b/examples/unit/hooks/template/test_hooks_template.py @@ -0,0 +1,63 @@ +from pathlib import Path +from typing import List + +import pytest + +from cloud_radar.cf.unit import ResourceHookContext, Template + + +@pytest.fixture() +def configure_hooks(): + # Add in locally defined template level hooks + Template.Hooks.template = [my_parameter_prefix_checks] + + yield + + # Clear the hooks after + Template.Hooks.template = [] + + +def _object_prefix_check(items: List[str], expected_prefix: str): + # Iterate through each parameter checking them + for item in items: + if not item.startswith(expected_prefix): + raise ValueError( + f"{item} does not follow the convention of starting with '{expected_prefix}'" + ) + + +# Example hook that checks that the cloudformation template +# name for all parameters starts with a "p". +def my_parameter_prefix_checks(template: Template) -> None: + # Get all the parameters + parameters = template.template.get("Parameters", {}) + + # Check them + _object_prefix_check(parameters, "p") + + +# Helper method to load a template file relative to this test file +def load_template(filename: str): + template_path = Path(__file__).parent / filename + template = Template.from_yaml(template_path) + + return template + + +# Test case showing that when all hooks pass, no errors are raised. +@pytest.mark.usefixtures("configure_hooks") +def test_basic_all_success(): + # Loading the template will validate the template + # level hooks + load_template("naming_resources.yaml") + + +# Test case showing that when a template hook causes an error, +# we see it at the point of loading the template +@pytest.mark.usefixtures("configure_hooks") +def test_basic_failure(): + # Loading the template will cause the failure + with pytest.raises( + ValueError, match="Name does not follow the convention of starting with 'p'" + ): + load_template("naming_resources_failure.yaml") diff --git a/examples/unit/naming/naming_resources.yaml b/examples/unit/naming/naming_resources.yaml index d655164..ea77d11 100644 --- a/examples/unit/naming/naming_resources.yaml +++ b/examples/unit/naming/naming_resources.yaml @@ -9,6 +9,10 @@ Resources: Type: 'AWS::S3::Bucket' Properties: BucketName: !Sub "${pName}-${AWS::Region}-bucket" + BucketEncryption: + ServerSideEncryptionConfiguration: + - ServerSideEncryptionByDefault: + SSEAlgorithm: AES256 DeletionPolicy: Retain rSampleBucketPolicy: diff --git a/src/cloud_radar/cf/unit/__init__.py b/src/cloud_radar/cf/unit/__init__.py index 141492f..1eaea3d 100644 --- a/src/cloud_radar/cf/unit/__init__.py +++ b/src/cloud_radar/cf/unit/__init__.py @@ -1,4 +1,6 @@ +from ._hooks import ResourceHookContext +from ._resource import Resource from ._stack import Stack from ._template import Template -__all__ = ["Template", "Stack"] +__all__ = ["Template", "Stack", "Resource", "ResourceHookContext"] diff --git a/src/cloud_radar/cf/unit/_hooks.py b/src/cloud_radar/cf/unit/_hooks.py new file mode 100644 index 0000000..4482b28 --- /dev/null +++ b/src/cloud_radar/cf/unit/_hooks.py @@ -0,0 +1,199 @@ +from dataclasses import dataclass + +# Work around some circular import issue until someone smarter +# can work out the right way to restructure / refactor this +# Solution from https://stackoverflow.com/a/39757388/230449 +from typing import TYPE_CHECKING, Callable, Dict, List, Union + +from ._resource import Resource +from ._stack import Stack + +if TYPE_CHECKING: + from ._template import Template + + +@dataclass +class ResourceHookContext: + """Class that contains the context for a resource hook to evaluate. + + Attributes: + logical_id (str): The logical ID for the resource in the CloudFormation + template which is to be evaluated. + resource_definition (Resource): The definition of the resource to be evaluated. + stack (Stack): the rendered stack that the resource is part of + template (Template): the template that is being rendered to produce the stack + """ + + logical_id: str + resource_definition: Resource + stack: Stack + template: "Template" + + +@dataclass +class ResourceHookCollection: + """ + Class that holds the two collections of hooks that we can evaluate against + individual Resource items in a rendered stack. + + Each Callable is expected to take in a single parameter - an instance of + ResourceHookContext. + + Attributes: + plugin (Dict[str, List[Callable]]): The dict of Resource Type to + list of hooks which were loaded from plugins. + local (Dict[str, List[Callable]]): The dict of Resource Type to + list of hooks which were defined with the template. + """ + + plugin: Dict[str, List[Callable]] + local: Dict[str, List[Callable]] + + +@dataclass +class TemplateHookCollection: + """ + Class that holds the two collections of hooks that we can evaluate + against a loaded Template. + + Each Callable is expected to take in a single parameter - an + instance of Template. + + Attributes: + plugin (List[Callable]): The list of hooks which were loaded from plugins. + local (List[Callable]): The list of hooks which were defined with the template. + """ + + plugin: List[Callable] + local: List[Callable] + + +class HookProcessor: + """ + Class that handles holding and evaluating the collections of hooks that we run + against Templates and Resources. + + In future this will include loading hooks from plugins, but that has not been + implemented yet. + + This supports suppressing rules based on Metadata at either the Template or + Resource level, using something like this: + + Metadata: + Cloud-Radar: + ignore-hooks: + - s3_check_bucket_name_region + + To allow this to work, you should ensure that the function names that you implement + for hooks have unique and descriptive names. + + """ + + def __init__(self) -> None: + self._resources: ResourceHookCollection = ResourceHookCollection( + plugin={}, local={} + ) + self._template: TemplateHookCollection = TemplateHookCollection( + plugin=[], local=[] + ) + # TODO: Add support for loading plugin hooks + + @property + def template(self): + return self._template.local + + @template.setter + def template(self, value: List[Callable]): + self._template.local = value + + @property + def resources(self): + return self._resources.local + + @resources.setter + def resources(self, value: Dict[str, List[Callable]]): + self._resources.local = value + + def _is_hook_suppressed_in_dict(self, hook_name: str, metadata: Dict) -> bool: + cloud_radar_metadata = metadata.get("Cloud-Radar", {}) + ignored_hooks = cloud_radar_metadata.get("ignore-hooks", {}) + + return hook_name in ignored_hooks + + def _is_hook_suppressed( + self, hook_name, template: "Template", resource: Union[Resource, None] = None + ) -> bool: + # First check if a suppression exists in the template, + # as that will take precedence. + hook_suppressed = self._is_hook_suppressed_in_dict( + hook_name, template.template.get("Metadata", {}) + ) + + if not hook_suppressed and resource: + # if not suppression was found at the template level, check the + # resource level + hook_suppressed = self._is_hook_suppressed_in_dict( + hook_name, resource.get("Metadata", {}) + ) + + return hook_suppressed + + def _evaluate_template_hooks( + self, hook_type: str, hooks: List[Callable], template: "Template" + ) -> None: + for single_hook in hooks: + # Only process the hook if it has not been marked as to be + # ignored + if not self._is_hook_suppressed(single_hook.__name__, template, None): + print(f"Processing {hook_type} hook {single_hook.__name__}") + + single_hook(template=template) + + def _evaluate_resource_hooks( + self, + hook_type: str, + hooks: Dict[str, List[Callable]], + stack: Stack, + template: "Template", + ) -> None: + # Iterate through the resources in the rendered stack + for logical_id in stack.data.get("Resources", {}): + print("Got resource " + logical_id) + resource_definition = stack.get_resource(logical_id) + + resource_type = resource_definition.get("Type") + + # Get the hooks that have been defined for this type of resource + type_hooks = hooks.get(resource_type, []) + + hook_context = ResourceHookContext( + logical_id=logical_id, + resource_definition=resource_definition, + stack=stack, + template=template, + ) + + # Iterate through each defined hook and call them. + for single_hook in type_hooks: + # Only process the hook if it has not been marked as to be + # ignored + if not self._is_hook_suppressed( + single_hook.__name__, template, resource_definition + ): + print(f"Processing {hook_type} hook {single_hook.__name__}") + + single_hook(context=hook_context) + + def evaluate_resource_hooks(self, stack: Stack, template: "Template") -> None: + # Evaluate the global hooks first, then the local ones + self._evaluate_resource_hooks("plugin", self._resources.plugin, stack, template) + self._evaluate_resource_hooks("local", self._resources.local, stack, template) + + def evaluate_template_hooks(self, template: "Template") -> None: + print(template) + # raise ValueError(type(template)) + # raise ValueError(template.template) + + # Evaluate the global hooks first, then the local ones + self._evaluate_template_hooks("plugin", self._template.plugin, template) + self._evaluate_template_hooks("local", self._template.local, template) diff --git a/src/cloud_radar/cf/unit/_template.py b/src/cloud_radar/cf/unit/_template.py index e707828..c8899df 100644 --- a/src/cloud_radar/cf/unit/_template.py +++ b/src/cloud_radar/cf/unit/_template.py @@ -9,6 +9,7 @@ from cfn_tools import dump_yaml, load_yaml # type: ignore # noqa: I100, I201 from . import functions +from ._hooks import HookProcessor from ._stack import Stack IntrinsicFunc = Callable[["Template", Any], Any] @@ -20,6 +21,7 @@ class Template: """ AccountId: str = "5" * 12 + Hooks = HookProcessor() NotificationARNs: list = [] NoValue: str = "" # Not yet implemented Partition: str = "aws" # Other regions not implemented @@ -79,6 +81,10 @@ def __init__( "Transform", None ) + # All loaded, validate against any template level hooks + # that have been configured + self.Hooks.evaluate_template_hooks(self) + @classmethod def from_yaml( cls, @@ -317,6 +323,9 @@ def create_stack( stack = Stack(self.template) + # Evaluate any hooks prior to returning this stack + self.Hooks.evaluate_resource_hooks(stack, self) + return stack def resolve_values( # noqa: max-complexity: 13 @@ -789,7 +798,7 @@ def validate_string_parameter_constraints( def add_metadata(template: Dict, region: str) -> None: """This functions adds the current region to the template - as metadate because we can't treat Region like a normal pseduo + as metadata because we can't treat Region like a normal pseudo variables because we don't want to update the class var for every run. Args: @@ -797,12 +806,16 @@ def add_metadata(template: Dict, region: str) -> None: region (str): The region that template will be tested with. """ - metadata = {"Cloud-Radar": {"Region": region}} - if "Metadata" not in template: template["Metadata"] = {} - template["Metadata"].update(metadata) + # Get the existing metadata (so we do not overwrite any + # hook suppressions), then set the region into it before + # updating the template + cloud_radar_metadata = template["Metadata"].get("Cloud-Radar", {}) + cloud_radar_metadata["Region"] = region + + template["Metadata"]["Cloud-Radar"] = cloud_radar_metadata # All the other Cloudformation intrinsic functions start with `Fn:` but for some reason