Skip to content

Commit

Permalink
Adding support for opensearch domains (#144)
Browse files Browse the repository at this point in the history
* adding initial support for opensearch domains

* Auto Format

* Fix var validation

* Updating outputs

* Auto Format

* Convert log options to dynamic blocks

* Removing restricition that prevents dev single instance instances

* Adding anonymous_iam_actions var/support

* Auto Format

* Adding support for additional policies

* Fixing additional_policy_documents var type

* Auto Format

* Ading policy sid

* Auto Format

* source_policy_documents

* Adding overrides for policy statements

* Auto Format

* Removing additional policy statements

* Auto Format

* update readme

* deduplicate domain

* terraform fmt

* update tflint

* Update elasticsearch_domain.tf

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* add new example, revert to join("" to avoid type changes during this update.

* Update elasticsearch_domain.tf

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* cleanup unused provider

* update readme and docs

* update readme

---------

Co-authored-by: Steven Hopkins <steve@cloudposse.com>
Co-authored-by: cloudpossebot <11232728+cloudpossebot@users.noreply.github.com>
Co-authored-by: Benjamin Smith <ben.smith.developer@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
  • Loading branch information
5 people authored Feb 13, 2024
1 parent 1285ccd commit 9f8894c
Show file tree
Hide file tree
Showing 10 changed files with 468 additions and 348 deletions.
279 changes: 94 additions & 185 deletions README.md

Large diffs are not rendered by default.

7 changes: 5 additions & 2 deletions docs/terraform.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.3 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.35.0 |
| <a name="requirement_null"></a> [null](#requirement\_null) | >= 2.0 |

## Providers

Expand All @@ -31,6 +30,8 @@
| [aws_elasticsearch_domain_policy.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticsearch_domain_policy) | resource |
| [aws_iam_role.elasticsearch_user](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_service_linked_role.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_service_linked_role) | resource |
| [aws_opensearch_domain.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/opensearch_domain) | resource |
| [aws_opensearch_domain_policy.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/opensearch_domain_policy) | resource |
| [aws_security_group.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
| [aws_security_group_rule.egress](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
| [aws_security_group_rule.ingress_cidr_blocks](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
Expand All @@ -50,11 +51,13 @@
| <a name="input_advanced_security_options_master_user_name"></a> [advanced\_security\_options\_master\_user\_name](#input\_advanced\_security\_options\_master\_user\_name) | Master user username (applicable if advanced\_security\_options\_internal\_user\_database\_enabled set to true) | `string` | `""` | no |
| <a name="input_advanced_security_options_master_user_password"></a> [advanced\_security\_options\_master\_user\_password](#input\_advanced\_security\_options\_master\_user\_password) | Master user password (applicable if advanced\_security\_options\_internal\_user\_database\_enabled set to true) | `string` | `""` | no |
| <a name="input_allowed_cidr_blocks"></a> [allowed\_cidr\_blocks](#input\_allowed\_cidr\_blocks) | List of CIDR blocks to be allowed to connect to the cluster | `list(string)` | `[]` | no |
| <a name="input_anonymous_iam_actions"></a> [anonymous\_iam\_actions](#input\_anonymous\_iam\_actions) | List of actions to allow for the anonymous (`*`) IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` | `list(string)` | `[]` | no |
| <a name="input_attributes"></a> [attributes](#input\_attributes) | ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`,<br>in the order they appear in the list. New attributes are appended to the<br>end of the list. The elements of the list are joined by the `delimiter`<br>and treated as a single ID element. | `list(string)` | `[]` | no |
| <a name="input_auto_tune"></a> [auto\_tune](#input\_auto\_tune) | This object represents the auto\_tune configuration. It contains the following filed:<br>- enabled - Whether to enable autotune.<br>- rollback\_on\_disable - Whether to roll back to default Auto-Tune settings when disabling Auto-Tune.<br>- starting\_time - Date and time at which to start the Auto-Tune maintenance schedule in RFC3339 format. Time should be in the future.<br>- cron\_schedule - A cron expression specifying the recurrence pattern for an Auto-Tune maintenance schedule.<br>- duration - Autotune maintanance window duration time in hours. | <pre>object({<br> enabled = bool<br> rollback_on_disable = string<br> starting_time = string<br> cron_schedule = string<br> duration = number<br> })</pre> | <pre>{<br> "cron_schedule": null,<br> "duration": null,<br> "enabled": false,<br> "rollback_on_disable": "NO_ROLLBACK",<br> "starting_time": null<br>}</pre> | no |
| <a name="input_automated_snapshot_start_hour"></a> [automated\_snapshot\_start\_hour](#input\_automated\_snapshot\_start\_hour) | Hour at which automated snapshots are taken, in UTC | `number` | `0` | no |
| <a name="input_availability_zone_count"></a> [availability\_zone\_count](#input\_availability\_zone\_count) | Number of Availability Zones for the domain to use. | `number` | `2` | no |
| <a name="input_aws_ec2_service_name"></a> [aws\_ec2\_service\_name](#input\_aws\_ec2\_service\_name) | AWS EC2 Service Name | `list(string)` | <pre>[<br> "ec2.amazonaws.com"<br>]</pre> | no |
| <a name="input_aws_service_type"></a> [aws\_service\_type](#input\_aws\_service\_type) | The type of AWS service to deploy (`elasticsearch` or `opensearch`). | `string` | `"elasticsearch"` | no |
| <a name="input_cognito_authentication_enabled"></a> [cognito\_authentication\_enabled](#input\_cognito\_authentication\_enabled) | Whether to enable Amazon Cognito authentication with Kibana | `bool` | `false` | no |
| <a name="input_cognito_iam_role_arn"></a> [cognito\_iam\_role\_arn](#input\_cognito\_iam\_role\_arn) | ARN of the IAM role that has the AmazonESCognitoAccess policy attached | `string` | `""` | no |
| <a name="input_cognito_identity_pool_id"></a> [cognito\_identity\_pool\_id](#input\_cognito\_identity\_pool\_id) | The ID of the Cognito Identity Pool to use | `string` | `""` | no |
Expand Down Expand Up @@ -86,7 +89,7 @@
| <a name="input_encrypt_at_rest_enabled"></a> [encrypt\_at\_rest\_enabled](#input\_encrypt\_at\_rest\_enabled) | Whether to enable encryption at rest | `bool` | `true` | no |
| <a name="input_encrypt_at_rest_kms_key_id"></a> [encrypt\_at\_rest\_kms\_key\_id](#input\_encrypt\_at\_rest\_kms\_key\_id) | The KMS key ID to encrypt the Elasticsearch domain with. If not specified, then it defaults to using the AWS/Elasticsearch service KMS key | `string` | `""` | no |
| <a name="input_environment"></a> [environment](#input\_environment) | ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' | `string` | `null` | no |
| <a name="input_iam_actions"></a> [iam\_actions](#input\_iam\_actions) | List of actions to allow for the IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` | `list(string)` | `[]` | no |
| <a name="input_iam_actions"></a> [iam\_actions](#input\_iam\_actions) | List of actions to allow for the user IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` | `list(string)` | `[]` | no |
| <a name="input_iam_authorizing_role_arns"></a> [iam\_authorizing\_role\_arns](#input\_iam\_authorizing\_role\_arns) | List of IAM role ARNs to permit to assume the Elasticsearch user role | `list(string)` | `[]` | no |
| <a name="input_iam_role_arns"></a> [iam\_role\_arns](#input\_iam\_role\_arns) | List of IAM role ARNs to permit access to the Elasticsearch domain | `list(string)` | `[]` | no |
| <a name="input_iam_role_max_session_duration"></a> [iam\_role\_max\_session\_duration](#input\_iam\_role\_max\_session\_duration) | The maximum session duration (in seconds) for the user role. Can have a value from 1 hour to 12 hours | `number` | `3600` | no |
Expand Down
146 changes: 146 additions & 0 deletions elasticsearch_domain.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
#
# Elasticsearch Domain
#

resource "aws_elasticsearch_domain_policy" "default" {
count = local.elasticsearch_enabled && (length(var.iam_authorizing_role_arns) > 0 || length(var.iam_role_arns) > 0) ? 1 : 0
domain_name = module.this.id
access_policies = join("", data.aws_iam_policy_document.default[*].json)
}

resource "aws_elasticsearch_domain" "default" {
count = local.elasticsearch_enabled ? 1 : 0
domain_name = module.this.id
elasticsearch_version = var.elasticsearch_version

advanced_options = var.advanced_options

advanced_security_options {
enabled = var.advanced_security_options_enabled
internal_user_database_enabled = var.advanced_security_options_internal_user_database_enabled
master_user_options {
master_user_arn = var.advanced_security_options_master_user_arn
master_user_name = var.advanced_security_options_master_user_name
master_user_password = var.advanced_security_options_master_user_password
}
}

ebs_options {
ebs_enabled = var.ebs_volume_size > 0 ? true : false
volume_size = var.ebs_volume_size
volume_type = var.ebs_volume_type
iops = var.ebs_iops
throughput = var.ebs_throughput
}

encrypt_at_rest {
enabled = var.encrypt_at_rest_enabled
kms_key_id = var.encrypt_at_rest_kms_key_id
}

domain_endpoint_options {
enforce_https = var.domain_endpoint_options_enforce_https
tls_security_policy = var.domain_endpoint_options_tls_security_policy
custom_endpoint_enabled = var.custom_endpoint_enabled
custom_endpoint = var.custom_endpoint_enabled ? var.custom_endpoint : null
custom_endpoint_certificate_arn = var.custom_endpoint_enabled ? var.custom_endpoint_certificate_arn : null
}

cluster_config {
instance_count = var.instance_count
instance_type = var.instance_type
dedicated_master_enabled = var.dedicated_master_enabled
dedicated_master_count = var.dedicated_master_enabled ? var.dedicated_master_count : null
dedicated_master_type = var.dedicated_master_enabled ? var.dedicated_master_type : null
zone_awareness_enabled = var.zone_awareness_enabled
warm_enabled = var.warm_enabled
warm_count = var.warm_enabled ? var.warm_count : null
warm_type = var.warm_enabled ? var.warm_type : null

dynamic "zone_awareness_config" {
for_each = var.availability_zone_count > 1 && var.zone_awareness_enabled ? [true] : []
content {
availability_zone_count = var.availability_zone_count
}
}

dynamic "cold_storage_options" {
for_each = var.cold_storage_enabled ? [true] : []
content {
enabled = var.cold_storage_enabled
}
}
}

dynamic "auto_tune_options" {
for_each = var.auto_tune.enabled ? [true] : []
content {
desired_state = "ENABLED"
rollback_on_disable = var.auto_tune.rollback_on_disable
maintenance_schedule {
# Required until https://github.com/hashicorp/terraform-provider-aws/issues/22239 would be resolved
start_at = var.auto_tune.starting_time == null ? timeadd(timestamp(), "1h") : var.auto_tune.starting_time
duration {
value = var.auto_tune.duration
unit = "HOURS"
}
cron_expression_for_recurrence = var.auto_tune.cron_schedule
}
}
}

node_to_node_encryption {
enabled = var.node_to_node_encryption_enabled
}

dynamic "vpc_options" {
for_each = var.vpc_enabled ? [true] : []

content {
security_group_ids = var.create_security_group ? [join("", aws_security_group.default[*].id)] : var.security_groups
subnet_ids = var.subnet_ids
}
}

snapshot_options {
automated_snapshot_start_hour = var.automated_snapshot_start_hour
}

dynamic "cognito_options" {
for_each = var.cognito_authentication_enabled ? [true] : []
content {
enabled = true
user_pool_id = var.cognito_user_pool_id
identity_pool_id = var.cognito_identity_pool_id
role_arn = var.cognito_iam_role_arn
}
}

log_publishing_options {
enabled = var.log_publishing_index_enabled
log_type = "INDEX_SLOW_LOGS"
cloudwatch_log_group_arn = var.log_publishing_index_cloudwatch_log_group_arn
}

log_publishing_options {
enabled = var.log_publishing_search_enabled
log_type = "SEARCH_SLOW_LOGS"
cloudwatch_log_group_arn = var.log_publishing_search_cloudwatch_log_group_arn
}

log_publishing_options {
enabled = var.log_publishing_audit_enabled
log_type = "AUDIT_LOGS"
cloudwatch_log_group_arn = var.log_publishing_audit_cloudwatch_log_group_arn
}

log_publishing_options {
enabled = var.log_publishing_application_enabled
log_type = "ES_APPLICATION_LOGS"
cloudwatch_log_group_arn = var.log_publishing_application_cloudwatch_log_group_arn
}

tags = module.this.tags

depends_on = [aws_iam_service_linked_role.default]
}
28 changes: 28 additions & 0 deletions examples/opensearch_basic/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
provider "aws" {
region = "us-east-2"
}

module "opensearch" {
source = "../../"
namespace = "eg"
stage = "dev"
name = "es"
dns_zone_id = "Z14EN2YD427LRQ"
security_groups = ["sg-XXXXXXXXX", "sg-YYYYYYYY"]
vpc_id = "vpc-XXXXXXXXX"
subnet_ids = ["subnet-XXXXXXXXX", "subnet-YYYYYYYY"]
zone_awareness_enabled = "true"
aws_service_type = "opensearch"
elasticsearch_version = "OpenSearch_2.9"
instance_type = "t3.small.search"
instance_count = 4
ebs_volume_size = 10
iam_role_arns = ["arn:aws:iam::XXXXXXXXX:role/ops", "arn:aws:iam::XXXXXXXXX:role/dev"]
iam_actions = ["es:ESHttpGet", "es:ESHttpPut", "es:ESHttpPost"]
encrypt_at_rest_enabled = "true"
kibana_subdomain_name = "kibana-es"

advanced_options = {
"rest.action.multi.allow_explicit_index" = "true"
}
}
10 changes: 10 additions & 0 deletions examples/opensearch_basic/versions.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
terraform {
required_version = ">= 1.3"

required_providers {
aws = {
source = "hashicorp/aws"
version = ">= 4.0"
}
}
}
Loading

0 comments on commit 9f8894c

Please sign in to comment.