Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding support for opensearch domains #144

Merged
merged 34 commits into from
Feb 13, 2024
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
be01e9e
adding initial support for opensearch domains
Oct 14, 2022
1e9c385
Auto Format
cloudpossebot Oct 14, 2022
5b3b835
Fix var validation
Oct 14, 2022
922c5f0
Merge branch 'add-opensearch' of github.com:cloudposse/terraform-aws-…
Oct 14, 2022
5926636
Updating outputs
Oct 14, 2022
b732798
Auto Format
cloudpossebot Oct 14, 2022
e025ec3
Convert log options to dynamic blocks
Oct 14, 2022
c236629
Removing restricition that prevents dev single instance instances
Oct 14, 2022
80d0ccd
Adding anonymous_iam_actions var/support
Oct 17, 2022
a74671f
Auto Format
cloudpossebot Oct 17, 2022
3acf280
Adding support for additional policies
Oct 18, 2022
28e0753
Merge branch 'add-opensearch' of github.com:cloudposse/terraform-aws-…
Oct 18, 2022
9c26327
Fixing additional_policy_documents var type
Oct 18, 2022
461951b
Auto Format
cloudpossebot Oct 18, 2022
4ca0cb5
Ading policy sid
Oct 18, 2022
abb90e5
Auto Format
cloudpossebot Oct 18, 2022
5898808
source_policy_documents
Oct 18, 2022
14fc08b
Merge branch 'add-opensearch' of github.com:cloudposse/terraform-aws-…
Oct 18, 2022
6e392ea
Adding overrides for policy statements
Oct 18, 2022
70cbd6d
Auto Format
cloudpossebot Oct 18, 2022
e57142c
Removing additional policy statements
Oct 19, 2022
949c2f8
Merge branch 'add-opensearch' of github.com:cloudposse/terraform-aws-…
Oct 19, 2022
29c1c32
Auto Format
cloudpossebot Oct 19, 2022
78a927f
Merge branch 'main' into add-opensearch
Benbentwo Feb 12, 2024
54a7973
update readme
Benbentwo Feb 12, 2024
83bf182
deduplicate domain
Benbentwo Feb 12, 2024
43cbd9c
terraform fmt
Benbentwo Feb 12, 2024
72e3385
update tflint
Benbentwo Feb 12, 2024
d81ae34
Update elasticsearch_domain.tf
Benbentwo Feb 12, 2024
d3868b6
add new example, revert to join("" to avoid type changes during this …
Benbentwo Feb 12, 2024
c06c310
Update elasticsearch_domain.tf
Benbentwo Feb 12, 2024
f862976
cleanup unused provider
Benbentwo Feb 12, 2024
1bcd7b3
update readme and docs
Benbentwo Feb 12, 2024
f469cc4
update readme
Benbentwo Feb 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/auto-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ version-resolver:
- 'bugfix'
- 'bug'
- 'hotfix'
- 'no-release'
default: 'minor'

categories:
Expand Down
2 changes: 1 addition & 1 deletion .github/renovate.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
":preserveSemverRanges"
],
"labels": ["auto-update"],
"dependencyDashboardAutoclose": true,
"enabledManagers": ["terraform"],
"terraform": {
"ignorePaths": ["**/context.tf", "examples/**"]
}
}

1 change: 1 addition & 0 deletions .github/workflows/validate-codeowners.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ jobs:
steps:
- name: "Checkout source code at current commit"
uses: actions/checkout@v2
# Leave pinned at 0.7.1 until https://github.com/mszostok/codeowners-validator/issues/173 is resolved
- uses: mszostok/codeowners-validator@v0.7.1
if: github.event.pull_request.head.repo.full_name == github.repository
name: "Full check of CODEOWNERS"
Expand Down
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,8 @@ Available targets:
| [aws_elasticsearch_domain_policy.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticsearch_domain_policy) | resource |
| [aws_iam_role.elasticsearch_user](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_service_linked_role.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_service_linked_role) | resource |
| [aws_opensearch_domain.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/opensearch_domain) | resource |
| [aws_opensearch_domain_policy.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/opensearch_domain_policy) | resource |
| [aws_security_group.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
| [aws_security_group_rule.egress](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
| [aws_security_group_rule.ingress_cidr_blocks](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
Expand All @@ -213,10 +215,12 @@ Available targets:
| <a name="input_advanced_security_options_master_user_name"></a> [advanced\_security\_options\_master\_user\_name](#input\_advanced\_security\_options\_master\_user\_name) | Master user username (applicable if advanced\_security\_options\_internal\_user\_database\_enabled set to true) | `string` | `""` | no |
| <a name="input_advanced_security_options_master_user_password"></a> [advanced\_security\_options\_master\_user\_password](#input\_advanced\_security\_options\_master\_user\_password) | Master user password (applicable if advanced\_security\_options\_internal\_user\_database\_enabled set to true) | `string` | `""` | no |
| <a name="input_allowed_cidr_blocks"></a> [allowed\_cidr\_blocks](#input\_allowed\_cidr\_blocks) | List of CIDR blocks to be allowed to connect to the cluster | `list(string)` | `[]` | no |
| <a name="input_anonymous_iam_actions"></a> [anonymous\_iam\_actions](#input\_anonymous\_iam\_actions) | List of actions to allow for the anonymous (`*`) IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` | `list(string)` | `[]` | no |
| <a name="input_attributes"></a> [attributes](#input\_attributes) | ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`,<br>in the order they appear in the list. New attributes are appended to the<br>end of the list. The elements of the list are joined by the `delimiter`<br>and treated as a single ID element. | `list(string)` | `[]` | no |
| <a name="input_automated_snapshot_start_hour"></a> [automated\_snapshot\_start\_hour](#input\_automated\_snapshot\_start\_hour) | Hour at which automated snapshots are taken, in UTC | `number` | `0` | no |
| <a name="input_availability_zone_count"></a> [availability\_zone\_count](#input\_availability\_zone\_count) | Number of Availability Zones for the domain to use. | `number` | `2` | no |
| <a name="input_aws_ec2_service_name"></a> [aws\_ec2\_service\_name](#input\_aws\_ec2\_service\_name) | AWS EC2 Service Name | `list(string)` | <pre>[<br> "ec2.amazonaws.com"<br>]</pre> | no |
| <a name="input_aws_service_type"></a> [aws\_service\_type](#input\_aws\_service\_type) | The type of AWS service to deploy (`elasticsearch` or `opensearch`). | `string` | `"elasticsearch"` | no |
| <a name="input_cognito_authentication_enabled"></a> [cognito\_authentication\_enabled](#input\_cognito\_authentication\_enabled) | Whether to enable Amazon Cognito authentication with Kibana | `bool` | `false` | no |
| <a name="input_cognito_iam_role_arn"></a> [cognito\_iam\_role\_arn](#input\_cognito\_iam\_role\_arn) | ARN of the IAM role that has the AmazonESCognitoAccess policy attached | `string` | `""` | no |
| <a name="input_cognito_identity_pool_id"></a> [cognito\_identity\_pool\_id](#input\_cognito\_identity\_pool\_id) | The ID of the Cognito Identity Pool to use | `string` | `""` | no |
Expand Down Expand Up @@ -244,7 +248,7 @@ Available targets:
| <a name="input_encrypt_at_rest_enabled"></a> [encrypt\_at\_rest\_enabled](#input\_encrypt\_at\_rest\_enabled) | Whether to enable encryption at rest | `bool` | `true` | no |
| <a name="input_encrypt_at_rest_kms_key_id"></a> [encrypt\_at\_rest\_kms\_key\_id](#input\_encrypt\_at\_rest\_kms\_key\_id) | The KMS key ID to encrypt the Elasticsearch domain with. If not specified, then it defaults to using the AWS/Elasticsearch service KMS key | `string` | `""` | no |
| <a name="input_environment"></a> [environment](#input\_environment) | ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' | `string` | `null` | no |
| <a name="input_iam_actions"></a> [iam\_actions](#input\_iam\_actions) | List of actions to allow for the IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` | `list(string)` | `[]` | no |
| <a name="input_iam_actions"></a> [iam\_actions](#input\_iam\_actions) | List of actions to allow for the user IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` | `list(string)` | `[]` | no |
| <a name="input_iam_authorizing_role_arns"></a> [iam\_authorizing\_role\_arns](#input\_iam\_authorizing\_role\_arns) | List of IAM role ARNs to permit to assume the Elasticsearch user role | `list(string)` | `[]` | no |
| <a name="input_iam_role_arns"></a> [iam\_role\_arns](#input\_iam\_role\_arns) | List of IAM role ARNs to permit access to the Elasticsearch domain | `list(string)` | `[]` | no |
| <a name="input_iam_role_max_session_duration"></a> [iam\_role\_max\_session\_duration](#input\_iam\_role\_max\_session\_duration) | The maximum session duration (in seconds) for the user role. Can have a value from 1 hour to 12 hours | `number` | `3600` | no |
Expand Down Expand Up @@ -465,7 +469,7 @@ Check out [our other projects][github], [follow us on twitter][twitter], [apply

[![README Footer][readme_footer_img]][readme_footer_link]
[![Beacon][beacon]][website]

<!-- markdownlint-disable -->
[logo]: https://cloudposse.com/logo-300x69.svg
[docs]: https://cpco.io/docs?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/terraform-aws-elasticsearch&utm_content=docs
[website]: https://cpco.io/homepage?utm_source=github&utm_medium=readme&utm_campaign=cloudposse/terraform-aws-elasticsearch&utm_content=website
Expand Down Expand Up @@ -496,3 +500,4 @@ Check out [our other projects][github], [follow us on twitter][twitter], [apply
[share_googleplus]: https://plus.google.com/share?url=https://github.com/cloudposse/terraform-aws-elasticsearch
[share_email]: mailto:?subject=terraform-aws-elasticsearch&body=https://github.com/cloudposse/terraform-aws-elasticsearch
[beacon]: https://ga-beacon.cloudposse.com/UA-76589703-4/cloudposse/terraform-aws-elasticsearch?pixel&cs=github&cm=readme&an=terraform-aws-elasticsearch
<!-- markdownlint-restore -->
6 changes: 5 additions & 1 deletion docs/terraform.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@
| [aws_elasticsearch_domain_policy.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticsearch_domain_policy) | resource |
| [aws_iam_role.elasticsearch_user](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_service_linked_role.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_service_linked_role) | resource |
| [aws_opensearch_domain.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/opensearch_domain) | resource |
| [aws_opensearch_domain_policy.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/opensearch_domain_policy) | resource |
| [aws_security_group.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
| [aws_security_group_rule.egress](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
| [aws_security_group_rule.ingress_cidr_blocks](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
Expand All @@ -50,10 +52,12 @@
| <a name="input_advanced_security_options_master_user_name"></a> [advanced\_security\_options\_master\_user\_name](#input\_advanced\_security\_options\_master\_user\_name) | Master user username (applicable if advanced\_security\_options\_internal\_user\_database\_enabled set to true) | `string` | `""` | no |
| <a name="input_advanced_security_options_master_user_password"></a> [advanced\_security\_options\_master\_user\_password](#input\_advanced\_security\_options\_master\_user\_password) | Master user password (applicable if advanced\_security\_options\_internal\_user\_database\_enabled set to true) | `string` | `""` | no |
| <a name="input_allowed_cidr_blocks"></a> [allowed\_cidr\_blocks](#input\_allowed\_cidr\_blocks) | List of CIDR blocks to be allowed to connect to the cluster | `list(string)` | `[]` | no |
| <a name="input_anonymous_iam_actions"></a> [anonymous\_iam\_actions](#input\_anonymous\_iam\_actions) | List of actions to allow for the anonymous (`*`) IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` | `list(string)` | `[]` | no |
| <a name="input_attributes"></a> [attributes](#input\_attributes) | ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`,<br>in the order they appear in the list. New attributes are appended to the<br>end of the list. The elements of the list are joined by the `delimiter`<br>and treated as a single ID element. | `list(string)` | `[]` | no |
| <a name="input_automated_snapshot_start_hour"></a> [automated\_snapshot\_start\_hour](#input\_automated\_snapshot\_start\_hour) | Hour at which automated snapshots are taken, in UTC | `number` | `0` | no |
| <a name="input_availability_zone_count"></a> [availability\_zone\_count](#input\_availability\_zone\_count) | Number of Availability Zones for the domain to use. | `number` | `2` | no |
| <a name="input_aws_ec2_service_name"></a> [aws\_ec2\_service\_name](#input\_aws\_ec2\_service\_name) | AWS EC2 Service Name | `list(string)` | <pre>[<br> "ec2.amazonaws.com"<br>]</pre> | no |
| <a name="input_aws_service_type"></a> [aws\_service\_type](#input\_aws\_service\_type) | The type of AWS service to deploy (`elasticsearch` or `opensearch`). | `string` | `"elasticsearch"` | no |
| <a name="input_cognito_authentication_enabled"></a> [cognito\_authentication\_enabled](#input\_cognito\_authentication\_enabled) | Whether to enable Amazon Cognito authentication with Kibana | `bool` | `false` | no |
| <a name="input_cognito_iam_role_arn"></a> [cognito\_iam\_role\_arn](#input\_cognito\_iam\_role\_arn) | ARN of the IAM role that has the AmazonESCognitoAccess policy attached | `string` | `""` | no |
| <a name="input_cognito_identity_pool_id"></a> [cognito\_identity\_pool\_id](#input\_cognito\_identity\_pool\_id) | The ID of the Cognito Identity Pool to use | `string` | `""` | no |
Expand Down Expand Up @@ -81,7 +85,7 @@
| <a name="input_encrypt_at_rest_enabled"></a> [encrypt\_at\_rest\_enabled](#input\_encrypt\_at\_rest\_enabled) | Whether to enable encryption at rest | `bool` | `true` | no |
| <a name="input_encrypt_at_rest_kms_key_id"></a> [encrypt\_at\_rest\_kms\_key\_id](#input\_encrypt\_at\_rest\_kms\_key\_id) | The KMS key ID to encrypt the Elasticsearch domain with. If not specified, then it defaults to using the AWS/Elasticsearch service KMS key | `string` | `""` | no |
| <a name="input_environment"></a> [environment](#input\_environment) | ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' | `string` | `null` | no |
| <a name="input_iam_actions"></a> [iam\_actions](#input\_iam\_actions) | List of actions to allow for the IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` | `list(string)` | `[]` | no |
| <a name="input_iam_actions"></a> [iam\_actions](#input\_iam\_actions) | List of actions to allow for the user IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` | `list(string)` | `[]` | no |
| <a name="input_iam_authorizing_role_arns"></a> [iam\_authorizing\_role\_arns](#input\_iam\_authorizing\_role\_arns) | List of IAM role ARNs to permit to assume the Elasticsearch user role | `list(string)` | `[]` | no |
| <a name="input_iam_role_arns"></a> [iam\_role\_arns](#input\_iam\_role\_arns) | List of IAM role ARNs to permit access to the Elasticsearch domain | `list(string)` | `[]` | no |
| <a name="input_iam_role_max_session_duration"></a> [iam\_role\_max\_session\_duration](#input\_iam\_role\_max\_session\_duration) | The maximum session duration (in seconds) for the user role. Can have a value from 1 hour to 12 hours | `number` | `3600` | no |
Expand Down
121 changes: 121 additions & 0 deletions elasticsearch_domain.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
#
# Elasticsearch Domain
#

resource "aws_elasticsearch_domain_policy" "default" {
count = local.elasticsearch_enabled && (length(var.iam_authorizing_role_arns) > 0 || length(var.iam_role_arns) > 0) ? 1 : 0
domain_name = module.this.id
access_policies = join("", data.aws_iam_policy_document.default.*.json)
Benbentwo marked this conversation as resolved.
Show resolved Hide resolved
}

resource "aws_elasticsearch_domain" "default" {
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
dudymas marked this conversation as resolved.
Show resolved Hide resolved
count = local.elasticsearch_enabled ? 1 : 0
domain_name = module.this.id
elasticsearch_version = var.elasticsearch_version

advanced_options = var.advanced_options

advanced_security_options {
enabled = var.advanced_security_options_enabled
internal_user_database_enabled = var.advanced_security_options_internal_user_database_enabled
master_user_options {
master_user_arn = var.advanced_security_options_master_user_arn
master_user_name = var.advanced_security_options_master_user_name
master_user_password = var.advanced_security_options_master_user_password
}
}

ebs_options {
ebs_enabled = var.ebs_volume_size > 0 ? true : false
volume_size = var.ebs_volume_size
volume_type = var.ebs_volume_type
iops = var.ebs_iops
}

encrypt_at_rest {
enabled = var.encrypt_at_rest_enabled
kms_key_id = var.encrypt_at_rest_kms_key_id
}

domain_endpoint_options {
enforce_https = var.domain_endpoint_options_enforce_https
tls_security_policy = var.domain_endpoint_options_tls_security_policy
custom_endpoint_enabled = var.custom_endpoint_enabled
custom_endpoint = var.custom_endpoint_enabled ? var.custom_endpoint : null
custom_endpoint_certificate_arn = var.custom_endpoint_enabled ? var.custom_endpoint_certificate_arn : null
}

cluster_config {
instance_count = var.instance_count
instance_type = var.instance_type
dedicated_master_enabled = var.dedicated_master_enabled
dedicated_master_count = var.dedicated_master_count
dedicated_master_type = var.dedicated_master_type
zone_awareness_enabled = var.zone_awareness_enabled
warm_enabled = var.warm_enabled
warm_count = var.warm_enabled ? var.warm_count : null
warm_type = var.warm_enabled ? var.warm_type : null

dynamic "zone_awareness_config" {
for_each = var.availability_zone_count > 1 && var.zone_awareness_enabled ? [true] : []
content {
availability_zone_count = var.availability_zone_count
}
}
}

node_to_node_encryption {
enabled = var.node_to_node_encryption_enabled
}

dynamic "vpc_options" {
for_each = var.vpc_enabled ? [true] : []

content {
security_group_ids = [join("", aws_security_group.default.*.id)]
subnet_ids = var.subnet_ids
}
}

snapshot_options {
automated_snapshot_start_hour = var.automated_snapshot_start_hour
}

dynamic "cognito_options" {
for_each = var.cognito_authentication_enabled ? [true] : []
content {
enabled = true
user_pool_id = var.cognito_user_pool_id
identity_pool_id = var.cognito_identity_pool_id
role_arn = var.cognito_iam_role_arn
}
}

log_publishing_options {
enabled = var.log_publishing_index_enabled
log_type = "INDEX_SLOW_LOGS"
cloudwatch_log_group_arn = var.log_publishing_index_cloudwatch_log_group_arn
}

log_publishing_options {
enabled = var.log_publishing_search_enabled
log_type = "SEARCH_SLOW_LOGS"
cloudwatch_log_group_arn = var.log_publishing_search_cloudwatch_log_group_arn
}

log_publishing_options {
enabled = var.log_publishing_audit_enabled
log_type = "AUDIT_LOGS"
cloudwatch_log_group_arn = var.log_publishing_audit_cloudwatch_log_group_arn
}

log_publishing_options {
enabled = var.log_publishing_application_enabled
log_type = "ES_APPLICATION_LOGS"
cloudwatch_log_group_arn = var.log_publishing_application_cloudwatch_log_group_arn
}

tags = module.this.tags

depends_on = [aws_iam_service_linked_role.default]
}
Loading