Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ec_deployment: Add snapshot settings #620

Merged
merged 2 commits into from
Apr 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .changelog/620.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:enhancement
resource/ec_deployment: Add snapshot settings (for usage with Elastic Cloud Enterprise only).
```
20 changes: 20 additions & 0 deletions docs/resources/ec_deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -308,6 +308,7 @@ The required `elasticsearch` block supports the following arguments:
* `ref_id` - (Optional) Can be set on the Elasticsearch resource. The default value `main-elasticsearch` is recommended.
* `config` (Optional) Elasticsearch settings applied to all topologies unless overridden in the `topology` element.
* `remote_cluster` (Optional) Elasticsearch remote clusters to configure for the Elasticsearch resource. Can be set multiple times.
* `snapshot` (Elastic Cloud Enterprise only, Optional) snapshot configuration settings for an Elasticsearch cluster.
* `snapshot_source` (Optional) Restores data from a snapshot of another deployment.
* `extension` (Optional) Custom Elasticsearch bundles or plugins. Can be set multiple times.
* `autoscale` (Optional) Enable or disable autoscaling. Defaults to the setting coming from the deployment template. Accepted values are `"true"` or `"false"`.
Expand Down Expand Up @@ -367,6 +368,25 @@ The optional `elasticsearch.remote_cluster` block can be set multiple times. It
* `ref_id` (Optional) Remote Elasticsearch `ref_id`. The default value `main-elasticsearch` is recommended.
* `skip_unavailable` (Optional) If true, skip the cluster during search when disconnected. Defaults to `false`.

##### Snapshot

~> **This setting can only be used with Elastic Cloud Enterprise** For Elastic Cloud SaaS please use the [elasticstack_elasticsearch_snapshot_repository](https://registry.terraform.io/providers/elastic/elasticstack/latest/docs/resources/elasticsearch_snapshot_repository) resource from the [Elastic Stack terraform provider](https://registry.terraform.io/providers/elastic/elasticstack/latest).

* `enabled` (Required) Indicates if Snapshotting is enabled.
* `repository` (Optional) Snapshot repository configuration.

##### Repository

~> **This setting can only be used with Elastic Cloud Enterprise** For Elastic Cloud SaaS please use the [elasticstack_elasticsearch_snapshot_repository](https://registry.terraform.io/providers/elastic/elasticstack/latest/docs/resources/elasticsearch_snapshot_repository) resource from the [Elastic Stack terraform provider](https://registry.terraform.io/providers/elastic/elasticstack/latest).

- `reference` (Optional) Cluster snapshot reference repository settings, containing the repository name in ECE fashion.

##### Reference

~> **This setting can only be used with Elastic Cloud Enterprise** For Elastic Cloud SaaS please use the [elasticstack_elasticsearch_snapshot_repository](https://registry.terraform.io/providers/elastic/elasticstack/latest/docs/resources/elasticsearch_snapshot_repository) resource from the [Elastic Stack terraform provider](https://registry.terraform.io/providers/elastic/elasticstack/latest).

- `repository_name` (Optional) ECE snapshot repository name, from the '/platform/configuration/snapshots/repositories' endpoint.

##### Snapshot source

The optional `elasticsearch.snapshot_source` block, which restores data from a snapshot of another deployment, supports the following arguments:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ type ElasticsearchTF struct {
MlTier types.Object `tfsdk:"ml"`
Config types.Object `tfsdk:"config"`
RemoteCluster types.Set `tfsdk:"remote_cluster"`
Snapshot types.Object `tfsdk:"snapshot"`
SnapshotSource types.Object `tfsdk:"snapshot_source"`
Extension types.Set `tfsdk:"extension"`
TrustAccount types.Set `tfsdk:"trust_account"`
Expand Down Expand Up @@ -106,6 +107,9 @@ func (es *ElasticsearchTF) payload(ctx context.Context, res *models.Elasticsearc
res.Plan.Elasticsearch, ds = elasticsearchConfigPayload(ctx, es.Config, res.Plan.Elasticsearch)
diags.Append(ds...)

res.Settings, ds = elasticsearchSnapshotPayload(ctx, es.Snapshot, res.Settings)
diags.Append(ds...)

diags.Append(elasticsearchSnapshotSourcePayload(ctx, es.SnapshotSource, res.Plan)...)

diags.Append(elasticsearchExtensionPayload(ctx, es.Extension, res.Plan.Elasticsearch)...)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1560,7 +1560,7 @@ func Test_writeElasticsearch(t *testing.T) {
}),
},
{
name: "parses an ES resource with snapshot settings",
name: "parses an ES resource with snapshot source settings",
args: args{
es: Elasticsearch{
RefId: ec.String("main-elasticsearch"),
Expand Down Expand Up @@ -1633,6 +1633,86 @@ func Test_writeElasticsearch(t *testing.T) {
},
}),
},
{
name: "parses an ES resource with snapshot settings",
args: args{
es: Elasticsearch{
RefId: ec.String("main-elasticsearch"),
ResourceId: ec.String(mock.ValidClusterID),
Region: ec.String("some-region"),
Snapshot: &ElasticsearchSnapshot{
Enabled: true,
Repository: &ElasticsearchSnapshotRepositoryInfo{
Reference: &ElasticsearchSnapshotRepositoryReference{
RepositoryName: "my-snapshot-repository",
},
},
},
HotTier: &ElasticsearchTopology{
id: "hot_content",
Size: ec.String("2g"),
ZoneCount: 1,
},
},
template: testutil.ParseDeploymentTemplate(t, "../../testdata/template-aws-io-optimized-v2.json"),
templateID: "aws-io-optimized-v2",
version: "7.7.0",
useNodeRoles: false,
},
want: EnrichWithEmptyTopologies(tp770(), &models.ElasticsearchPayload{
Region: ec.String("some-region"),
RefID: ec.String("main-elasticsearch"),
Settings: &models.ElasticsearchClusterSettings{
DedicatedMastersThreshold: 6,
Snapshot: &models.ClusterSnapshotSettings{
Enabled: ec.Bool(true),
Repository: &models.ClusterSnapshotRepositoryInfo{
Reference: &models.ClusterSnapshotRepositoryReference{
RepositoryName: "my-snapshot-repository",
},
},
},
},
Plan: &models.ElasticsearchClusterPlan{
AutoscalingEnabled: ec.Bool(false),
Elasticsearch: &models.ElasticsearchConfiguration{
Version: "7.7.0",
},
DeploymentTemplate: &models.DeploymentTemplateReference{
ID: ec.String("aws-io-optimized-v2"),
},
ClusterTopology: []*models.ElasticsearchClusterTopologyElement{
{
ID: "hot_content",
ZoneCount: 1,
InstanceConfigurationID: "aws.data.highio.i3",
Size: &models.TopologySize{
Resource: ec.String("memory"),
Value: ec.Int32(2048),
},
NodeType: &models.ElasticsearchNodeType{
Data: ec.Bool(true),
Ingest: ec.Bool(true),
Master: ec.Bool(true),
},
Elasticsearch: &models.ElasticsearchConfiguration{
NodeAttributes: map[string]string{"data": "hot"},
},
TopologyElementControl: &models.TopologyElementControl{
Min: &models.TopologySize{
Resource: ec.String("memory"),
Value: ec.Int32(1024),
},
},
AutoscalingMax: &models.TopologySize{
Value: ec.Int32(118784),
Resource: ec.String("memory"),
},
},
},
},
}),
},
{
name: "parse autodetect configuration strategy",
args: args{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ type Elasticsearch struct {
MlTier *ElasticsearchTopology `tfsdk:"ml"`
Config *ElasticsearchConfig `tfsdk:"config"`
RemoteCluster ElasticsearchRemoteClusters `tfsdk:"remote_cluster"`
Snapshot *ElasticsearchSnapshot `tfsdk:"snapshot"`
SnapshotSource *ElasticsearchSnapshotSource `tfsdk:"snapshot_source"`
Extension ElasticsearchExtensions `tfsdk:"extension"`
TrustAccount ElasticsearchTrustAccounts `tfsdk:"trust_account"`
Expand Down Expand Up @@ -118,6 +119,12 @@ func readElasticsearch(in *models.ElasticsearchResourceInfo, remotes *models.Rem
}
es.Extension = extensions

snapshot, err := readElasticsearchSnapshot(in.Info.Settings)
if err != nil {
return nil, err
}
es.Snapshot = snapshot

accounts, err := readElasticsearchTrustAccounts(in.Info.Settings)
if err != nil {
return nil, err
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,136 @@ func Test_readElasticsearch(t *testing.T) {
},
},
},
{
name: "parses an elasticsearch resource with snapshot repository",
args: args{in: []*models.ElasticsearchResourceInfo{
{
Region: ec.String("some-region"),
RefID: ec.String("main-elasticsearch"),
Info: &models.ElasticsearchClusterInfo{
ClusterID: &mock.ValidClusterID,
Region: "some-region",
Status: ec.String("started"),
Metadata: &models.ClusterMetadataInfo{
CloudID: "some CLOUD ID",
Endpoint: "somecluster.cloud.elastic.co",
Ports: &models.ClusterMetadataPortInfo{
HTTP: ec.Int32(9200),
HTTPS: ec.Int32(9243),
},
},
Settings: &models.ElasticsearchClusterSettings{
Snapshot: &models.ClusterSnapshotSettings{
Enabled: ec.Bool(true),
Repository: &models.ClusterSnapshotRepositoryInfo{
Reference: &models.ClusterSnapshotRepositoryReference{
RepositoryName: "my-snapshot-repository",
},
},
},
},
PlanInfo: &models.ElasticsearchClusterPlansInfo{
Current: &models.ElasticsearchClusterPlanInfo{
Plan: &models.ElasticsearchClusterPlan{
Elasticsearch: &models.ElasticsearchConfiguration{
Version: "7.7.0",
},
ClusterTopology: []*models.ElasticsearchClusterTopologyElement{
{
ID: "hot_content",
ZoneCount: 1,
InstanceConfigurationID: "aws.data.highio.i3",
Size: &models.TopologySize{
Resource: ec.String("memory"),
Value: ec.Int32(2048),
},
NodeType: &models.ElasticsearchNodeType{
Data: ec.Bool(true),
Ingest: ec.Bool(true),
Master: ec.Bool(true),
Ml: ec.Bool(false),
},
},
},
},
},
},
},
},
{
Region: ec.String("some-region"),
RefID: ec.String("main-elasticsearch"),
Info: &models.ElasticsearchClusterInfo{
ClusterID: &mock.ValidClusterID,
Region: "some-region",
Status: ec.String("stopped"),
Metadata: &models.ClusterMetadataInfo{
CloudID: "some CLOUD ID",
Endpoint: "somecluster.cloud.elastic.co",
Ports: &models.ClusterMetadataPortInfo{
HTTP: ec.Int32(9200),
HTTPS: ec.Int32(9243),
},
},
PlanInfo: &models.ElasticsearchClusterPlansInfo{
Current: &models.ElasticsearchClusterPlanInfo{
Plan: &models.ElasticsearchClusterPlan{
Elasticsearch: &models.ElasticsearchConfiguration{
Version: "7.7.0",
},
ClusterTopology: []*models.ElasticsearchClusterTopologyElement{
{
ID: "hot_content",
ZoneCount: 1,
InstanceConfigurationID: "aws.data.highio.i3",
Size: &models.TopologySize{
Resource: ec.String("memory"),
Value: ec.Int32(2048),
},
NodeType: &models.ElasticsearchNodeType{
Data: ec.Bool(true),
Ingest: ec.Bool(true),
Master: ec.Bool(true),
Ml: ec.Bool(false),
},
},
},
},
},
},
},
},
}},
want: &Elasticsearch{
RefId: ec.String("main-elasticsearch"),
ResourceId: ec.String(mock.ValidClusterID),
Region: ec.String("some-region"),
CloudID: ec.String("some CLOUD ID"),
HttpEndpoint: ec.String("http://somecluster.cloud.elastic.co:9200"),
HttpsEndpoint: ec.String("https://somecluster.cloud.elastic.co:9243"),
Config: &ElasticsearchConfig{},
HotTier: &ElasticsearchTopology{
id: "hot_content",
InstanceConfigurationId: ec.String("aws.data.highio.i3"),
Size: ec.String("2g"),
SizeResource: ec.String("memory"),
NodeTypeData: ec.String("true"),
NodeTypeIngest: ec.String("true"),
NodeTypeMaster: ec.String("true"),
NodeTypeMl: ec.String("false"),
ZoneCount: 1,
Autoscaling: &ElasticsearchTopologyAutoscaling{},
},
Snapshot: &ElasticsearchSnapshot{
Enabled: true,
Repository: &ElasticsearchSnapshotRepositoryInfo{
Reference: &ElasticsearchSnapshotRepositoryReference{
RepositoryName: "my-snapshot-repository",
},
},
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
Expand Down
Loading