Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scope discovery-ec2 IMDS lookups to this cluster formation. #36

Merged
merged 1 commit into from
Jun 8, 2023

Conversation

dblock
Copy link
Member

@dblock dblock commented Jun 8, 2023

Description

Scope discovery-ec2 IMDS lookups for seed/manager nodes down to this cluster. Here's a successful output.

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|   timestamp   |                                                                                                                                                    message                                                                                                                                                     |
|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1686244606265 | [2023-06-08T17:16:46,146][DEBUG][o.o.d.e.AwsEc2SeedHostsProvider] [ip-10-0-5-156.ec2.internal] using host_type [private_ip], tags [{Name=[opensearch-infra-stack-270-multinode/seedNodeAsg, opensearch-infra-stack-270-multinode/managerNodeAsg]}], groups [[]] with any_group [true], availability_zones [[]] |
| 1686244608270 | [2023-06-08T17:16:48,118][TRACE][o.o.d.e.AwsEc2SeedHostsProvider] [ip-10-0-5-156.ec2.internal] finding seed nodes...                                                                                                                                                                                           |
| 1686244608270 | [2023-06-08T17:16:48,119][TRACE][o.o.d.e.AwsEc2SeedHostsProvider] [ip-10-0-5-156.ec2.internal] adding i-0e1b8f3a73ed4195b, address 10.0.4.146, transport_address 10.0.4.146:9300                                                                                                                               |
| 1686244608270 | [2023-06-08T17:16:48,119][TRACE][o.o.d.e.AwsEc2SeedHostsProvider] [ip-10-0-5-156.ec2.internal] adding i-0bad8a1561668117f, address 10.0.5.216, transport_address 10.0.5.216:9300                                                                                                                               |
| 1686244608270 | [2023-06-08T17:16:48,120][TRACE][o.o.d.e.AwsEc2SeedHostsProvider] [ip-10-0-5-156.ec2.internal] adding i-0a452e2dfdc05691f, address 10.0.3.33, transport_address 10.0.3.33:9300                                                                                                                                 |
| 1686244608520 | [2023-06-08T17:16:48,120][DEBUG][o.o.d.e.AwsEc2SeedHostsProvider] [ip-10-0-5-156.ec2.internal] using dynamic transport addresses [10.0.4.146:9300, 10.0.5.216:9300, 10.0.3.33:9300]                                                                                                                            |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

I tried using aws:cloudformation:stack-name, but OpenSearch doesn't support settings named discovery.ec2.tag.aws:cloudformation:stack-name.

Issues Resolved

Fixes #34.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: dblock <dblock@amazon.com>
@codecov
Copy link

codecov bot commented Jun 8, 2023

Codecov Report

Merging #36 (9bb3d6a) into main (fce3055) will increase coverage by 0.05%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #36      +/-   ##
==========================================
+ Coverage   79.71%   79.76%   +0.05%     
==========================================
  Files           5        5              
  Lines         345      346       +1     
  Branches      104      104              
==========================================
+ Hits          275      276       +1     
  Misses         70       70              
Impacted Files Coverage Δ
lib/infra/infra-stack.ts 93.63% <100.00%> (+0.05%) ⬆️

@@ -425,6 +425,10 @@ export class InfraStack extends Stack {
const baseConfig: any = load(readFileSync(`${configFileDir}/multi-node-base-config.yml`, 'utf-8'));

baseConfig['cluster.name'] = `${scope.stackName}-${scope.account}-${scope.region}`;

// use discovery-ec2 to find manager nodes by querying IMDS
baseConfig['discovery.ec2.tag.Name'] = `${scope.stackName}/seedNodeAsg,${scope.stackName}/managerNodeAsg`;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same tag name should be associated with ec2 nodes as well, right?
https://github.com/opensearch-project/opensearch-cluster-cdk/blob/main/lib/infra/infra-stack.ts#L195

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's already there. Here's what a node gets:

Screenshot 2023-06-08 at 1 58 09 PM

@rishabh6788 rishabh6788 merged commit 7e7d6cf into opensearch-project:main Jun 8, 2023
@dblock dblock deleted the discovery-ec2-scope branch June 8, 2023 18:05
opensearch-trigger-bot bot pushed a commit that referenced this pull request Oct 24, 2023
Signed-off-by: dblock <dblock@amazon.com>
(cherry picked from commit 7e7d6cf)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Cluster formation probes manager nodes from other clusters
2 participants