Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(aws-lambda-opensearch): created new construct #818

Merged
merged 9 commits into from
Oct 11, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
|![Java Logo](https://docs.aws.amazon.com/cdk/api/latest/img/java32.png) Java|`software.amazon.awsconstructs.services.lambdaelasticsearchkibana`|

## Overview
This AWS Solutions Construct implements the AWS Lambda function and Amazon Elasticsearch Service with the least privileged permissions.
This AWS Solutions Construct implements an AWS Lambda function and Amazon Elasticsearch Service with the least privileged permissions.

**Some cluster configurations (e.g VPC access) require the existence of the `AWSServiceRoleForAmazonElasticsearchService` Service-Linked Role in your account.**

Expand Down Expand Up @@ -111,7 +111,7 @@ new LambdaToElasticSearchAndKibana(this, "sample",
|esDomainProps?|[`elasticsearch.CfnDomainProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_elasticsearch.CfnDomainProps.html)|Optional user provided props to override the default props for the Elasticsearch Service|
|domainName|`string`|Domain name for the Cognito and the Elasticsearch Service|
|cognitoDomainName?|`string`|Optional Cognito Domain Name, if provided it will be used for Cognito Domain, and domainName will be used for the Elasticsearch Domain|
|createCloudWatchAlarms|`boolean`|Whether to create recommended CloudWatch alarms|
|createCloudWatchAlarms?|`boolean`|Whether to create recommended CloudWatch alarms|
|domainEndpointEnvironmentVariableName?|`string`|Optional Name for the ElasticSearch domain endpoint environment variable set for the Lambda function.|
|existingVpc?|[`ec2.IVpc`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.IVpc.html)|An optional, existing VPC into which this pattern should be deployed. When deployed in a VPC, the Lambda function will use ENIs in the VPC to access network resources. If an existing VPC is provided, the `deployVpc` property cannot be `true`. This uses `ec2.IVpc` to allow clients to supply VPCs that exist outside the stack using the [`ec2.Vpc.fromLookup()`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.Vpc.html#static-fromwbrlookupscope-id-options) method.|
|vpcProps?|[`ec2.VpcProps`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.VpcProps.html)|Optional user provided properties to override the default properties for the new VPC. `enableDnsHostnames`, `enableDnsSupport`, `natGateways` and `subnetConfiguration` are set by the pattern, so any values for those properties supplied here will be overrriden. If `deployVpc` is not `true` then this property will be ignored.|
Expand All @@ -121,13 +121,13 @@ new LambdaToElasticSearchAndKibana(this, "sample",

| **Name** | **Type** | **Description** |
|:-------------|:----------------|-----------------|
|lambdaFunction|[`lambda.Function`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_lambda.Function.html)|Returns an instance of lambda.Function created by the construct|
|userPool|[`cognito.UserPool`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cognito.UserPool.html)|Returns an instance of cognito.UserPool created by the construct|
|userPoolClient|[`cognito.UserPoolClient`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cognito.UserPoolClient.html)|Returns an instance of cognito.UserPoolClient created by the construct|
|identityPool|[`cognito.CfnIdentityPool`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cognito.CfnIdentityPool.html)|Returns an instance of cognito.CfnIdentityPool created by the construct|
|elasticsearchDomain|[`elasticsearch.CfnDomain`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_elasticsearch.CfnDomain.html)|Returns an instance of elasticsearch.CfnDomain created by the construct|
|elasticsearchDomain|[`iam.Role`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_iam.Role.html)|Returns an instance of iam.Role created by the construct for elasticsearch.CfnDomain|
|cloudwatchAlarms?|[`cloudwatch.Alarm[]`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cloudwatch.Alarm.html)|Returns a list of cloudwatch.Alarm created by the construct|
|lambdaFunction|[`lambda.Function`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_lambda.Function.html)|Returns an instance of `lambda.Function` created by the construct|
|userPool|[`cognito.UserPool`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cognito.UserPool.html)|Returns an instance of `cognito.UserPool` created by the construct|
|userPoolClient|[`cognito.UserPoolClient`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cognito.UserPoolClient.html)|Returns an instance of `cognito.UserPoolClient` created by the construct|
|identityPool|[`cognito.CfnIdentityPool`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cognito.CfnIdentityPool.html)|Returns an instance of `cognito.CfnIdentityPool` created by the construct|
|elasticsearchDomain|[`elasticsearch.CfnDomain`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_elasticsearch.CfnDomain.html)|Returns an instance of `elasticsearch.CfnDomain` created by the construct|
|elasticsearchRole|[`iam.Role`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_iam.Role.html)|Returns an instance of `iam.Role` created by the construct for `elasticsearch.CfnDomain`|
|cloudwatchAlarms?|[`cloudwatch.Alarm[]`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_cloudwatch.Alarm.html)|Returns a list of `cloudwatch.Alarm` created by the construct|
|vpc?|[`ec2.IVpc`](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.IVpc.html)|Returns an interface on the VPC used by the pattern (if any). This may be a VPC created by the pattern or the VPC supplied to the pattern constructor.|

## Lambda Function
Expand All @@ -136,26 +136,26 @@ This pattern requires a lambda function that can post data into the Elasticsearc

## Default settings

Out of the box implementation of the Construct without any override will set the following defaults:
Out of the box implementation of the Construct without any overrides will set the following defaults:

### AWS Lambda Function
* Configure limited privilege access IAM role for Lambda function
* Enable reusing connections with Keep-Alive for NodeJs Lambda function
* Enable reusing connections with Keep-Alive for Node.js Lambda function
* Enable X-Ray Tracing
* Set Environment Variables
* (default) DOMAIN_ENDPOINT
* AWS_NODEJS_CONNECTION_REUSE_ENABLED (for Node 10.x and higher functions)
* AWS_NODEJS_CONNECTION_REUSE_ENABLED

### Amazon Cognito
* Set password policy for User Pools
* Enforce the advanced security mode for User Pools

### Amazon Elasticsearch Service
* Deploy best practices CloudWatch Alarms for the Elasticsearch Domain
* Deploy best practices CloudWatch Alarms for the Elasticsearch Service domain
* Secure the Kibana dashboard access with Cognito User Pools
* Enable server-side encryption for Elasticsearch Domain using AWS managed KMS Key
* Enable node-to-node encryption for Elasticsearch Domain
* Configure the cluster for the Amazon ES domain
* Enable server-side encryption for the Elasticsearch Service domain using AWS managed KMS Key
* Enable node-to-node encryption for the Elasticsearch Service domain
* Configure the cluster for the Elasticsearch Service domain

## Architecture
![Architecture Diagram](architecture.png)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,11 @@ import * as cognito from 'aws-cdk-lib/aws-cognito';
import * as defaults from '@aws-solutions-constructs/core';
// Note: To ensure CDKv2 compatibility, keep the import statement for Construct separate
import { Construct } from 'constructs';
import { Role } from 'aws-cdk-lib/aws-iam';
import * as cloudwatch from 'aws-cdk-lib/aws-cloudwatch';
import * as ec2 from 'aws-cdk-lib/aws-ec2';

/**
* @summary The properties for the CognitoToApiGatewayToLambda Construct
* @summary The properties for the LambdaToElasticSearchAndKibana Construct
*/
export interface LambdaToElasticSearchAndKibanaProps {
/**
Expand Down Expand Up @@ -99,10 +98,10 @@ export class LambdaToElasticSearchAndKibana extends Construct {
public readonly vpc?: ec2.IVpc;

/**
* @summary Constructs a new instance of the CognitoToApiGatewayToLambda class.
* @param {cdk.App} scope - represents the scope for all the resources.
* @summary Constructs a new instance of the LambdaToElasticSearchAndKibana class.
* @param {Constructs} scope - represents the scope for all the resources.
* @param {string} id - this is a a scope-unique id.
* @param {CognitoToApiGatewayToLambdaProps} props - user provided props for the construct
* @param {LambdaToElasticSearchAndKibanaProps} props - user provided props for the construct
* @since 0.8.0
* @access public
*/
Expand Down Expand Up @@ -146,7 +145,7 @@ export class LambdaToElasticSearchAndKibana extends Construct {
let cognitoAuthorizedRole: iam.Role;

[this.userPool, this.userPoolClient, this.identityPool, cognitoAuthorizedRole] =
this.setupCognito(this, props.cognitoDomainName ?? props.domainName);
defaults.buildCognitoForSearchService(this, props.cognitoDomainName ?? props.domainName);

const buildElasticSearchProps: any = {
userpool: this.userPool,
Expand Down Expand Up @@ -174,18 +173,4 @@ export class LambdaToElasticSearchAndKibana extends Construct {
this.cloudwatchAlarms = defaults.buildElasticSearchCWAlarms(this);
}
}

setupCognito(scope: Construct, domainName: string): [cognito.UserPool, cognito.UserPoolClient, cognito.CfnIdentityPool, iam.Role] {
const userPool = defaults.buildUserPool(scope);
const userPoolClient = defaults.buildUserPoolClient(scope, userPool);
const identityPool = defaults.buildIdentityPool(scope, userPool, userPoolClient);

const cognitoAuthorizedRole: Role = defaults.setupCognitoForElasticSearch(scope, domainName, {
userpool: userPool,
identitypool: identityPool,
userpoolclient: userPoolClient
});

return [userPool, userPoolClient, identityPool, cognitoAuthorizedRole];
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@
"S3Bucket": {
"Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
},
"S3Key": "67a9971e29baab2bde3043bb70ce5b53318b95429a1ce9b189cf65223e8682db.zip"
"S3Key": "35bbbc7b04b21f225891f139adf234188f348ebad5f4bbc2c06edf8aa3784c97.zip"
},
"Role": {
"Fn::GetAtt": [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -804,7 +804,7 @@
"Properties": {
"Code": {
"S3Bucket": "cdk-hnb659fds-assets-12345678-test-region",
"S3Key": "67a9971e29baab2bde3043bb70ce5b53318b95429a1ce9b189cf65223e8682db.zip"
"S3Key": "35bbbc7b04b21f225891f139adf234188f348ebad5f4bbc2c06edf8aa3784c97.zip"
},
"Role": {
"Fn::GetAtt": [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@
"Properties": {
"Code": {
"S3Bucket": "cdk-hnb659fds-assets-12345678-test-region",
"S3Key": "67a9971e29baab2bde3043bb70ce5b53318b95429a1ce9b189cf65223e8682db.zip"
"S3Key": "35bbbc7b04b21f225891f139adf234188f348ebad5f4bbc2c06edf8aa3784c97.zip"
},
"Role": {
"Fn::GetAtt": [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@
"S3Bucket": {
"Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
},
"S3Key": "67a9971e29baab2bde3043bb70ce5b53318b95429a1ce9b189cf65223e8682db.zip"
"S3Key": "35bbbc7b04b21f225891f139adf234188f348ebad5f4bbc2c06edf8aa3784c97.zip"
},
"Role": {
"Fn::GetAtt": [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@
"S3Bucket": {
"Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
},
"S3Key": "67a9971e29baab2bde3043bb70ce5b53318b95429a1ce9b189cf65223e8682db.zip"
"S3Key": "35bbbc7b04b21f225891f139adf234188f348ebad5f4bbc2c06edf8aa3784c97.zip"
},
"Role": {
"Fn::GetAtt": [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@
"S3Bucket": {
"Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
},
"S3Key": "67a9971e29baab2bde3043bb70ce5b53318b95429a1ce9b189cf65223e8682db.zip"
"S3Key": "35bbbc7b04b21f225891f139adf234188f348ebad5f4bbc2c06edf8aa3784c97.zip"
},
"Role": {
"Fn::GetAtt": [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@
"S3Bucket": {
"Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
},
"S3Key": "67a9971e29baab2bde3043bb70ce5b53318b95429a1ce9b189cf65223e8682db.zip"
"S3Key": "35bbbc7b04b21f225891f139adf234188f348ebad5f4bbc2c06edf8aa3784c97.zip"
},
"Role": {
"Fn::GetAtt": [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ function postDocumentToES(doc, context) {
var body = '';
httpResp.on('data', function (chunk) {
body += chunk;
});
});
httpResp.on('end', function (chunk) {
console.log('All movie records added to ES.');
context.succeed();
Expand All @@ -47,10 +47,13 @@ function postDocumentToES(doc, context) {

exports.handler = (event, context) => {
console.log('Received event:', JSON.stringify(event, null, 2));
postDocumentToES("{ \"title\": \"Spirited Away\" }", context);
return {
      statusCode: 200,
      headers: { 'Content-Type': 'text/plain' },
      body: `Hello from Project Vesper! You've hit ${event.path}\n`
    };
postDocumentToOpenSearch("{ \"title\": \"Moby Dick\" }", context);
postDocumentToOpenSearch("{ \"title\": \"A Tale of Two Cities\" }", context);
postDocumentToOpenSearch("{ \"title\": \"The Phantom of the Opera\" }", context);

return {
statusCode: 200,
headers: { 'Content-Type': 'text/plain' },
body: `Hello from AWS Solutions Constructs!\n`
};
};
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
lib/*.js
test/*.js
*.d.ts
coverage
test/lambda/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
lib/*.js
test/*.js
!test/lambda/*
*.js.map
*.d.ts
node_modules
*.generated.ts
dist
.jsii

.LAST_BUILD
.nyc_output
coverage
.nycrc
.LAST_PACKAGE
*.snk
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Exclude typescript source and config
*.ts
tsconfig.json
coverage
.nyc_output
*.tgz
*.snk
*.tsbuildinfo

# Include javascript files and typescript declarations
!*.js
!*.d.ts

# Exclude jsii outdir
dist

# Include .jsii
!.jsii

# Include .jsii
!.jsii
Loading