Skip to content

Latest commit

 

History

History
147 lines (117 loc) · 46.5 KB

File metadata and controls

147 lines (117 loc) · 46.5 KB

Configuration Options

These tables list all CDK context configuration values a user can specify for this project. These will normally be added to the cdk.context.json file in the same directory as this markdown file.

Migration Service Options

Name Type Example Description
migrationAssistanceEnabled boolean true Flag which controls deploying common Migration Service resources such as MSK, EFS, and an ECS cluster. Note: This option must be enabled to use any Migration service
addOnMigrationDeployId string "cit2-replay" Specify an ID string to use for an additional replay scenario. Note: This option should not be used for initial deployments
defaultFargateCpuArch string "X86_64", "ARM64" Provide a default CPU architecture that should be used for all containers. Defaults to using process.arch to determine the proper architecture to use
replayerOutputEFSRemovalPolicy string "DESTROY" Policy to apply when the Replayer output EFS filesystem is removed from the CloudFormation stack
artifactBucketRemovalPolicy string "RETAIN" Policy to apply when the artifact S3 bucket is removed from the CloudFormation stack
migrationConsoleServiceEnabled boolean true Enable deploying the given service, via a new CloudFormation stack
migrationAPIEnabled boolean true Experimental flag for enabling an API on the Migration Console for controlling migration actions
migrationAPIAllowedHosts string "test-endpoint1, localhost" Comma delimited string of host or domain names that the API will serve. Other domains will receive a 400: bad request error
migrationConsoleEnableOSI boolean true Experimental: When enabled, will setup necessary IAM roles and a CloudWatch log group for controlling an OpenSearch Ingestion pipeline from the Migration Console
trafficReplayerServiceEnabled boolean true Enable deploying the given service, via a new CloudFormation stack
captureProxyServiceEnabled boolean true Enable deploying the given service, via a new CloudFormation stack
captureProxyExtraArgs string "--suppressCaptureForHeaderMatch user-agent .*elastic-java/7.17.0.*" Extra arguments to provide to the Capture Proxy command. This includes available arguments specified by the Capture Proxy. 1
kafkaBrokerServiceEnabled boolean false Enable deploying the given service, via a new CloudFormation stack. This stack is experimental and should only be used for development
otelCollectorEnabled boolean true Enable deploying an otel-collector within each service that uses it, in order to publish data to CloudWatch and X-Ray.
trafficReplayerGroupId string "logging-group-default" The Kafka consumer group ID the Replayer will use, if not specified a default ID will be used
trafficReplayerUserAgentSuffix string "AWS/test/v1.0.0" A custom user agent that will be provided to the Replayer using the --user-agent flag. This will append the provided user agent to any existing user agents when requests are made to the target cluster. This setting could also be specified with the trafficReplayerExtraArgs option
trafficReplayerExtraArgs string "--sigv4-auth-header-service-region es,us-east-1 --speedup-factor 5" Extra arguments to provide to the Replayer command. This includes auth header options and other parameters supported by the Traffic Replayer. 1
trafficReplayerMaxUptime string "P1D" The maximum uptime for the Traffic Replayer service, specified in ISO 8601 duration format. This controls how long the Traffic Replayer will run before automatically shutting down. Example values: "PT1H" (hourly), "P1D" (daily). When this duration is reached, ECS will initiate the startup of a new Traffic Replayer task to ensure continuous operation. This mechanism ensures that the Traffic Replayer service can manage its resources effectively and prevent issues associated with long running processes. Set to the greater of the given value 5 minutes. When not specified, the replayer will run continuously.
albAcmCertArn string "arn:aws:acm:us-east-1:12345678912:certificate/abc123de-4888-4fa7-a508-3811e2d49fc3" The ACM certificate ARN to use for the ALB. If not specified, and an alb is required based on other deployment parameters, a custom resource will be deployed to create one. If creation must happen locally, a script has been provded to create and upload a cert and can be invoked with npm run create-acm-cert and will return the uploaded cert arn.
targetClusterProxyServiceEnabled boolean false Enable a non-capturing proxy to use a load balancer against a managed OpenSearch cluster. This is needed to enable alb cuttover to a managed opensearch cluster as there is no way to directly route an ALB listener to an AWS Managed OpenSearch Service. If this is specified an ALB will be deployed with listener.

Cluster Definition Options

Name Type Example Description
sourceCluster object {"endpoint": "https://source-cluster.com", "auth": {"type": "basic", "username": "admin", "password": "admin"}} A json object defining the source cluster endpoint, auth type, and other details. See below for detailed options.
targetCluster object {"endpoint": "https://target-cluster.com", "auth": {"type": "sigv4", "serviceSigningName": "aoss", "region": "us-east-1"}} A json object defining the target cluster endpoint, auth type, and other details. See below for detailed options.

Structure of the cluster objects

If no source cluster is being configured, the source cluster object should be `{"disabled": true} and no other fields are necessary.

In all other cases, the required components of each cluster object are:

  • endpoint -- the fully specified endpoint for the cluster
  • version -- the Elasticsearch or OpenSearch version of the cluster, in the format of OS_x.y or ES_x.y This will be passed to the ReindexFromSnapshot service, if enabled, and provided for the metadata migration on the Migration Console.
  • auth -- what authorization strategy the cluster has. The supported options are:
    1. No auth: {"type": "none"}
    2. Sigv4 Signing: {"type": "sigv4", "region": "us-east-1", "serviceSigningName": "es"} The serviceSigningName is es for Elasticsearch and OpenSearch managed service domains, and aoss for Amazon OpenSearch Serverless
    3. Basic auth with plaintext password (only supported for the source cluster and not recommended): {"type": "basic", "username": "admin", "password": "admin123"}
    4. Basic auth with password in secrets manager (recommended): {"type": "basic", "username": "admin", "passwordFromSecretArn": "arn:aws:secretsmanager:us-east-1:12345678912:secret:master-user-os-pass-123abc"}

Reindex from Snapshot (RFS) Service Options

Name Type Example Description
reindexFromSnapshotServiceEnabled boolean true Create resources for deploying and configuring the RFS ECS service
reindexFromSnapshotExtraArgs string "--target-aws-region us-east-1 --target-aws-service-signing-name es" Extra arguments to provide to the Document Migration command with space separation. See RFS Arguments. 1
sourceClusterEndpoint string "https://source-cluster.elb.us-east-1.endpoint.com" The endpoint for the source cluster from which RFS will take a snapshot
managedServiceSourceSnapshotEnabled boolean true Create the necessary roles and trust relationships to take a snapshot of a managed service source cluster. This is only compatible with SigV4 auth. Default as true if not specified and source cluster is set with sigv4 auth.
reindexFromSnapshotMaxShardSizeGiB integer 80 OPTIONAL: The size, in whole GiB, of the largest shard you want to migrate across all indices; used to ensure we have enough disk space reserved to perform the migration. Default: 80 GiB
reindexFromSnapshotWorkerSize enum default maximum

VPC Options

Name Type Example Description
vpcEnabled boolean true Enable VPC to place Domain and Migration resources in. If a vpcId is not provided a new VPC will be created
vpcId string "vpc-123456789abcdefgh" Specify an existing VPC to place the domain inside of
vpcAZCount number 2 The number of Availability Zones for the created VPC. Note: Only applicable if creating a new VPC (thus vpcId must not be provided)

MSK(Kafka) Options

Name Type Example Description
mskBrokersPerAZCount number 1 The number of broker nodes per MSK Availability Zone
mskSubnetIds string array ["subnet-123456789abcdefgh", "subnet-223456789abcdefgh"] Specify the subnet IDs of an existing VPC to place MSK brokers in. NOTE MSK currently requires either 2 or 3 subnets to be specified and EACH subnet should use a different Availability Zone.
mskAZCount number 2 The number of Availability Zones for the MSK cluster to use. NOTE This value must be 2 or 3
mskARN (Not currently available) string "arn:aws:kafka:us-east-2:12345678912:cluster/msk-cluster-test/81fbae45-5d25-44bb-aff0-108e71cc079b-7" Supply an existing MSK cluster ARN to use. NOTE As MSK is using an L1 construct this is not currently available for use

Options being deprecated

A number of options are currently available but deprecated. While they function now, we do not recommend using them, and they may be removed without warning in a future version.

OpenSearch Domain Options

Name Type Example Description
engineVersion string "OS_1.3" The Elasticsearch/OpenSearch version that your domain will leverage. In the format of OS_x.y or ES_x.y
domainName string "os-service-domain" Name to use for the OpenSearch Service Domain
dataNodeType string "r6g.large.search" The instance type for your data nodes. Supported values can be found here
dataNodeCount number 1 The number of data nodes to use in the OpenSearch Service Domain
dedicatedManagerNodeType string "r6g.large.search" The instance type for your manager nodes. Supported values can be found here
dedicatedManagerNodeCount number 3 The number of manager nodes to use in the OpenSearch Service Domain
warmNodeType string "ultrawarm1.medium.search" The instance type for your warm nodes. Supported values can be found here
warmNodeCount number 3 The number of warm nodes to use in the OpenSearch Service Domain
accessPolicies JSON {"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":"arn:aws:iam::12345678912:user/test-user"},"Action":"es:ESHttp*","Resource":"arn:aws:es:us-east-1:12345678912:domain/cdk-os-service-domain/*"}]} Domain access policies
useUnsignedBasicAuth boolean false Configures the domain so that unsigned basic auth is enabled
fineGrainedManagerUserARN string "arn:aws:iam::12345678912:user/test-user" The IAM User ARN of the manager user.
Fine grained access control also requires nodeToNodeEncryptionEnabled and encryptionAtRestEnabled to be enabled.
Either fineGrainedMasterUserARN or fineGrainedMasterUserName can be enabled, but not both.
fineGrainedManagerUserName string "admin" Username for the manager user. Not needed if providing fineGrainedManagerUserARN
fineGrainedManagerUser
SecretManagerKeyARN
string "arn:aws:secretsmanager:us-east-1:12345678912:secret:master-user-os-pass-123abc" Password for the manager user, in the form of an AWS Secrets Manager key
enableDemoAdmin boolean false Demo mode setting that creates a basic manager user with username: admin and password: Admin123! . NOTE: This should not be used in a production environment and is not compatible with previous fineGrained settings
enforceHTTPS boolean true Require that all traffic to the domain arrive over HTTPS
tlsSecurityPolicy string "TLS_1_2" The minimum TLS version required for traffic to the domain
ebsEnabled boolean true Specify whether Amazon EBS volumes are attached to data nodes. Some instance types (i.e. r6gd) require that EBS be disabled
ebsIops number 4000 The number of I/O operations per second (IOPS) that the volume supports
ebsVolumeSize number 15 The size (in GiB) of the EBS volume for each data node
ebsVolumeType string "GP3" The EBS volume type to use with the Amazon OpenSearch Service domain. Supported values can be found here
encryptionAtRestEnabled boolean true Enable Domain to encrypt data at rest
encryptionAtRestKmsKeyARN string "arn:aws:kms:us-east-1:12345678912:key/abc123de-4888-4fa7-a508-3811e2d49fc3" Supply the KMS key to use for encryption at rest. If encryptionAtRestEnabled is enabled and this value is not provided, the default KMS key for OpenSearch Service will be used
loggingAppLogEnabled boolean true Specify if Amazon OpenSearch Service application logging should be set up
loggingAppLogGroupARN string "arn:aws:logs:us-east-1:12345678912:log-group:test-log-group:*" Supply the CloudWatch log group to use for application logging. If not provided and application logs are enabled, a CloudWatch log group will be created
nodeToNodeEncryptionEnabled boolean true Specify if node to node encryption should be enabled
vpcSubnetIds string array ["subnet-123456789abcdefgh", "subnet-223456789abcdefgh"] Specify the subnet IDs of an existing VPC to place the Domain in. Requires vpcId to be specified
vpcSecurityGroupIds string array ["sg-123456789abcdefgh", "sg-223456789abcdefgh"] Specify the Security Groups that will be associated with the VPC endpoints for the Domain. Requires vpcId to be specified
domainAZCount number 2 The number of Availability Zones for the Domain to place nodes into
openAccessPolicyEnabled boolean false Applies an open access policy to the Domain. NOTE: This setting should only be used for Domains placed within a VPC, and is applicable to many use cases where access controlled by Security Groups on the VPC is sufficient.
domainRemovalPolicy string "RETAIN" Policy to apply when the domain is removed from the CloudFormation stack

Imported Target Cluster Options

Name Type Example Description
targetClusterEndpoint string "https://vpc-demo-opensearch-cluster-cv6hggdb66ybpk4kxssqt6zdhu.us-west-2.es.amazonaws.com:443" The target cluster endpoint which will be used to replay captured traffic against. This endpoint must be in the same VPC as the Migration services to allow for proper linking

Imported Source Cluster Options

Name Type Example Description
sourceClusterDisabled boolean true Disable configuring a source cluster in any way. This is incompatible with specifying any type of capture proxy or a source cluster endpoint. It's suitable for backfill migrations using ReindexFromSnapshot from an already-existing snapshot.
sourceClusterEndpoint string "https://my-source-cluster.com:443" The URI of the source cluster from that the migration will reference. Note: if this is not provided and elasticsearchService or captureProxyESService is enabled, the migration will reference a uri for that service.

Other Options

Name Type Example Description
captureProxyESServiceEnabled boolean true Enable deploying the given service, via a new CloudFormation stack
captureProxyESExtraArgs string "--suppressCaptureForHeaderMatch user-agent .*elastic-java/7.17.0.*" Extra arguments to provide to the Capture Proxy command. This includes available arguments specified by the Capture Proxy. 1
elasticsearchServiceEnabled boolean true Enable deploying the given service, via a new CloudFormation stack
osContainerServiceEnabled boolean false Enable deploying the given service, via a new CloudFormation stack.

Footnotes

  1. Extra arguments can be added, overridden, or removed as follows:

    • To add/override an argument: Include the argument with the value, e.g., "--new-arg value"
    • Include quotes/escaping as appropriate for bash processing "--new-arg \"my value\""
    2 3 4