Update to version v1.0.3 (#30)

.github/ISSUE_TEMPLATE/bug_report.md

@@ -17,7 +17,7 @@ Steps to reproduce the behavior.
 A clear and concise description of what you expected to happen.
 
 **Please complete the following information about the solution:**
-- [ ] Version: [e.g. v1.0.2]
+- [ ] Version: [e.g. v1.0.3]
 
 To get the version of the solution, you can look at the description of the created CloudFormation stack. For example, "_(SO0021) - Video On Demand workflow with AWS Step Functions, MediaConvert, MediaPackage, S3, CloudFront and DynamoDB. Version **v5.0.0**_". If the description does not contain the version information, you can look at the mappings section of the template:
 

.gitignore

@@ -4,6 +4,7 @@ build/
 **/.vscode/*
 reports/
 coverage/
+coverage-reports/
 .aws-sam/
 output/
 target
@@ -13,6 +14,15 @@ druid-bin
 zookeeper-bin
 *_version.txt
 
+# Python build files
+.venv/
+__pycache__/
+py_modules/
+
+# derived build assets
+**/deployment/global-s3-assets
+**/deployment/regional-s3-assets
+
 *.js
 !jest.config.js
 !jest.setup.js

CHANGELOG.md

@@ -5,9 +5,25 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.0.3] - 2024-08-19
+
+### Security
+
+- Upgrade Zookeeper to the latest patch version of v3.8.4
+- Upgrade Druid to v30.0.0
+- Upgrade Druid-Operator to be v1.2.3
+- EC2 instances now require IMDSv2
+- Patch axios vulnerability
+
+### Fixed
+
+- Empty S3 buckets during teardown if needed
+- Gracefully delete CloudWatch Synthetics Canary network interfaces
+
 ## [1.0.2] - 2024-08-07
 
 ### Security
+
 - Patched fast-xml-parser vulnerability
 
 ## [1.0.1] - 2024-07-01

README.md

@@ -126,6 +126,7 @@ The solution deploys the following components that work together to provide a pr
 -   The latest version of the [AWS CLI](https://aws.amazon.com/cli/), installed and configured.
 -   The latest version of the [AWS CDK](https://docs.aws.amazon.com/cdk/latest/guide/home.html).
 -   [Nodejs](https://docs.npmjs.com/getting-started) version 18 or newer.
+-   [Python](https://www.python.org/) version 3.12 or newer.
 -   [Git](https://git-scm.com/) command line
 -   Java Runtime
     -   The solution requires a Java 8 Runtime. We strongly recommend using [Amazon Corretto 8](https://docs.aws.amazon.com/corretto/latest/corretto-8-ug/downloads-list.html). Alternatively, you can also use other OpenJDKs such as [Eclipse Temurin](https://adoptium.net/en-GB/temurin/releases/?version=8).
@@ -294,7 +295,7 @@ It is important to note that the redirect URI on the IDP side should be configur
 -   **Druid version**: Apache Druid release version (eg. 27.0.0) that you want to run. It is recommended to use the latest stable [Druid version](#https://druid.apache.org/downloads.html).
 
     ```
-    "druidVersion": "27.0.0",
+    "druidVersion": "30.0.0",
     ```
 
 -   **Druid cluster name**: A sequence of ASCII characters that uniquely identifies each Druid cluster. If there are multiple deployments, please make sure that you have an unique cluster name for each cluster. The cluster name will be appended to the CloudFormation stack name.
@@ -797,6 +798,7 @@ Upon successfully cloning the repository into your local development environment
 |- NOTICE.txt                   - Copyrights for this solution.
 |- THIRDPARTY_LICENSE.txt       - Copyrights licenses for third party software that was used in this solution
 |- README.md                    - this file.
+|- SECURITY.md                  - detailed information about reporting security issues.
 ```
 
 ---

SECURITY.md

@@ -0,0 +1,6 @@
+Reporting Security Issues
+----------------------------------------------------------------------------------------------------------
+We take all security reports seriously. When we receive such reports, we will investigate and 
+subsequently address any potential vulnerabilities as quickly as possible. If you discover a potential 
+security issue in this project, please notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/) or 
+directly via email to [AWS Security](mailto:aws-security@amazon.com). Please do not create a public GitHub issue in this project.

deployment/build-s3-dist.sh

@@ -21,7 +21,7 @@
 # Parameters:
 #  - source-bucket-base-name: Name for the S3 bucket location where the template will source the Lambda
 #    code from. The template will append '-[region_name]' to this bucket name.
-#    For example: ./build-s3-dist.sh solutions v1.0.2
+#    For example: ./build-s3-dist.sh solutions v1.0.3
 #    The template will then expect the source code to be located in the solutions-[region_name] bucket
 #  - solution-name: name of the solution for consistency
 #  - version-code: version of the package
@@ -60,7 +60,7 @@ usage()
 {
     echo "Usage: $0 bucket solution-name version"
     echo "Please provide the base source bucket name, trademarked solution name, and version." 
-    echo "For example: ./build-s3-dist.sh mybucket my-solution v1.0.2" 
+    echo "For example: ./build-s3-dist.sh mybucket my-solution v1.0.3" 
     exit 1 
 }
 
@@ -214,6 +214,9 @@ cleanup_temporary_generted_files()
 
     # Delete the downloaded files
     do_cmd rm -rf druid-bin zookeeper-bin
+
+    # Cleanup the generated runtime version files
+    do_cmd rm -f lib/uploads/config/*_version.txt 
 }
 
 fn_exists()

deployment/run-unit-tests.sh

@@ -13,6 +13,20 @@
 [ "$DEBUG" == 'true' ] && set -x
 set -e
 
+venv_folder="./.venv/"
+
+setup_python_env() {
+    if [ -d "$venv_folder" ]; then
+        echo "Re-using Python venv in $venv_folder"
+        return
+    fi
+
+    python3 -m venv $venv_folder
+    source $venv_folder/bin/activate
+    pip3 install -q -r requirements.txt -r requirements-test.txt
+    deactivate
+}
+
 prepare_jest_coverage_report() {
 	local component_name=$1
 
@@ -29,6 +43,40 @@ prepare_jest_coverage_report() {
     mv coverage $coverage_report_path
 }
 
+run_python_test() {
+    local component_path=$1
+    local component_name=$2
+
+    echo "------------------------------------------------------------------------------"
+    echo "[Test] Run lambda unit test with coverage for $component_path $component_name"
+    echo "------------------------------------------------------------------------------"
+
+    [ "${CLEAN:-true}" = "true" ] && rm -rf $venv_folder
+
+    setup_python_env
+    source $venv_folder/bin/activate
+
+    pytest_coverage_report_path=$coverage_reports_top_path/pytest
+    mkdir -p $pytest_coverage_report_path
+    coverage_report_path=$pytest_coverage_report_path/$component_name.coverage.xml
+    echo "Coverage report path set to $coverage_report_path"
+
+    python3 -m pytest --cov --cov-report=term-missing --cov-report "xml:$coverage_report_path"
+    if [ "$?" = "1" ]; then
+        echo "(deployment/run-unit-tests.sh) ERROR: there is likely output above." 1>&2
+        exit 1
+    fi
+
+	sed -i -e "s,<source>$source_dir,<source>source,g" $coverage_report_path
+
+    deactivate
+
+    if [ "${CLEAN:-true}" = "true" ]; then
+        # Note: leaving $source_dir/test/coverage-reports to allow further processing of coverage reports
+        rm -rf $venv_folder coverage .coverage
+    fi
+}
+
 run_javascript_test() {
     local component_path=$1
 	local component_name=$2
@@ -65,7 +113,7 @@ run_cdk_project_test() {
 	# export overrideWarningsEnabled=false
 
 	# run unit tests
-	npm run test -- -u
+	npm run test:jest -- -u
 
     # prepare coverage reports
 	prepare_jest_coverage_report $component_name
@@ -90,6 +138,19 @@ echo "Running unit tests"
 source_dir="$(cd $PWD/../source; pwd -P)"
 coverage_reports_top_path=$source_dir/coverage-reports
 
+# Test the Python Lambda functions
+cd $source_dir/lib/lambdas
+for folder in */ ; do
+    cd "$folder"
+    function_name=${PWD##*/}
+
+    if [ -e "requirements.txt" ]; then
+        run_python_test $source_dir/lib/lambdas $function_name
+    fi
+
+    cd ..
+done
+
 # Test the CDK project
 run_cdk_project_test $source_dir
 

source/.eslintrc.js

@@ -30,7 +30,7 @@ module.exports = {
         '@typescript-eslint/explicit-member-accessibility': ['warn'],
         '@typescript-eslint/naming-convention': [
             'error',
-            { selector: 'variableLike', format: ['camelCase'] },
+            { selector: 'variableLike', format: ['camelCase'], leadingUnderscore: 'allow' },
             { selector: 'memberLike', format: ['camelCase'] },
             { selector: 'typeLike', format: ['PascalCase'] },
         ],

source/DruidCloudwatchExtension/src/main/java/com/amazon/solutions/druid/cloudwatch/CloudwatchEmitter.java

@@ -1,17 +1,6 @@
 /* 
-  Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
-  
-  Licensed under the Apache License, Version 2.0 (the "License").
-  You may not use this file except in compliance with the License.
-  You may obtain a copy of the License at
-  
-      http://www.apache.org/licenses/LICENSE-2.0
-  
-  Unless required by applicable law or agreed to in writing, software
-  distributed under the License is distributed on an "AS IS" BASIS,
-  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-  See the License for the specific language governing permissions and
-  limitations under the License.
+ Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ SPDX-License-Identifier: Apache-2.0
 */
 package com.amazon.solutions.druid.cloudwatch;
 

source/DruidCloudwatchExtension/src/main/java/com/amazon/solutions/druid/cloudwatch/CloudwatchEmitterConfig.java

@@ -1,17 +1,6 @@
 /* 
-  Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
-  
-  Licensed under the Apache License, Version 2.0 (the "License").
-  You may not use this file except in compliance with the License.
-  You may obtain a copy of the License at
-  
-      http://www.apache.org/licenses/LICENSE-2.0
-  
-  Unless required by applicable law or agreed to in writing, software
-  distributed under the License is distributed on an "AS IS" BASIS,
-  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-  See the License for the specific language governing permissions and
-  limitations under the License.
+ Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ SPDX-License-Identifier: Apache-2.0
 */
 package com.amazon.solutions.druid.cloudwatch;
 
@@ -24,7 +13,7 @@
 @Data
 public class CloudwatchEmitterConfig {
     static final int CLOUDWATCH_METRICS_MEMORY_LIMIT = 100000000;
-    static final String SOLUTION_VERSION = "v1.0.2";
+    static final String SOLUTION_VERSION = "v1.0.3";
 
     @JsonProperty("batchSize")
     @Nullable

source/DruidCloudwatchExtension/src/main/java/com/amazon/solutions/druid/cloudwatch/CloudwatchEmitterModule.java

@@ -1,17 +1,6 @@
 /* 
-  Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
-  
-  Licensed under the Apache License, Version 2.0 (the "License").
-  You may not use this file except in compliance with the License.
-  You may obtain a copy of the License at
-  
-      http://www.apache.org/licenses/LICENSE-2.0
-  
-  Unless required by applicable law or agreed to in writing, software
-  distributed under the License is distributed on an "AS IS" BASIS,
-  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-  See the License for the specific language governing permissions and
-  limitations under the License.
+ Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ SPDX-License-Identifier: Apache-2.0
 */
 package com.amazon.solutions.druid.cloudwatch;
 

source/DruidCloudwatchExtension/src/main/java/com/amazon/solutions/druid/cloudwatch/CloudwatchMetricUnits.java

@@ -1,17 +1,6 @@
 /* 
-  Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
-  
-  Licensed under the Apache License, Version 2.0 (the "License").
-  You may not use this file except in compliance with the License.
-  You may obtain a copy of the License at
-  
-      http://www.apache.org/licenses/LICENSE-2.0
-  
-  Unless required by applicable law or agreed to in writing, software
-  distributed under the License is distributed on an "AS IS" BASIS,
-  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-  See the License for the specific language governing permissions and
-  limitations under the License.
+ Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ SPDX-License-Identifier: Apache-2.0
 */
 package com.amazon.solutions.druid.cloudwatch;
 

source/DruidCloudwatchExtension/src/main/java/com/amazon/solutions/druid/cloudwatch/DruidMonitoringMetricsFactory.java

@@ -1,17 +1,6 @@
 /* 
-  Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
-  
-  Licensed under the Apache License, Version 2.0 (the "License").
-  You may not use this file except in compliance with the License.
-  You may obtain a copy of the License at
-  
-      http://www.apache.org/licenses/LICENSE-2.0
-  
-  Unless required by applicable law or agreed to in writing, software
-  distributed under the License is distributed on an "AS IS" BASIS,
-  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-  See the License for the specific language governing permissions and
-  limitations under the License.
+ Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ SPDX-License-Identifier: Apache-2.0
 */
 package com.amazon.solutions.druid.cloudwatch;
 

source/DruidCloudwatchExtension/src/main/java/com/amazon/solutions/druid/cloudwatch/MemoryBoundLinkedBlockingQueue.java

@@ -1,17 +1,6 @@
 /* 
-  Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
-  
-  Licensed under the Apache License, Version 2.0 (the "License").
-  You may not use this file except in compliance with the License.
-  You may obtain a copy of the License at
-  
-      http://www.apache.org/licenses/LICENSE-2.0
-  
-  Unless required by applicable law or agreed to in writing, software
-  distributed under the License is distributed on an "AS IS" BASIS,
-  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-  See the License for the specific language governing permissions and
-  limitations under the License.
+ Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ SPDX-License-Identifier: Apache-2.0
 */
 package com.amazon.solutions.druid.cloudwatch;
 

source/DruidCloudwatchExtension/src/test/java/com/amazon/solutions/druid/cloudwatch/CloudwatchEmitterConfigTest.java

@@ -1,17 +1,6 @@
 /* 
-  Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
-  
-  Licensed under the Apache License, Version 2.0 (the "License").
-  You may not use this file except in compliance with the License.
-  You may obtain a copy of the License at
-  
-      http://www.apache.org/licenses/LICENSE-2.0
-  
-  Unless required by applicable law or agreed to in writing, software
-  distributed under the License is distributed on an "AS IS" BASIS,
-  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-  See the License for the specific language governing permissions and
-  limitations under the License.
+ Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ SPDX-License-Identifier: Apache-2.0
 */
 package com.amazon.solutions.druid.cloudwatch;
 
@@ -57,15 +46,15 @@ public void testGetBatchSize_withDefaultBatchSize() {
     @Test
     public void testToString_withValidConfig() {
         // arrange
-        CloudwatchEmitterConfig config = new CloudwatchEmitterConfig("test-cluster", 200, "v1.0.2");
+        CloudwatchEmitterConfig config = new CloudwatchEmitterConfig("test-cluster", 200, "v1.0.3");
 
         // act
         String actual = config.toString();
 
         // assert
         Assert.assertTrue(actual.contains("test-cluster"));
         Assert.assertTrue(actual.contains("200"));
-        Assert.assertTrue(actual.contains("v1.0.2"));
+        Assert.assertTrue(actual.contains("v1.0.3"));
         Assert.assertTrue(actual.contains("CloudwatchEmitterConfig"));
     }