diff --git a/build-parent/pom.xml b/build-parent/pom.xml index 9e4c1132b..aa64468af 100644 --- a/build-parent/pom.xml +++ b/build-parent/pom.xml @@ -66,6 +66,7 @@ 2.0.7 0.6.0 1.19.8 + 5.1.0 3.4.0 diff --git a/docs/modules/ROOT/pages/guides/guides-configuration-store.adoc b/docs/modules/ROOT/pages/guides/guides-configuration-store.adoc index 94cb34b5e..9fecc9a04 100644 --- a/docs/modules/ROOT/pages/guides/guides-configuration-store.adoc +++ b/docs/modules/ROOT/pages/guides/guides-configuration-store.adoc @@ -7,12 +7,6 @@ environment-specific configurations, based on the project's development lifecycl project https://github.com/TechnologyBrewery/krausening[Krausening]. Usage limitations and regeneration strategies can also be set, allowing configurations to automatically refreshed, thereby bolstering security of sensitive properties. -Currently, the Configuration Store supports integration with Vault. This is an excellent solution for large -production/test environments that need to have configuration encrypted at rest. However, this may not be ideal for -small or local test environments. For this the Configuration Store can be configured to leverage an in-memory -solution. Either solution will have the exact same usage and only differ in the setup and initialization of the -Configuration Store. - === Setup The Configuration Store tool is currently under development. Stay tuned for its release! @@ -80,6 +74,15 @@ client.getProperty("messaging", "auth-token") ---> auth-secret For more information on property definition and usage, visit https://github.com/TechnologyBrewery/krausening[Krausening]. +==== Properties Storage Options +The universal config store offers robust production ready configuration storage using vault. This is an excellent +solution for large enterprise applications. For smaller applications looking for a more light weight but still robust +storage option, the universal config can be set to use Krausening. The universal config can also be set up to use in +memory storage if you need to run some quick local tests. + +The default property storage is Krausening. To change it to use one of the other options you can set the environment +variable STORAGE_CLASS. The values currently available are `krausening`, `vault`, and `inMemory` + ==== Property Regeneration Policies Policies are used to define limitations and rules for when a given property should be regenerated. This allows the user to outline which properties they would like to be regenerated, when they would like them to be regenerated, and how to diff --git a/extensions/extensions-configuration-store-vault/LICENSE.txt b/extensions/extensions-configuration-store-vault/LICENSE.txt deleted file mode 100644 index a9bd80cda..000000000 --- a/extensions/extensions-configuration-store-vault/LICENSE.txt +++ /dev/null @@ -1,49 +0,0 @@ -Booz Allen Public License v1.0 ------------------------------- - -INTRODUCTION -The Booz Allen Public License allows government, non-profit academic, other non-profit, and commercial entities access to distinctive, disruptive, and robust code with the goal of Empowering People to Change the World(SM). Products licensed under the Booz Allen Public License are founded on the basis that collective ingenuity can make the largest impact in the community. - -DEFINITIONS -* **Commercial Entity.** “Commercial Entity” means any individual or entity other than a government, non-profit academic, or other non-profit entity. -* **Derivative.** “Derivative” means any work of authorship in Source Code or Object Code form that results from an addition to, deletion from, or modification of the Source Code of the Product. -* **License.** “License” means this Booz Allen Public License. -* **Object Code.** “Object Code” means the form resulting from transformation or translation of Source Code into machine readable code, including but not limited to, compiled object code. -* **Originator.** “Originator” means each individual or legal entity that creates, contributes to the creation of, or owns the Product. -* **Patent Claims.** “Patent Claims” means any patent claim(s) in any patent to which Originator has a right to grant a license that would be infringed by Your making, using, selling, offering for sale, having made, or importing of the Product, but for the grant of this License. -* **Product.** “Product” means the Source Code of the software which the initial Originator made available under this License, and any Derivative of such Source Code. -* **Source Code.** “Source Code” means software in human-readable form. -* **You.** “You” means either an individual or an entity (if you are taking this license on behalf of an entity) that exercises the rights granted under this License. - -LICENSE -**Government/Non-Profit Academic/Other Non-Profit.** -This Section applies if You are not a Commercial Entity. - -* **License.** Subject to the terms and conditions of this License, each Originator hereby grants You a perpetual, worldwide, non-exclusive, royalty-free license to reproduce, display, perform, modify, distribute and otherwise use the Product and Derivatives, in Source Code and Object Code form, in accordance with the terms and conditions of this License in order to support the general public good and for your internal business purposes. -* **Distribution.** You may distribute to third parties copies of the Product, including any Derivative that You create, in Source Code or Object Code form. If You distribute copies of the Product, including any Derivative that You create, in Source Code form, such distribution must be under the terms of this License and You must inform recipients of the Source Code that the Product is governed under this License and how they can obtain a copy of this License. You may distribute to third parties copies of the Product, including any Derivative that You create, in Object Code form, or allow third parties to access or use the Product, including any Derivative that You create, under a license of Your choice. -* **Commercial Sales.** You may not distribute, or allow third parties to access or use, the Product or any Derivative for a fee, unless You first obtain permission from the Originator. If Booz Allen Hamilton is the Originator, please contact Booz Allen Hamilton at . - -**Commercial Entities**. -This Section applies if You are a Commercial Entity. - -* **License.** Subject to the terms and conditions of this License, each Originator hereby grants You a perpetual, worldwide, non-exclusive, royalty-free license to reproduce, display, perform, modify, distribute and otherwise use the Product and Derivatives, in Source Code and Object Code form, in accordance with the terms and conditions of this License for the sole purpose of Your internal business purposes and the provision of services to government, non-profit academic, and other non-profit entities. -* **Distribution and Derivatives.** You may distribute to third parties copies of the Product, including any Derivative that You create, in Source Code or Object Code form. If You distribute copies of the Product, including any Derivative that You create, in Source Code form, such distribution must be under the terms of this License and You must inform recipients of the Source Code that the Product is governed under this License and how they can obtain a copy of this License. You may distribute to third parties copies of the Product, including any Derivative that You create, in Object Code form, or allow third parties to access or use the Product, including any Derivative that You create, under a license of Your choice, provided that You make available, and inform the recipient of such distribution how they can obtain, a copy of the Source Code thereof, at no charge, and inform the recipient of the Source Code that the Product is governed under this License and how they can obtain a copy of this License. -* **Commercial Sales.** You may not distribute, or allow third parties to access or use, the Product or any Derivative for a fee, unless You first obtain permission from the Originator. If Booz Allen Hamilton, please contact Booz Allen Hamilton at . - - -**Patent Claim(s)**. -This Section applies regardless of whether You are a government, non-profit academic, or other non-profit entity or a Commercial Entity. - -* **Patent License.** Subject to the limitations in the Sections above, each Originator hereby grants You a perpetual, worldwide, non-exclusive, royalty-free license under Patent Claims of such Originator to make, use, sell, offer for sale, have made, and import the Product. The foregoing patent license does not apply (a) to any code that an Originator has removed from the Product, or (b) for infringement caused by Your modifications of the Product or the combination of any Derivative created by You or on Your behalf with other software. - -GENERAL TERMS -This Section applies regardless of whether You are a government, non-profit academic, or other non-profit entity or a Commercial Entity. - -* **Required Notices.** If You distribute the Product or a Derivative, in Object Code or Source Code form, You shall not remove or otherwise modify any proprietary markings or notices contained within or placed upon the Product or any Derivative. Any distribution of the Product or a Derivative, in Object Code or Source Code form, shall contain a clear and conspicuous Originator copyright and license reference in accordance with the below: - * *Unmodified Product Notice*: “This software package is licensed under the Booz Allen Public License. Copyright © 20__ [Copyright Holder Name]. All Rights Reserved.” - * *Derivative Notice*: “This software package is licensed under the Booz Allen Public License. Portions of this code are Copyright © 20__ [Copyright Holder Name]. All Rights Reserved.” -* **Compliance with Laws.** You agree that You shall not reproduce, display, perform, modify, distribute and otherwise use the Product in any way that violates applicable law or regulation or infringes or violates the rights of others, including, but not limited to, third party intellectual property, privacy, and publicity rights. -* **Disclaimer.** You understand that the Product is licensed to You, and not sold. The Product is provided on an “As Is” basis, without any warranties, representations, and guarantees, whether oral or written, express, implied or statutory, with regard to the Product, including without limitation, warranties of merchantability, fitness for a particular purpose, title, non-infringement, non-interference, and warranties arising from course of dealing or usage of trade, to the maximum extent permitted by applicable law. Originator does not warrant that (i) the Product will meet your needs; (ii) the Product will be error-free or accessible at all times; or (iii) the use or the results of the use of the Product will be correct, accurate, timely, or otherwise reliable. You acknowledge that the Product has not been prepared to meet Your individual requirements, whether or not such requirements have been communicated to Originator. You assume all responsibility for use of the Product. -* **Limitation of Liability.** Under no circumstances and under no legal theory, whether tort (including negligence), contract, or otherwise, shall any Originator, or anyone who distributes the Product in accordance with this License, be liable to You for any direct, indirect, special, incidental, or consequential damages of any character including, without limitation, damages for lost profits, loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses, even if informed of the possibility of such damages. -* **Export Control.** The Product is subject to U.S. export control laws and may be subject to export or import regulations in other countries. You agree to strictly comply with all such laws and regulations and acknowledges that You are responsible for obtaining such licenses to export, re-export, or import as may be required. -* **Severability.** If the application of any provision of this License to any particular facts or circumstances shall be held to be invalid or unenforceable, then the validity and enforceability of other provisions of this License shall not in any way be affected or impaired thereby. diff --git a/extensions/extensions-configuration-store-vault/README.md b/extensions/extensions-configuration-store-vault/README.md deleted file mode 100644 index 92e0817df..000000000 --- a/extensions/extensions-configuration-store-vault/README.md +++ /dev/null @@ -1,26 +0,0 @@ -This module enables the foundation-configuration-store module ConfigLoader class to read/write configurations to Vault. - -1. Include this module to the dependency -```pom.xml - - com.boozallen.aissemble - extensions-configuration-store-vault - ${version.aissemble} - -``` -2. Instantiate the ConfigLoader with PropertyDao class -``` - ConfigLoader configLoader = new ConfigLoader(new PropertyDao()); -``` - -3. ConfigLoader can read/write configuration to Vault. e.g.: -``` - // write property to vault - Property property = new Property("myGroupName", "myTestKey", "myTestValue"); - configLoader.write(property); - - // read property from server - Property property = configLoader.read("myGroupName", "myTestKey"); -``` - - diff --git a/extensions/extensions-configuration-store-vault/pom.xml b/extensions/extensions-configuration-store-vault/pom.xml deleted file mode 100644 index e3b14c562..000000000 --- a/extensions/extensions-configuration-store-vault/pom.xml +++ /dev/null @@ -1,163 +0,0 @@ - - - 4.0.0 - - - com.boozallen.aissemble - extensions - 1.8.0-SNAPSHOT - - - extensions-configuration-store-vault - - aiSSEMBLE::Extensions::Configuration::Store::Vault - Provides centralized secure storage of configurations - jar - - 5.1.0 - - - - - org.apache.maven.plugins - maven-source-plugin - - - org.apache.maven.plugins - maven-surefire-plugin - - true - - ${project.version} - - - - - io.quarkus - quarkus-maven-plugin - true - - - - build - - - - - - org.apache.maven.plugins - maven-assembly-plugin - - - package - - single - - - - - - ${project.basedir}/src/main/resources/quarkus-app.xml - - quarkus - true - - - - - - - integration-test - - - - org.apache.maven.plugins - maven-surefire-plugin - - false - - - - net.masterthought - maven-cucumber-reporting - - - - - - net.masterthought - cucumber-reporting - ${version.cucumber.reporting.plugin} - - - - - - - - ${project.groupId} - bom-component - ${project.version} - pom - import - - - - - - io.cucumber - cucumber-java - test - - - io.cucumber - cucumber-junit - test - - - org.slf4j - slf4j-api - - - com.bettercloud - vault-java-driver - ${version.vault} - - - com.boozallen.aissemble - extensions-encryption-vault-java - ${version} - - - org.testcontainers - vault - ${version.testcontainers} - test - - - com.boozallen.aissemble - foundation-configuration-store - ${version} - - - io.quarkiverse.cucumber - quarkus-cucumber - - - io.quarkus - quarkus-junit5 - - - - - io.quarkus - quarkus-resteasy-reactive - ${version.quarkus} - - - io.quarkus - quarkus-core - ${version.quarkus} - - - diff --git a/extensions/extensions-configuration-store-vault/src/main/resources/META-INF/beans.xml b/extensions/extensions-configuration-store-vault/src/main/resources/META-INF/beans.xml deleted file mode 100644 index 1cc1e1761..000000000 --- a/extensions/extensions-configuration-store-vault/src/main/resources/META-INF/beans.xml +++ /dev/null @@ -1,17 +0,0 @@ - - - - - - diff --git a/extensions/extensions-configuration-store-vault/src/main/resources/quarkus-app.xml b/extensions/extensions-configuration-store-vault/src/main/resources/quarkus-app.xml deleted file mode 100644 index 7f1d33869..000000000 --- a/extensions/extensions-configuration-store-vault/src/main/resources/quarkus-app.xml +++ /dev/null @@ -1,28 +0,0 @@ - - - app - - false - - - zip - - - - - ${project.build.directory}/quarkus-app/ - quarkus-app/ - - - - diff --git a/extensions/extensions-configuration-store-vault/src/test/java/com/boozallen/aissemble/configuration/dao/VaultPropertyDaoSteps.java b/extensions/extensions-configuration-store-vault/src/test/java/com/boozallen/aissemble/configuration/dao/VaultPropertyDaoSteps.java deleted file mode 100644 index 9b325a8c7..000000000 --- a/extensions/extensions-configuration-store-vault/src/test/java/com/boozallen/aissemble/configuration/dao/VaultPropertyDaoSteps.java +++ /dev/null @@ -1,118 +0,0 @@ -package com.boozallen.aissemble.configuration.dao; - -/*- - * #%L - * aiSSEMBLE::Extensions::Configuration::Store::Vault - * %% - * Copyright (C) 2021 Booz Allen - * %% - * This software package is licensed under the Booz Allen Public License. All Rights Reserved. - * #L% - */ - - -import com.boozallen.aissemble.configuration.store.ConfigLoader; -import com.boozallen.aissemble.configuration.store.Property; -import io.cucumber.java.Before; -import io.cucumber.java.After; -import io.cucumber.java.en.Given; -import io.cucumber.java.en.When; -import io.cucumber.java.en.And; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.technologybrewery.krausening.Krausening; -import org.testcontainers.containers.wait.strategy.Wait; -import org.testcontainers.utility.DockerImageName; -import org.testcontainers.vault.VaultContainer; - -import java.io.IOException; -import java.io.InputStream; -import java.util.Properties; -import java.util.Set; - -import static org.junit.Assert.assertEquals; - -public class VaultPropertyDaoSteps { - - private static final Logger logger = LoggerFactory.getLogger(VaultPropertyDaoSteps.class); - private final VaultPropertyDao propertyDao = new VaultPropertyDao(); - private ConfigLoader configLoader = new ConfigLoader(); - private VaultContainer vaultContainer; - private Set properties; - private String baseURI; - private String environmentURI; - - @Before("@vault") - public void setup() { - setupVaultContainer(); - } - - @After("@vault") - public void cleanup() { - vaultContainer = null; - properties = null; - } - - @Before("@config-loader") - public void configLoaderSetup() { - configLoader = new ConfigLoader(); - configLoader.setPropertyDao(propertyDao); - properties = null; - } - - @After("@config-loader") - public void configLoaderCleanup() { - baseURI = null; - environmentURI = null; - configLoader = null; - } - - @Given("set of properties to be loaded") - public void configLoaderReadsThePropertiesAtURIs() { - baseURI = "src/test/resources/configurations/base"; - environmentURI = "src/test/resources/configurations/example-env"; - properties = configLoader.loadConfigs(baseURI, environmentURI); - } - - @When("the configurations are loaded") - public void theConfigurationsAreLoaded() { - configLoader.write(properties); - } - - @And("the properties can be read from the vault server") - public void theConfigLoaderUsesVaultPropertyDaoToWriteToVault() { - for (Property property : properties) { - assertEquals(propertyDao.read(property.getPropertyKey()), property); - } - } - - private void setupVaultContainer() { - final Properties encryptProperties = Krausening.getInstance().getProperties("encrypt.properties"); - String dockerImage = "ghcr.io/boozallen/aissemble-vault:" + System.getProperty("version.aissemble.vault"); - final DockerImageName vaultImage = DockerImageName.parse(dockerImage).asCompatibleSubstituteFor("vault"); - vaultContainer = new VaultContainer(vaultImage); - vaultContainer.setWaitStrategy(Wait.forListeningPort()); - vaultContainer.start(); - - // We override the secrets.host.url property in order to bring the port in-sync with the testcontainers port. - encryptProperties.setProperty("secrets.host.url", vaultContainer.getHttpHostAddress()); - - // Override secrets.root.key values from the aissemble-vault image for VaultPropertyDao vault access. - String rootKey = vaultContainer.copyFileFromContainer("/root_key.txt", this::inputStreamToString); - encryptProperties.setProperty("secrets.root.key", rootKey); - - // Override secrets.root.key values from the aissemble-vault image for VaultPropertyDao vault access. - String unsealKeysJSONArrayString = vaultContainer.copyFileFromContainer("/unseal_keys.txt", this::inputStreamToString); - encryptProperties.setProperty("secrets.unseal.keys", unsealKeysJSONArrayString.replaceAll("[\\s|\\[\\]\"]", "")); - } - - private String inputStreamToString(InputStream inputStream) { - try { - return new String(inputStream.readAllBytes()); - } catch (IOException e) { - logger.error("Failed to read key or token file from aissemble-vault container. Error while converting inputSteam to a String.", e); - } - return ""; - } - -} diff --git a/extensions/extensions-configuration-store-vault/src/test/resources/configurations/base/aws-credentials.yaml b/extensions/extensions-configuration-store-vault/src/test/resources/configurations/base/aws-credentials.yaml deleted file mode 100644 index 17ae79d24..000000000 --- a/extensions/extensions-configuration-store-vault/src/test/resources/configurations/base/aws-credentials.yaml +++ /dev/null @@ -1,6 +0,0 @@ -groupName: aws-access -properties: - - name: AWS_ACCESS_KEY_ID - value: base-access-key-id - - name: AWS_SECRET_ACCESS_KEY - value: base-secret-access-key \ No newline at end of file diff --git a/extensions/extensions-configuration-store-vault/src/test/resources/configurations/base/microprofile-config-data-lineage.yaml b/extensions/extensions-configuration-store-vault/src/test/resources/configurations/base/microprofile-config-data-lineage.yaml deleted file mode 100644 index e500182c3..000000000 --- a/extensions/extensions-configuration-store-vault/src/test/resources/configurations/base/microprofile-config-data-lineage.yaml +++ /dev/null @@ -1,10 +0,0 @@ -groupName: data-lineage-yaml-test -properties: - - name: connector - value: smallrye-kafka - - name: serializer - value: apache.StringSerializer - - name: topic - value: lineage-topic - - name: cloud-events - value: false \ No newline at end of file diff --git a/extensions/extensions-configuration-store-vault/src/test/resources/configurations/base/microprofile-config-messaging.yaml b/extensions/extensions-configuration-store-vault/src/test/resources/configurations/base/microprofile-config-messaging.yaml deleted file mode 100644 index 9f9e78e8c..000000000 --- a/extensions/extensions-configuration-store-vault/src/test/resources/configurations/base/microprofile-config-messaging.yaml +++ /dev/null @@ -1,8 +0,0 @@ -groupName: messaging-yaml-test -properties: - - name: connector - value: smallrye-kafka - - name: serializer - value: apache.StringSerializer - - name: topic - value: messaging-topic \ No newline at end of file diff --git a/extensions/extensions-configuration-store-vault/src/test/resources/configurations/example-env/aws-credentials.yaml b/extensions/extensions-configuration-store-vault/src/test/resources/configurations/example-env/aws-credentials.yaml deleted file mode 100644 index 840dd5e5e..000000000 --- a/extensions/extensions-configuration-store-vault/src/test/resources/configurations/example-env/aws-credentials.yaml +++ /dev/null @@ -1,6 +0,0 @@ -groupName: model-training-api -properties: - - name: AWS_ACCESS_KEY_ID - value: env-access-key-id - - name: AWS_SECRET_ACCESS_KEY - value: env-secret-access-key \ No newline at end of file diff --git a/extensions/extensions-configuration-store-vault/src/test/resources/configurations/example-env/microprofile-config-data-lineage.yaml b/extensions/extensions-configuration-store-vault/src/test/resources/configurations/example-env/microprofile-config-data-lineage.yaml deleted file mode 100644 index 56a706de7..000000000 --- a/extensions/extensions-configuration-store-vault/src/test/resources/configurations/example-env/microprofile-config-data-lineage.yaml +++ /dev/null @@ -1,6 +0,0 @@ -groupName: data-lineage -properties: - - name: cloud-events - value: true - - name: added-property - value: example-value \ No newline at end of file diff --git a/extensions/extensions-configuration-store-vault/src/test/resources/krausening/arm64/encrypt.properties b/extensions/extensions-configuration-store-vault/src/test/resources/krausening/arm64/encrypt.properties deleted file mode 100644 index 3c338ed97..000000000 --- a/extensions/extensions-configuration-store-vault/src/test/resources/krausening/arm64/encrypt.properties +++ /dev/null @@ -1,14 +0,0 @@ -### -# #%L -# aiSSEMBLE Data Encryption::Encryption (Python) -# %% -# Copyright (C) 2021 Booz Allen -# %% -# This software package is licensed under the Booz Allen Public License. All Rights Reserved. -# #L% -### -# Overridden in PropertyDao.java to work around TestContainers random host port -secrets.host.url=http://127.0.0.1:8200 -# Overridden in PropertyDao.java with keys from the aissemble-vault image: -secrets.unseal.keys= -secrets.root.key= diff --git a/extensions/extensions-configuration-store-vault/src/test/resources/krausening/base/encrypt.properties b/extensions/extensions-configuration-store-vault/src/test/resources/krausening/base/encrypt.properties deleted file mode 100644 index 3c338ed97..000000000 --- a/extensions/extensions-configuration-store-vault/src/test/resources/krausening/base/encrypt.properties +++ /dev/null @@ -1,14 +0,0 @@ -### -# #%L -# aiSSEMBLE Data Encryption::Encryption (Python) -# %% -# Copyright (C) 2021 Booz Allen -# %% -# This software package is licensed under the Booz Allen Public License. All Rights Reserved. -# #L% -### -# Overridden in PropertyDao.java to work around TestContainers random host port -secrets.host.url=http://127.0.0.1:8200 -# Overridden in PropertyDao.java with keys from the aissemble-vault image: -secrets.unseal.keys= -secrets.root.key= diff --git a/extensions/extensions-configuration-store-vault/src/test/resources/specifications/vault-property-dao.feature b/extensions/extensions-configuration-store-vault/src/test/resources/specifications/vault-property-dao.feature deleted file mode 100644 index f83db19a6..000000000 --- a/extensions/extensions-configuration-store-vault/src/test/resources/specifications/vault-property-dao.feature +++ /dev/null @@ -1,8 +0,0 @@ -@vault @integration -Feature: Configuration can be stored to Vault - - @config-loader - Scenario: The ConfigLoader can read and write to the Vault server - Given set of properties to be loaded - When the configurations are loaded - And the properties can be read from the vault server \ No newline at end of file diff --git a/extensions/pom.xml b/extensions/pom.xml index 983e5bb55..94d213353 100644 --- a/extensions/pom.xml +++ b/extensions/pom.xml @@ -32,6 +32,5 @@ extensions-helm aissemble-extensions-model-training-api-sagemaker extensions-pipeline-invocation-service - extensions-configuration-store-vault diff --git a/foundation/foundation-configuration-store/pom.xml b/foundation/foundation-configuration-store/pom.xml index 19b2baceb..5827f45f1 100644 --- a/foundation/foundation-configuration-store/pom.xml +++ b/foundation/foundation-configuration-store/pom.xml @@ -13,7 +13,26 @@ aiSSEMBLE::Foundation::Configuration::Store Provides centralized secure storage of configurations jar + + + + + org.apache.maven.plugins + maven-failsafe-plugin + ${version.failsafe.plugin} + + + ${project.basedir}/src/test/resources/configurations/base + ${project.basedir}/src/test/resources/configurations/example-env + 3uQ2j_=wmP5A2q8b + ${project.version} + + + + + + org.apache.maven.plugins @@ -60,7 +79,11 @@ ${project.basedir}/src/test/resources/configurations/base ${project.basedir}/src/test/resources/configurations/example-env 3uQ2j_=wmP5A2q8b + ${project.version} + + com.boozallen.aissemble.util.TestPropertyDao + @@ -69,6 +92,7 @@ + @@ -86,6 +110,7 @@ + com.boozallen.aissemble @@ -129,6 +154,24 @@ quarkus-smallrye-openapi + + + com.bettercloud + vault-java-driver + ${version.vault} + + + com.boozallen.aissemble + extensions-encryption-vault-java + ${version} + + + org.testcontainers + vault + ${version.testcontainers} + test + + io.quarkus diff --git a/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/ConfigStoreInit.java b/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/ConfigStoreInit.java index 3b410bc88..1e0c7222c 100644 --- a/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/ConfigStoreInit.java +++ b/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/ConfigStoreInit.java @@ -10,11 +10,14 @@ * #L% */ +import com.boozallen.aissemble.configuration.dao.DaoClass; import com.boozallen.aissemble.configuration.store.ConfigLoader; import com.boozallen.aissemble.configuration.store.Property; import com.boozallen.aissemble.configuration.policy.PropertyRegenerationPolicy; import io.quarkus.runtime.Startup; + +import org.apache.commons.lang3.EnumUtils; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -39,20 +42,41 @@ public class ConfigStoreInit { @PostConstruct public void init() { logger.info("Initialize store configuration properties and policies..."); - // We are loading all the properties upfront. In the future it might be more advantageous to load them as needed - String basePropertyUri = getBootstrapConfiguration("KRAUSENING_BASE"); - String environmentPropertyUri = getBootstrapConfiguration("KRAUSENING_EXTENSIONS"); - String basePolicyUri = getBootstrapConfiguration("BASE_POLICY_URI"); - String environmentPolicyUri = getBootstrapConfiguration("ENVIRONMENT_POLICY_URI"); + + // Get users desired storage class. If not set then default to Krausening + String storageClass = getBootstrapConfiguration("STORAGE_CLASS") != null ? + getBootstrapConfiguration("STORAGE_CLASS") : DaoClass.krausening.name(); + String propertyDaoClass = EnumUtils.isValidEnum(DaoClass.class, storageClass)? + DaoClass.valueOf(storageClass).getValue() : storageClass; + + logger.info("Using property Dao class: {}", propertyDaoClass); try { ConfigLoader configLoader = CDI.current().select(ConfigLoader.class,new Any.Literal()).get(); + configLoader.setPropertyDaoClass(propertyDaoClass); if (configLoader.isFullyLoaded()) { - logger.info("Properties are already fully loaded. Skipping reload."); + logger.info("Properties and polices are already fully loaded. Skipping reload."); status = Status.LOAD_SKIPPED; return; } + loadConfigs(configLoader); + loadPolicies(configLoader); + + logger.info("Successfully initialized store configuration properties and policies."); + status = Status.LOAD_COMPLETE; + configLoader.updateLoadStatus(true); + + } catch (Exception e) { + logger.warn("Error loading properties:", e); + } + } + + private void loadConfigs(ConfigLoader configLoader) { + if(configLoader.doInitialConfigLoad()) { + String basePropertyUri = getBootstrapConfiguration("KRAUSENING_BASE"); + String environmentPropertyUri = getBootstrapConfiguration("KRAUSENING_EXTENSIONS"); + // Load and validate the properties Set properties; if (StringUtils.isNotBlank(basePropertyUri)) { @@ -68,32 +92,31 @@ public void init() { else { throw new RuntimeException("Undefined environment variables: KRAUSENING_BASE and KRAUSENING_EXTENSIONS"); } + configLoader.write(properties); + } else { + logger.info("Skipping config loading on initialization"); + } + } - // Load and validate the policies - Set policies; - if (StringUtils.isNotBlank(basePolicyUri)) { - if (StringUtils.isNotBlank(environmentPolicyUri)) { - logger.debug("Loading base and environment policies"); - policies = configLoader.loadPolicies(basePolicyUri,environmentPolicyUri); - } - else { - logger.debug("Loading base policies only"); - policies = configLoader.loadPolicies(basePolicyUri); - } - } + private void loadPolicies(ConfigLoader configLoader) { + // Load and validate the policies + String basePolicyUri = getBootstrapConfiguration("BASE_POLICY_URI"); + String environmentPolicyUri = getBootstrapConfiguration("ENVIRONMENT_POLICY_URI"); + Set policies; + if (StringUtils.isNotBlank(basePolicyUri)) { + if (StringUtils.isNotBlank(environmentPolicyUri)) { + logger.debug("Loading base and environment policies"); + policies = configLoader.loadPolicies(basePolicyUri,environmentPolicyUri); + } else { - logger.info("No policies found to load"); + logger.debug("Loading base policies only"); + policies = configLoader.loadPolicies(basePolicyUri); } - - configLoader.write(properties); - // TODO configLoader.write(policies); - - logger.info("Successfully initialized store configuration properties and policies."); - status = Status.LOAD_COMPLETE; - - } catch (Exception e) { - logger.warn("Error loading properties:", e); } + else { + logger.info("No policies found to load"); + } + // TODO configLoader.write(policies); } private static String getBootstrapConfiguration(String propertyName) { @@ -115,7 +138,7 @@ private enum Status { LOAD_COMPLETE("Load Complete"), LOAD_SKIPPED("Load Skipped"); - private String value; + private final String value; Status(final String value){ this.value = value; diff --git a/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/DaoClass.java b/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/DaoClass.java new file mode 100644 index 000000000..404de09cb --- /dev/null +++ b/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/DaoClass.java @@ -0,0 +1,27 @@ +package com.boozallen.aissemble.configuration.dao; + +/*- + * #%L + * aiSSEMBLE::Foundation::Configuration::Store + * %% + * Copyright (C) 2021 Booz Allen + * %% + * This software package is licensed under the Booz Allen Public License. All Rights Reserved. + * #L% + */ + +public enum DaoClass { + vault("com.boozallen.aissemble.configuration.dao.VaultPropertyDao"), + inMemory("com.boozallen.aissemble.configuration.dao.InMemoryPropertyDao"), + krausening("com.boozallen.aissemble.configuration.dao.KrauseningPropertyDao"); + + private final String value; + DaoClass(final String value){ + this.value = value; + } + + public String getValue(){ + return this.value; + } + +} diff --git a/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/InMemoryPropertyDao.java b/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/InMemoryPropertyDao.java index cb027ef95..a3e304679 100644 --- a/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/InMemoryPropertyDao.java +++ b/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/InMemoryPropertyDao.java @@ -58,4 +58,9 @@ public void write(Set properties) { } logger.info("Write properties request received"); } + + @Override + public boolean requiresInitialConfigLoad() { + return true; + } } diff --git a/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/KrauseningPropertyDao.java b/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/KrauseningPropertyDao.java new file mode 100644 index 000000000..275a4ee91 --- /dev/null +++ b/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/KrauseningPropertyDao.java @@ -0,0 +1,89 @@ +package com.boozallen.aissemble.configuration.dao; + +/*- + * #%L + * aiSSEMBLE::Foundation::Configuration::Store + * %% + * Copyright (C) 2021 Booz Allen + * %% + * This software package is licensed under the Booz Allen Public License. All Rights Reserved. + * #L% + */ + + +import java.io.FileOutputStream; +import java.io.IOException; +import java.util.Properties; +import java.util.Set; + +import javax.enterprise.context.ApplicationScoped; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.technologybrewery.krausening.Krausening; + +import com.boozallen.aissemble.configuration.store.Property; +import com.boozallen.aissemble.configuration.store.PropertyKey; + +/** + * InMemoryPropertyDao + * This is not a production ready class, which serves as a simple save to memory property dao class + * ,and it can be used to check whether the request is received + */ + +@ApplicationScoped +public class KrauseningPropertyDao implements PropertyDao { + private static final Logger logger = LoggerFactory.getLogger(KrauseningPropertyDao.class); + private static final String PROPERTIES_FILE_EXTENSION="properties"; + + @Override + public boolean checkEmpty() { + throw new RuntimeException("not implemented"); + } + + @Override + public Property read(PropertyKey propertyKey) { + logger.info(String.format("Read property request received, groupName: %s, propertyName: %s", + propertyKey.getGroupName(), propertyKey.getPropertyName())); + String fileName = String.format("%s.%s", propertyKey.getGroupName(), PROPERTIES_FILE_EXTENSION); + Krausening krausening = Krausening.getInstance(); + Properties krauseningProperties = krausening.getProperties(fileName); + + // If the config is not found then return null + if(krauseningProperties == null || krauseningProperties.get(propertyKey.getPropertyName()) == null){ + return null; + } + return new Property(propertyKey, krauseningProperties.get(propertyKey.getPropertyName()).toString()); + } + + @Override + public void write(Property property) { + try { + logger.info(String.format("Write property request received, groupName: %s, propertyName: %s", + property.getPropertyKey().getGroupName(), property.getPropertyKey().getPropertyName())); + String fileName = String.format("%s.%s", property.getPropertyKey().getGroupName(), + PROPERTIES_FILE_EXTENSION); + Krausening krausening = Krausening.getInstance(); + Properties properties = krausening.getProperties(fileName) != null? krausening.getProperties(fileName): + new Properties(); + properties.setProperty(property.getPropertyKey().getPropertyName(), property.getValue()); + properties.store(new FileOutputStream(String.format("%s/%s", System.getProperty("KRAUSENING_BASE"), + fileName)), null); + }catch (IOException e){ + logger.error("Error updating properties.", e); + } + } + + @Override + public void write(Set properties) { + logger.info("Write properties request received"); + for (Property property: properties) { + this.write(property); + } + } + + @Override + public boolean requiresInitialConfigLoad() { + return false; + } +} diff --git a/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/PropertyDao.java b/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/PropertyDao.java index 53ab21ef9..79f512e96 100644 --- a/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/PropertyDao.java +++ b/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/PropertyDao.java @@ -22,7 +22,7 @@ public interface PropertyDao extends Dao { /** * Read property from store with given {@link PropertyKey} containing the group name and property name - * @param PropertyKey property key + * @param propertyKey property key * @return Property */ Property read(PropertyKey propertyKey); @@ -39,4 +39,10 @@ public interface PropertyDao extends Dao { * @param properties to be written to store */ void write(Set properties); + + /** + * Determines if the configs need to be loaded on start up + * @return true if the properties need to be loaded on start up + */ + boolean requiresInitialConfigLoad(); } diff --git a/extensions/extensions-configuration-store-vault/src/main/java/com/boozallen/aissemble/configuration/dao/VaultPropertyDao.java b/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/VaultPropertyDao.java similarity index 97% rename from extensions/extensions-configuration-store-vault/src/main/java/com/boozallen/aissemble/configuration/dao/VaultPropertyDao.java rename to foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/VaultPropertyDao.java index 558054a3c..b554c4252 100644 --- a/extensions/extensions-configuration-store-vault/src/main/java/com/boozallen/aissemble/configuration/dao/VaultPropertyDao.java +++ b/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/dao/VaultPropertyDao.java @@ -48,7 +48,7 @@ public boolean checkEmpty() { /** * Read the property from Vault server with the given {@link PropertyKey} containing the group name and property name - * @param PropertyKey property key + * @param propertyKey property key * @return Property */ @Override @@ -102,6 +102,11 @@ public void write(Set properties) { } + @Override + public boolean requiresInitialConfigLoad() { + return true; + } + private void write(Vault vault, List properties) throws VaultException { for (Property property : properties) { // path: aissemble-properties/<> diff --git a/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/store/ConfigLoader.java b/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/store/ConfigLoader.java index b20e56c19..ab26b8fe6 100644 --- a/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/store/ConfigLoader.java +++ b/foundation/foundation-configuration-store/src/main/java/com/boozallen/aissemble/configuration/store/ConfigLoader.java @@ -29,6 +29,7 @@ import javax.enterprise.inject.Instance; import javax.inject.Inject; +import org.apache.commons.lang3.BooleanUtils; import org.eclipse.microprofile.config.inject.ConfigProperty; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -48,17 +49,26 @@ public class ConfigLoader { private static final Logger logger = LoggerFactory.getLogger(ConfigLoader.class); private PropertyDao propertyDao; - - @ConfigProperty(name = "config.store.property.dao.class") - public String propertyDaoClass; + private Instance instances; @Inject public void setPropertyDao(Instance instances) { - instances.forEach(propertyDao -> { + this.instances = instances; + } + + public void setPropertyDaoClass(String propertyDaoClass) throws RuntimeException { + this.instances.forEach(propertyDao -> { if (propertyDao.getClass().getName().contains(propertyDaoClass)) { this.propertyDao = propertyDao; } }); + if(this.propertyDao == null){ + throw new RuntimeException("Invalid Storage Class"); + } + } + + protected PropertyDao getPropertyDao() { + return this.propertyDao; } public void setPropertyDao(PropertyDao propertyDao) { @@ -86,6 +96,10 @@ public Set loadConfigs(String baseURI) { return loadPropertiesURI(baseURI); } + public boolean doInitialConfigLoad(){ + return propertyDao.requiresInitialConfigLoad(); + } + /** * Inspects config file(s) at the given URI and gathers all properties. * @param uri Location housing configuration properties. @@ -256,7 +270,6 @@ public void write(Set properties) { try { propertyDao.write(properties); logger.info("Successfully wrote all properties to the store."); - updateLoadStatus(true); } catch (Exception e) { logger.error("Error updating properties.", e); updateLoadStatus(false); @@ -276,14 +289,14 @@ public Property read(PropertyKey propertyKey) { public boolean isFullyLoaded() { try { Property statusProperty = propertyDao.read(new PropertyKey("load-status", "fully-loaded")); - return statusProperty != null && "true".equals(statusProperty.getValue()); + return statusProperty != null && BooleanUtils.toBoolean(statusProperty.getValue()); } catch (Exception e) { logger.warn("Properties are not loaded previously, continue", e); return false; } } - private void updateLoadStatus(boolean status) { + public void updateLoadStatus(boolean status) { Property statusProperty = new Property("load-status", "fully-loaded", String.valueOf(status)); try { propertyDao.write(statusProperty); diff --git a/foundation/foundation-configuration-store/src/main/resources/application.properties b/foundation/foundation-configuration-store/src/main/resources/application.properties index 3caef79a1..f34485d98 100644 --- a/foundation/foundation-configuration-store/src/main/resources/application.properties +++ b/foundation/foundation-configuration-store/src/main/resources/application.properties @@ -10,4 +10,3 @@ quarkus.http.port=8080 quarkus.http.test-port=14800 -config.store.property.dao.class=com.boozallen.aissemble.configuration.dao.InMemoryPropertyDao \ No newline at end of file diff --git a/extensions/extensions-configuration-store-vault/src/test/java/CucumberTest.java b/foundation/foundation-configuration-store/src/test/java/CucumberIT.java similarity index 53% rename from extensions/extensions-configuration-store-vault/src/test/java/CucumberTest.java rename to foundation/foundation-configuration-store/src/test/java/CucumberIT.java index 06fd957d8..a288ebc5f 100644 --- a/extensions/extensions-configuration-store-vault/src/test/java/CucumberTest.java +++ b/foundation/foundation-configuration-store/src/test/java/CucumberIT.java @@ -1,21 +1,24 @@ /*- * #%L - * aiSSEMBLE::Extensions::Configuration::Store::Vault + * aiSSEMBLE::Foundation::Configuration::Store * %% * Copyright (C) 2021 Booz Allen * %% * This software package is licensed under the Booz Allen Public License. All Rights Reserved. * #L% */ -import io.cucumber.junit.Cucumber; -import io.cucumber.junit.CucumberOptions; -import org.junit.runner.RunWith; -@RunWith(Cucumber.class) +import io.quarkiverse.cucumber.CucumberOptions; + +import io.quarkiverse.cucumber.CucumberQuarkusTest; + @CucumberOptions( features = "src/test/resources/specifications", plugin = {"json:target/cucumber-reports/cucumber.json"}, tags = "@integration") -public class CucumberTest { - +public class CucumberIT extends CucumberQuarkusTest { + public static void main(String[] args) { + runMain(CucumberIT.class, args); + } } + diff --git a/foundation/foundation-configuration-store/src/test/java/CucumberTest.java b/foundation/foundation-configuration-store/src/test/java/CucumberTest.java index 5a9e73dda..a6d150d0f 100644 --- a/foundation/foundation-configuration-store/src/test/java/CucumberTest.java +++ b/foundation/foundation-configuration-store/src/test/java/CucumberTest.java @@ -14,7 +14,7 @@ @CucumberOptions( features = "src/test/resources/specifications", plugin = {"json:target/cucumber-reports/cucumber.json"}, - tags = "not @manual") + tags = "not @manual and not @integration") public class CucumberTest extends CucumberQuarkusTest { public static void main(String[] args) { runMain(CucumberTest.class, args); diff --git a/foundation/foundation-configuration-store/src/test/java/com/boozallen/aissemble/configuration/store/LoadConfigurationsSteps.java b/foundation/foundation-configuration-store/src/test/java/com/boozallen/aissemble/configuration/store/LoadConfigurationsSteps.java index f4db68193..adb985834 100644 --- a/foundation/foundation-configuration-store/src/test/java/com/boozallen/aissemble/configuration/store/LoadConfigurationsSteps.java +++ b/foundation/foundation-configuration-store/src/test/java/com/boozallen/aissemble/configuration/store/LoadConfigurationsSteps.java @@ -13,14 +13,28 @@ import static io.restassured.RestAssured.given; import static org.hamcrest.CoreMatchers.is; import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertNotNull; import static org.junit.Assert.assertNull; import static org.junit.Assert.assertTrue; +import java.io.IOException; +import java.io.InputStream; import java.util.HashSet; +import java.util.Properties; import java.util.Set; +import javax.enterprise.inject.Any; +import javax.enterprise.inject.spi.CDI; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.technologybrewery.krausening.Krausening; +import org.testcontainers.containers.wait.strategy.Wait; +import org.testcontainers.utility.DockerImageName; +import org.testcontainers.vault.VaultContainer; + import com.boozallen.aissemble.configuration.ConfigStoreInit; +import com.boozallen.aissemble.configuration.dao.DaoClass; +import com.boozallen.aissemble.configuration.dao.PropertyDao; import com.boozallen.aissemble.configuration.policy.PropertyRegenerationPolicy; import com.boozallen.aissemble.configuration.policy.exception.PropertyRegenerationPolicyException; import com.boozallen.aissemble.util.TestPropertyDao; @@ -35,9 +49,12 @@ public class LoadConfigurationsSteps { + private static final Logger logger = LoggerFactory.getLogger(LoadConfigurationsSteps.class); private final Property expectedProperty = new Property("microprofile-config-messaging", "topic", "messaging-topic"); private final Property expectedDecryptedProperty = new Property("aws-credentials", "AWS_SECRET_ACCESS_KEY", "env-secret-access-key"); private final Property fullyLoadProperty = new Property("load-status", "fully-loaded", "true"); + private Set properties; + private PropertyDao propertyDao; private Set policies; private ConfigLoader configLoader; private static final String basePropertyUri = "src/test/resources/configurations/base"; @@ -46,6 +63,7 @@ public class LoadConfigurationsSteps { private String environmentPolicyURI; private Exception foundError; private ValidatableResponse response; + private VaultContainer vaultContainer; @Before("@config-loader") public void setup() { @@ -59,6 +77,17 @@ public void cleanup() { policies = null; } + @Before("@vault") + public void setupVault() { + setupVaultContainer(); + } + + @After("@vault") + public void cleanupVault() { + vaultContainer = null; + properties = null; + } + @Given("URIs pointing to valid base and environment properties") public void URIsPointingToValidBaseAndEnvironmentProperties() { // Set in statics @@ -115,7 +144,7 @@ public void notifyOnPreviousConfigurationFullLoad() { @When("the properties are loaded") public void thePropertiesAreLoaded() { try { - configLoader.loadConfigs(basePropertyUri, environmentPropertyURI); + properties = configLoader.loadConfigs(basePropertyUri, environmentPropertyURI); } catch (Exception error) { foundError = error; } @@ -133,7 +162,7 @@ public void consumesTheBaseProperties() { @Then("augments the base with the environment properties") public void augmentsTheBaseWithTheEnvironmentConfigurations() { - assertEquals(10, TestPropertyDao.loadedProperties.size()); + assertEquals(14, TestPropertyDao.loadedProperties.size()); assertPropertySetsEqual(createExpectedProperties(), new HashSet<>(TestPropertyDao.loadedProperties.values())); } @@ -223,6 +252,54 @@ public void theLoadedPropertiesContainsTheDecryptedValue() { assertEquals(expectedDecryptedProperty.toJsonString(), responseBody); } + @Given("the config loader to configured to use {string}") + public void theConfigLoaderToConfiguredToUse(String propertyDao) { + configLoader = CDI.current().select(ConfigLoader.class, new Any.Literal()).get(); + configLoader.setPropertyDaoClass(DaoClass.valueOf(propertyDao).getValue()); + this.propertyDao = configLoader.getPropertyDao(); + } + + @When("the configurations are written") + public void theConfigurationsAreWritten() { + configLoader.write(properties); + } + + @Then("the properties can be read") + public void thePropertiesCanBeRead() { + for (Property property : properties) { + assertEquals(propertyDao.read(property.getPropertyKey()), property); + } + } + + private void setupVaultContainer() { + final Properties encryptProperties = Krausening.getInstance().getProperties("encrypt.properties"); + String dockerImage = "ghcr.io/boozallen/aissemble-vault:" + System.getProperty("version.aissemble.vault"); + final DockerImageName vaultImage = DockerImageName.parse(dockerImage).asCompatibleSubstituteFor("vault"); + vaultContainer = new VaultContainer(vaultImage); + vaultContainer.setWaitStrategy(Wait.forListeningPort()); + vaultContainer.start(); + + // We override the secrets.host.url property in order to bring the port in-sync with the testcontainers port. + encryptProperties.setProperty("secrets.host.url", vaultContainer.getHttpHostAddress()); + + // Override secrets.root.key values from the aissemble-vault image for VaultPropertyDao vault access. + String rootKey = vaultContainer.copyFileFromContainer("/root_key.txt", this::inputStreamToString); + encryptProperties.setProperty("secrets.root.key", rootKey); + + // Override secrets.root.key values from the aissemble-vault image for VaultPropertyDao vault access. + String unsealKeysJSONArrayString = vaultContainer.copyFileFromContainer("/unseal_keys.txt", this::inputStreamToString); + encryptProperties.setProperty("secrets.unseal.keys", unsealKeysJSONArrayString.replaceAll("[\\s|\\[\\]\"]", "")); + } + + private String inputStreamToString(InputStream inputStream) { + try { + return new String(inputStream.readAllBytes()); + } catch (IOException e) { + logger.error("Failed to read key or token file from aissemble-vault container. Error while converting inputSteam to a String.", e); + } + return ""; + } + public Set createExpectedProperties() { Set expectedProperties = new HashSet<>(); @@ -235,6 +312,11 @@ public Set createExpectedProperties() { expectedProperties.add(new Property("microprofile-config-data-lineage", "added-property", "example-value")); expectedProperties.add(new Property("microprofile-config-messaging", "connector", "smallrye-kafka")); expectedProperties.add(new Property("microprofile-config-messaging", "serializer", "apache.StringSerializer")); + + expectedProperties.add(new Property("encrypt", "secrets.unseal.keys", "")); + expectedProperties.add(new Property("encrypt", "secrets.root.key", "")); + expectedProperties.add(new Property("encrypt", "secrets.host.url", "http://127.0.0.1:8200")); + expectedProperties.add(new Property("load-status", "fully-loaded", "true")); expectedProperties.add(expectedProperty); return expectedProperties; diff --git a/foundation/foundation-configuration-store/src/test/resources/application.properties b/foundation/foundation-configuration-store/src/test/resources/application.properties index 30607ad4a..4c855370d 100644 --- a/foundation/foundation-configuration-store/src/test/resources/application.properties +++ b/foundation/foundation-configuration-store/src/test/resources/application.properties @@ -10,5 +10,4 @@ quarkus.http.port=8080 quarkus.http.test-port=14800 -config.store.property.dao.class=com.boozallen.aissemble.util.TestPropertyDao -quarkus.kafka.devservices.image-name=vectorized/redpanda:v24.1.7 \ No newline at end of file +quarkus.kafka.devservices.image-name=vectorized/redpanda:v24.1.7 diff --git a/foundation/foundation-configuration-store/src/test/resources/configurations/base/aws-credentials.properties b/foundation/foundation-configuration-store/src/test/resources/configurations/base/aws-credentials.properties index 275ef5b69..f966a31fe 100644 --- a/foundation/foundation-configuration-store/src/test/resources/configurations/base/aws-credentials.properties +++ b/foundation/foundation-configuration-store/src/test/resources/configurations/base/aws-credentials.properties @@ -7,6 +7,5 @@ # This software package is licensed under the Booz Allen Public License. All Rights Reserved. # #L% ### -AWS_ACCESS_KEY_ID=base-access-key-id -# This is a test only value and holds no significance -AWS_SECRET_ACCESS_KEY=ENC(Hs0eE3pKIxqntLXVuqgBJE9e9uFy29W8bMDocqbiakbFPhQf6Xp5oAL1Z6m6tdd3i0sEREN2pX2chS4q6KxiIw==) +AWS_ACCESS_KEY_ID=env-access-key-id +AWS_SECRET_ACCESS_KEY=env-secret-access-key diff --git a/foundation/foundation-configuration-store/src/test/resources/configurations/base/encrypt.properties b/foundation/foundation-configuration-store/src/test/resources/configurations/base/encrypt.properties new file mode 100644 index 000000000..4b8f3f935 --- /dev/null +++ b/foundation/foundation-configuration-store/src/test/resources/configurations/base/encrypt.properties @@ -0,0 +1,12 @@ +### +# #%L +# aiSSEMBLE::Foundation::Configuration::Store +# %% +# Copyright (C) 2021 Booz Allen +# %% +# This software package is licensed under the Booz Allen Public License. All Rights Reserved. +# #L% +### +secrets.unseal.keys= +secrets.root.key= +secrets.host.url=http\://127.0.0.1\:8200 diff --git a/extensions/extensions-configuration-store-vault/src/main/resources/application.properties b/foundation/foundation-configuration-store/src/test/resources/configurations/base/load-status.properties similarity index 52% rename from extensions/extensions-configuration-store-vault/src/main/resources/application.properties rename to foundation/foundation-configuration-store/src/test/resources/configurations/base/load-status.properties index 5e68a182e..d6fd32c39 100644 --- a/extensions/extensions-configuration-store-vault/src/main/resources/application.properties +++ b/foundation/foundation-configuration-store/src/test/resources/configurations/base/load-status.properties @@ -1,10 +1,10 @@ ### # #%L -# aiSSEMBLE::Extensions::Configuration::Store::Vault +# aiSSEMBLE::Foundation::Configuration::Store # %% # Copyright (C) 2021 Booz Allen # %% # This software package is licensed under the Booz Allen Public License. All Rights Reserved. # #L% ### -config.store.property.dao.class=com.boozallen.aissemble.configuration.dao.VaultPropertyDao +fully-loaded=true diff --git a/foundation/foundation-configuration-store/src/test/resources/configurations/base/microprofile-config-data-lineage.properties b/foundation/foundation-configuration-store/src/test/resources/configurations/base/microprofile-config-data-lineage.properties index ce86ccc1c..9a81e2957 100644 --- a/foundation/foundation-configuration-store/src/test/resources/configurations/base/microprofile-config-data-lineage.properties +++ b/foundation/foundation-configuration-store/src/test/resources/configurations/base/microprofile-config-data-lineage.properties @@ -7,7 +7,8 @@ # This software package is licensed under the Booz Allen Public License. All Rights Reserved. # #L% ### +cloud-events=true connector=smallrye-kafka serializer=apache.StringSerializer topic=lineage-topic -cloud-events=false +added-property=example-value diff --git a/foundation/foundation-configuration-store/src/test/resources/specifications/load-configurations.feature b/foundation/foundation-configuration-store/src/test/resources/specifications/load-configurations.feature index d77408fd2..70e294292 100644 --- a/foundation/foundation-configuration-store/src/test/resources/specifications/load-configurations.feature +++ b/foundation/foundation-configuration-store/src/test/resources/specifications/load-configurations.feature @@ -52,13 +52,13 @@ Feature: Load configurations at specified URI based on environment context Then an exception is thrown stating a policy attribute is undefined Examples: - | attribute | - | targets | - | targets-retrieve-url | - | rules | - | rules-classname | - | regeneration-method | - | regeneration-method-classname | + | attribute | + | targets | + | targets-retrieve-url | + | rules | + | rules-classname | + | regeneration-method | + | regeneration-method-classname | Scenario: A property has multiple policies Given URIs pointing to policies targeting the same property @@ -69,3 +69,16 @@ Feature: Load configurations at specified URI based on environment context Given there exists a krausening_password and encrypted properties When the configuration service starts Then the loaded properties contains the decrypted value + + Scenario: The ConfigLoader can read and write to a specific dao type + Given the config loader to configured to use "inMemory" + And the properties are loaded + When the configurations are written + Then the properties can be read + + @vault @integration + Scenario: The ConfigLoader can read and write to the Vault server + Given the config loader to configured to use "vault" + And the properties are loaded + When the configurations are written + Then the properties can be read diff --git a/foundation/foundation-mda/src/main/resources/templates/deployment/configuration-store/v2/configuration-store.values-dev.yaml.vm b/foundation/foundation-mda/src/main/resources/templates/deployment/configuration-store/v2/configuration-store.values-dev.yaml.vm index e2ebdfc7c..21ee2a2df 100644 --- a/foundation/foundation-mda/src/main/resources/templates/deployment/configuration-store/v2/configuration-store.values-dev.yaml.vm +++ b/foundation/foundation-mda/src/main/resources/templates/deployment/configuration-store/v2/configuration-store.values-dev.yaml.vm @@ -20,4 +20,3 @@ aissemble-configuration-store-chart: volumeMounts: - mountPath: /configurations name: configurations - readOnly: true