US-389994 Initial helm charts SRS backing service

.github/CODEOWNERS

@@ -8,3 +8,12 @@
 # Documentation
 *.md        @taz-pega-work
 docs/*      @taz-pega-work
+
+# Addons Chart
+charts/pega/addons/*  @pegasystems/deployment
+
+# Pega Chart
+charts/pega/pega/*    @pegasystems/deployment
+
+# SRS Chart
+charts/backingservices/charts/srs/*  @pegasystems/searchservice

.travis.yml

@@ -36,6 +36,7 @@ script:
   #  ; test ${PIPESTATUS[0]} -eq 0 - check if go test was finished success. Without it pipe return exit code from last command.
   - go test test/pega | grep -v command.go:158 ; test ${PIPESTATUS[0]} -eq 0
   - go test test/addons | grep -v command.go:158 ; test ${PIPESTATUS[0]} -eq 0
+  - go test test/backingservices | grep -v command.go:158 ; test ${PIPESTATUS[0]} -eq 0
   - cd $TRAVIS_BUILD_DIR
   - chmod 777 before_deploy.sh
 before_deploy:

Makefile

@@ -6,7 +6,8 @@ dependencies:
 	helm repo add elastic https://helm.elastic.co
 	helm repo list	
 	helm dependency update ./charts/pega/
-	helm dependency update ./charts/addons/ 
+	helm dependency update ./charts/addons/
+	helm dependency update ./charts/backingservices/
 
 examples: 
 	mkdir -p ./build/kubernetes

README.md

@@ -37,49 +37,70 @@ $ helm repo add pega https://dl.bintray.com/pegasystems/pega-helm-charts
 
 ```bash
 $ helm search repo pega
-NAME       	CHART VERSION	APP VERSION	DESCRIPTION
-pega/pega  	1.2.0        	           	Pega installation on kubernetes
-pega/addons	1.2.0        	1.0        	A Helm chart for Kubernetes 
+NAME       	            CHART VERSION	APP VERSION	DESCRIPTION
+pega/pega  	              1.2.0        	           	Pega installation on kubernetes
+pega/addons	              1.2.0        	1.0        	A Helm chart for Kubernetes 
+pega/backingservices          1.2.0        	                Helm Chart to provisioning backing services for Pega (eg. Search and Reporting Service)
 ```
 
-There are two charts available in this repository - addons and pega.
+There are three charts available in this repository - addons, backingservices, and pega.
 
 The addons chart installs a collection of supporting services and tools required for a Pega deployment. The services you will need to deploy will depend on your cloud environment - for example you may need a load balancer on Minikube, but not for EKS. These supporting services are deployed once per Kubernetes environment, regardless of how many Pega Infinity instances are deployed.
 
-3. Download the values file for pega/pega and pega/addons.
+The backingservices chart installs services like 'Search and Reporting Service' (SRS) that you can configure with one or more Pega deployments. You can deploy these backing services in their own namespace; you can isolate the services to a single environment or share them across multiple Pega Infinity environments.
+
+**Example:**
+_Single backing service shared across all pega environments:_
+
+backingservice 'Search and Reporting Service' deployed and the service endpoint configured across dev, staging and production pega environments. The service provides isolation of data in a shared setup.
+
+_Multiple backing service deployments:_
+
+You can deploy more than one instance of backing service deployments, in case you want to host a separate deployment of 'Search and Reporting Service' for non-production and production deployments of Pega Infinity. You must configure the appropriate service endpoint using the Pega Infinity deployment values.
+
+3. Download the values file for pega/pega, pega/addons and pega/backingservices.
 
 ```bash
 $ helm inspect values pega/pega > pega.yaml
 $ helm inspect values pega/addons > addons.yaml
+$ helm inspect values pega/backingservices > backingservices.yaml
 ```
 
 4. Edit your values yaml files to specify all required information and customizations for your environment.
 
 * [Instructions to configure the Pega chart](charts/pega/README.md)
 * [Instructions to configure the Pega addons](charts/addons/README.md)
+* [Instructions to configure the Pega backingservices](charts/backingservices/README.md)
 
-5. Create namespaces for your Pega deployment and the addons (if applicable for your environment).
+5. Create namespaces for your Pega deployment, backingservices and the addons (if applicable for your environment).
 
 ```bash
 $ kubectl create namespace mypega
 $ kubectl create namespace pegaaddons
+$ kubectl create namespace pegabackingservices
 ```
 
-6. To install the addons chart, run the following helm command after configuring your values.yaml file (if applicable for your environment). 
+6. To install the backingservices chart, run the following helm command after configuring your values.yaml file (if applicable for your environment). 
+
+```bash
+$ helm install backingservices pega/backingservices --namespace pegabackingservices --values backingservices.yaml
+```
+
+7. To install the addons chart, run the following helm command after configuring your values.yaml file (if applicable for your environment). 
 
 ```bash
 $ helm install addons pega/addons --namespace pegaaddons --values addons.yaml
 ```
 
-7. Now you can deploy Pega using the Helm chart. Before installing using the chart, it is a good idea to review the detailed [deployment guide](https://community.pega.com/knowledgebase/articles/deploying-pega-platform-using-kubernetes) to understand how Pega deploys as a distributed system. Running a Helm installation using the pega chart installs a Pega Infinity instance into a specified namespace.  
+8. With addons and backservices deployed, you are ready to deploy Pega Infinity using the pega chart. Before installing using the chart, it is a good idea to review the detailed [deployment guide](https://community.pega.com/knowledgebase/articles/deploying-pega-platform-using-kubernetes) to understand how Pega deploys as a distributed system. Running a Helm installation using the pega chart installs a Pega Infinity instance into a specified namespace. After you edit the chart with your configuration requirements, run the following command to install the pega chart. 
 
 ```bash
 $ helm install mypega pega/pega --namespace mypega --values pega.yaml
 ```
 
-*If you want to edit the charts and build using your local copy, replace pega/addons or pega/pega with the path to your chart directory.*
+*To edit the charts and deploy using a local version of the pega/addons, pega/backingservices, or pega/pega charts, you must include the filepath to your local chart directory in your Helm chart reference.*
 
-8. If you wish to delete your deployment of Pega nodes, enter the following command (this will not delete your database):
+9. To delete your deployment of Pega nodes, enter the command (this will not delete your database):
 
 ```bash
 $ helm delete mypega
@@ -104,16 +125,19 @@ For details, see the examples listed in the preparation runbook:
 
 Pegasystems uses a standard naming practice of hostname/product/image:tag. Pega images are available from the host site, pega-docker.downloads.pega.com. Pega maintains three types of required Docker images for Client-managed Cloud deployments of Pega Platform:
 
- Name        | Description                                           | Tags     |
--------------|-------------------------------------------------------|----------|
-`platform/installer`   | A utility image with which you install all of the Pega-specific rules and database tables in the “Pega” database that you have configured for your deployment. This installation is required before a deployment can take place.| `<version>` |
-`platform/pega`  | (Download required) Deploys Pega Platform with its customized version of the Tomcat application server.| `<version>` or `<version>-YYYYMMDD` |
-`platform/search` | (Download required) Deploys the required search engine for Pega Platform search and reporting capabilities. This Docker image contains Elasticsearch and includes all required plugins.| `<version>` or `<version>-YYYYMMDD` |
+ Name                                           | Description                                           | Tags     |
+------------------------------------------------|-------------------------------------------------------|----------|
+`platform/installer`                            | A utility image with which you install all of the Pega-specific rules and database tables in the “Pega” database that you have configured for your deployment. This installation is required before a deployment can take place.| `<version>` |
+`platform/pega`                                 | (Download required) Deploys Pega Platform with its customized version of the Tomcat application server.| `<version>` or `<version>-YYYYMMDD` |
+`platform/search`                               | (Download required) Deploys the required search engine for Pega Platform search and reporting capabilities. This Docker image contains Elasticsearch and includes all required plugins.| `<version>` or `<version>-YYYYMMDD` |
+`platform-services/search-n-reporting-service`  | (Download required) Deploys the required search engine for the latest generation of search and reporting capabilities in Pega Infinity. This Docker image contains Pega Search and Reporting Service.| `<version>` or `<version>-YYYYMMDD` |
 
 For the `platform/installer` image, the :tag represents the version of Pega you want to install, for example the tag :8.5.1 will install Pega Platform version 8.5.1.
 
 For `platform/pega` and `platform/search` images, Pega also offers an image with a version tag appended with a datestamp using the pattern `pegaVersion-YYYYMMDD` to indicate the version and the date that Pega built the image. For example, if you pull the `platform/pega` with a tag, `pega:8.5.1-20201026`, the tag indicates that Pega built this 8.5.1 image on 26 October 2020. Using the version tag without the datestamp will always point to the most recently built image for that version.
 
+The docker image `platform-services/search-n-reporting-service` is part of the chart `backingservices\addons`. The `srs` subchart deployment provisions the latest generation of search and reporting capabilities in Pega Infinity; this service is an alternate search feature to the previous `platform/search` image-based search deployment.  [Instructions to configure the Pega backingservices](charts/backingservices/README.md). 
+
 The datestamp ensures that the image you download includes the changes that Pega engineering commits to the repository using pull requests by a certain date. While Pega builds the most current patch version of each minor release one time each day, Pega makes the last five daily-built images available for client downloads.  After Pega releases a new patch version, the prior patch version no longer receives daily builds with a datestamp tag.
 
 After you obtain access to the Pega-provided host repository and pull each image, you can re-tag and push each of the three Pega-provided images to your preferred Docker registry to make them available to the deployment. For an overview of tagging and managing Docker images, see the Docker article, [Deploy a registry server](https://docs.docker.com/registry/deploying/).

before_deploy.sh

@@ -5,8 +5,10 @@
 export CHART_VERSION=$(expr ${TRAVIS_TAG:1})
 export PEGA_FILE_NAME=pega-${CHART_VERSION}.tgz
 export ADDONS_FILE_NAME=addons-${CHART_VERSION}.tgz
-cat descriptor-template.json | jq '.files[0].includePattern=env.PEGA_FILE_NAME' | jq '.files[0].uploadPattern=env.PEGA_FILE_NAME' | jq '.files[1].includePattern=env.ADDONS_FILE_NAME' | jq '.files[1].uploadPattern=env.ADDONS_FILE_NAME' > descriptor.json
+export BACKINGSERVICES_FILE_NAME=backingservices-${CHART_VERSION}.tgz
+cat descriptor-template.json | jq '.files[0].includePattern=env.PEGA_FILE_NAME' | jq '.files[0].uploadPattern=env.PEGA_FILE_NAME' | jq '.files[1].includePattern=env.ADDONS_FILE_NAME' | jq '.files[1].uploadPattern=env.ADDONS_FILE_NAME' | jq '.files[1].includePattern=env.BACKINGSERVICES_FILE_NAME' | jq '.files[1].uploadPattern=env.BACKINGSERVICES_FILE_NAME' > descriptor.json
 curl -o index.yaml https://dl.bintray.com/pegasystems/pega-helm-charts/index.yaml
 helm package --version ${CHART_VERSION} ./charts/pega/
 helm package --version ${CHART_VERSION} ./charts/addons/
+helm package --version ${CHART_VERSION} ./charts/backingservices/
 helm repo index --merge index.yaml --url https://dl.bintray.com/pegasystems/pega-helm-charts/ .

charts/backingservices/.helmignore

@@ -0,0 +1,23 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/

charts/backingservices/Chart.yaml

@@ -0,0 +1,19 @@
+---
+apiVersion: v1
+name: backingservices
+description: A Helm chart for Kubernetes
+
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# The chart version: Pega provides this as a useful way to track changes you make to this chart.
+# As a best practice, you should increment the version number each time you make changes to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 1.2.0

charts/backingservices/README.md

@@ -0,0 +1,81 @@
+# BackingServices Helm chart
+
+The Pega Infinity backing service is a feature which you can deploy as an independent service module. For example `Search and Reporting Service` or `SRS` backing service can replace the embedded search feature of Pega Infinity Platform. To use it in your deployment, you provision and deploy it independently as an external service which provides search and reporting capabilities with one or more Pega Infinity environments.  
+
+The backingservices chart supports deployment options for `Search and Reporting Service` (abbreviated as SRS). A backing service may be configured with one or more Pega deployments. 
+These backing services should be deployed in their own namespace and may be shared across multiple Pega Infinity environments.
+
+**Example:**
+
+**_Single backing service shared across all pega environments:_**
+You can provision the backingservice `Search and Reporting Service` in its own namespace, with the service endpoint configured across dev, staging and production Pega Infinity environments. This service configuration provides isolation of data in a shared setup.
+
+**_Multiple backing service deployments:_**
+You can deploy more than one instance of backing service deployments, in case you want to host a separate deployment of 'Search and Reporting Service' for non-production and production deployments of Pega Infinity. You must configure each service endpoint appropriately for each Pega Infinity deployment.
+
+### Search and Reporting Service
+
+The Search and Reporting Service provides next generation search and reporting capabilities for Pega Infinity 8.6 and later. 
+
+This service replaces the legacy search module from the platform with an independently deployable and scalable service along with the built-in capabilities to support more than one Pega environments with its data isolation features. 
+The service deployment provisions runtime service pods along with a dependency on a backing technology ElasticSearch service for storage and retrieval of data. 
+
+#### SRS Version compatibility matrix
+Pega Infinity version   | SRS version   | ElasticSearch version     | Description
+---                     | ---           | ---                       | ---
+< 8.6                   | NA            | NA                        | SRS service can be used with Pega Infinity 8.6 and above
+\>= 8.6                 | \> 1.6.0      | 7.1.x                     | Pega Infinity 8.6 and above version may use SRS Image tag version 1.6.0 and above. Current SRS versions are certified to work with Elasticsearch version 7.1.x.
+
+
+#### SRS service runtime configuration:
+
+The values.yaml provides configuration options to define the deployment resources along with option to either provision ElasticSearch cluster automatically for data storage, or you can choose to configure an existing managed elasticsearch cluster to use as a datastore with the SRS service runtime. 
+
+If an externally managed elasticsearch cluster is being used, make sure the service is accessible to the k8s cluster where SRS is deployed.
+
+You may enable the component of ([Elasticsearch](https://github.com/helm/charts/tree/master/stable/elasticsearch/values.yaml) in the backingservices by configuring the 'srs.srsStorage' section in values.yaml file to deploy ElasticSearch cluster automatically. For more configuration options available for each of the components, see their Helm Charts.
+
+#### configuration settings:
+Configuration                       | Usage
+---                                 | ---
+`enabled`                           | Enable the Search and Reporting Service deployment as a backing service.
+`deploymentName`                    | The name of your SRS cluster.  Resources created will be prefixed with this string. This is also the service name for SRS.
+`srsRuntime`                        | This section defines the SRS service specific resource configuration options like image, replica count, cpu and memory resource settings etc.
+`elasticsearch`                     | Define the elasticsearch cluster configurations using this section. The chart from ([Elasticsearch](https://github.com/helm/charts/tree/master/stable/elasticsearch/values.yaml) is used for provisioning the cluster.
+`srsStorage.provisionInternalESCluster` | This setting when enabled will provision Elasticsearch cluster automatically with SRS runtime. Disable this to use an existing external ElasticSearch cluster with the SRS runtime.
+
+Example:
+
+```yaml
+srs:
+  enabled: true
+  deploymentName: "YOUR_SRS_DEPLOYMENT_NAME"
+  srsRuntime:
+    #srs-service values
+    replicaCount: 2
+    srsImage: platform-services/search-n-reporting-service:1.6.1
+    imagepullPolicy: IfNotPresent
+    resources:
+        limits:
+            cpu: 1300m
+            memory: "2Gi"
+        requests:
+            cpu: 650m
+            memory: "2Gi"
+    serviceType: "ClusterIP"
+    env:
+        #AuthEnabled may be set to true when there is an authentication mechanism in place between SRS and Pega Infinity.
+        AuthEnabled: false
+        PublicKeyURL:
+
+  srsStorage:
+    # srsStorage.provisionInternalESCluster true will provision an internal elasticsearch cluster with specified configuration
+    provisionInternalESCluster: true
+    # set the external Elasticsearch cluster URL and port details below when using an externally managed elasticsearch
+    # make sure the endpoint is accessible from the kubernetes cluster pods.
+    # Currently the elasticsearch connection does not support any modes of authentication and should be es endpoint APIs' accessible without authentication.
+#    domain: managed-elasticsearch.acme.io
+#    port: 443
+#    protocol: https
+#    set `requireInternetAccess` to true when the elasticsearch domain is outside of the Kubernetes cluster network and is available over internet
+#    requireInternetAccess: true

charts/backingservices/charts/srs/.helmignore

@@ -0,0 +1,22 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/