Add logstash helm chart (elastic#7143)

* Add logstash helm chart with examples and documentation
* Update eck-stack chart to include logstash chart.
* Update eck-stack values to include logstash chart, disabled by default.
* Include logstash tests.
---------
Signed-off-by: Michael Montgomery <mmontg1@gmail.com>
Co-authored-by: Michael Montgomery <mmontg1@gmail.com>
Co-authored-by: Peter Brachwitz <peter.brachwitz@gmail.com>

deploy/eck-stack/Chart.yaml

@@ -21,3 +21,6 @@ dependencies:
   - name: eck-beats
     condition: eck-beats.enabled
     version: "0.8.0-SNAPSHOT"
+  - name: eck-logstash
+    condition: eck-logstash.enabled
+    version: "0.8.0-SNAPSHOT"

deploy/eck-stack/README.md

@@ -12,6 +12,7 @@ The following Elastic Stack resources are currently supported.
 - Elastic Agent
 - Fleet Server
 - Beats
+- Logstash
 
 Additional resources will be supported in future releases of this Helm Chart.
 
@@ -75,6 +76,7 @@ The following table lists the configurable parameters of the eck-stack chart and
 | `eck-kibana.enabled` | If `true`, create a Kibana resource (using the eck-kibana Chart) | `true` |
 | `eck-agent.enabled` | If `true`, create an Elastic Agent resource (using the eck-agent Chart) | `false` |
 | `eck-fleet-server.enabled` | If `true`, create a Fleet Server resource (using the eck-fleet-server Chart) | `false` |
+| `eck-logstash.enabled` | If `true`, create a Logstash resource (using the eck-logstash Chart) | `false` |
 
 Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`.
 

deploy/eck-stack/charts/eck-logstash/.helmignore

@@ -0,0 +1,24 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/
+templates/tests

deploy/eck-stack/charts/eck-logstash/Chart.yaml

@@ -0,0 +1,10 @@
+apiVersion: v2
+name: eck-logstash
+description: Logstash managed by the ECK operator
+kubeVersion: ">= 1.21.0-0"
+type: application
+version: 0.8.0-SNAPSHOT
+sources:
+  - https://github.com/elastic/cloud-on-k8s
+  - https://github.com/elastic/logstash
+icon: https://helm.elastic.co/icons/logstash.png

deploy/eck-stack/charts/eck-logstash/LICENSE

@@ -0,0 +1,93 @@
+Elastic License 2.0
+
+URL: https://www.elastic.co/licensing/elastic-license
+
+## Acceptance
+
+By using the software, you agree to all of the terms and conditions below.
+
+## Copyright License
+
+The licensor grants you a non-exclusive, royalty-free, worldwide,
+non-sublicensable, non-transferable license to use, copy, distribute, make
+available, and prepare derivative works of the software, in each case subject to
+the limitations and conditions below.
+
+## Limitations
+
+You may not provide the software to third parties as a hosted or managed
+service, where the service provides users with access to any substantial set of
+the features or functionality of the software.
+
+You may not move, change, disable, or circumvent the license key functionality
+in the software, and you may not remove or obscure any functionality in the
+software that is protected by the license key.
+
+You may not alter, remove, or obscure any licensing, copyright, or other notices
+of the licensor in the software. Any use of the licensor’s trademarks is subject
+to applicable law.
+
+## Patents
+
+The licensor grants you a license, under any patent claims the licensor can
+license, or becomes able to license, to make, have made, use, sell, offer for
+sale, import and have imported the software, in each case subject to the
+limitations and conditions in this license. This license does not cover any
+patent claims that you cause to be infringed by modifications or additions to
+the software. If you or your company make any written claim that the software
+infringes or contributes to infringement of any patent, your patent license for
+the software granted under these terms ends immediately. If your company makes
+such a claim, your patent license ends immediately for work on behalf of your
+company.
+
+## Notices
+
+You must ensure that anyone who gets a copy of any part of the software from you
+also gets a copy of these terms.
+
+If you modify the software, you must include in any modified copies of the
+software prominent notices stating that you have modified the software.
+
+## No Other Rights
+
+These terms do not imply any licenses other than those expressly granted in
+these terms.
+
+## Termination
+
+If you use the software in violation of these terms, such use is not licensed,
+and your licenses will automatically terminate. If the licensor provides you
+with a notice of your violation, and you cease all violation of this license no
+later than 30 days after you receive that notice, your licenses will be
+reinstated retroactively. However, if you violate these terms after such
+reinstatement, any additional violation of these terms will cause your licenses
+to terminate automatically and permanently.
+
+## No Liability
+
+*As far as the law allows, the software comes as is, without any warranty or
+condition, and the licensor will not be liable to you for any damages arising
+out of these terms or the use or nature of the software, under any kind of
+legal claim.*
+
+## Definitions
+
+The **licensor** is the entity offering these terms, and the **software** is the
+software the licensor makes available under these terms, including any portion
+of it.
+
+**you** refers to the individual or entity agreeing to these terms.
+
+**your company** is any legal entity, sole proprietorship, or other kind of
+organization that you work for, plus all organizations that have control over,
+are under the control of, or are under common control with that
+organization. **control** means ownership of substantially all the assets of an
+entity, or the power to direct its management and policies by vote, contract, or
+otherwise. Control can be direct or indirect.
+
+**your licenses** are all the licenses granted to you for the software under
+these terms.
+
+**use** means anything you do with the software requiring one of your licenses.
+
+**trademark** means trademarks, service marks, and similar rights.

deploy/eck-stack/charts/eck-logstash/examples/basic-eck.yaml

@@ -0,0 +1,44 @@
+---
+# values corresponding to config/recipes/logstash/logstash-eck.yaml
+version: 8.11.0-SNAPSHOT
+
+elasticsearchRefs:
+  - clusterName: eck
+    name: elasticsearch
+
+pipelines:
+  - pipeline.id: main
+    config.string: |
+      input {
+        beats {
+          port => 5044
+        }
+      }
+      filter {
+        grok {
+          match => { "message" => "%{HTTPD_COMMONLOG}"}
+        }
+        geoip {
+          source => "[source][address]"
+          target => "[source]"
+        }
+      }
+      output {
+        elasticsearch {
+          hosts => [ "${ECK_ES_HOSTS}" ]
+          user => "${ECK_ES_USER}"
+          password => "${ECK_ES_PASSWORD}"
+          ssl_certificate_authorities => "${ECK_ES_SSL_CERTIFICATE_AUTHORITY}"
+        }
+      }
+
+services:
+  - name: beats
+    service:
+      spec:
+        type: ClusterIP
+        ports:
+          - port: 5044
+            name: "filebeat"
+            protocol: TCP
+            targetPort: 5044

deploy/eck-stack/charts/eck-logstash/examples/es-role.yaml

@@ -0,0 +1,25 @@
+---
+# values corresponding to config/recipes/logstash/logstash-es-role.yaml
+version: 8.11.0-SNAPSHOT
+
+elasticsearchRefs:
+  - clusterName: eck
+    name: elasticsearch
+
+pipelines:
+  - pipeline.id: main
+    config.string: |
+      input { exec { command => "uptime" interval => 10 } } 
+      output { 
+        elasticsearch {
+          hosts => [ "${ECK_ES_HOSTS}" ]
+          ssl_enabled => true
+          ssl_certificate_authorities => "${ECK_ES_SSL_CERTIFICATE_AUTHORITY}"
+          user => "${ECK_ES_USER}"
+          password => "${ECK_ES_PASSWORD}"
+          index => "my-index"
+          data_stream => false
+          ilm_enabled => false
+          manage_template => false
+        } 
+      }

deploy/eck-stack/charts/eck-logstash/examples/monitored.yaml

@@ -0,0 +1,49 @@
+---
+# values corresponding to config/recipes/logstash/logstash-monitored.yaml
+version: 8.11.0-SNAPSHOT
+
+monitoring:
+  metrics:
+    elasticsearchRefs:
+      - name: elasticsearch-monitoring
+
+pipelines:
+  - pipeline.id: main
+    config.string: |
+      input {
+        beats {
+          port => 5044
+        }
+      }
+      filter {
+        grok {
+          match => { "message" => "%{HTTPD_COMMONLOG}"}
+        }
+        geoip {
+          source => "[source][address]"
+          target => "[source]"
+        }
+      }
+      output {
+        elasticsearch {
+          hosts => [ "${ECK_ES_HOSTS}" ]
+          user => "${ECK_ES_USER}"
+          password => "${ECK_ES_PASSWORD}"
+          ssl_certificate_authorities => "${ECK_ES_SSL_CERTIFICATE_AUTHORITY}"
+        }
+      }
+
+elasticsearchRefs:
+  - clusterName: eck
+    name: elasticsearch
+
+services:
+  - name: beats
+    service:
+      spec:
+        type: ClusterIP
+        ports:
+          - port: 5044
+            name: "filebeat"
+            protocol: TCP
+            targetPort: 5044

deploy/eck-stack/charts/eck-logstash/examples/multi.yaml

@@ -0,0 +1,78 @@
+---
+# values corresponding to config/recipes/logstash/logstash-multi.yaml
+version: 8.11.0-SNAPSHOT
+
+pipelines:
+  - pipeline.id: main
+    config.string: |
+      input {
+        beats {
+          port => 5044
+        }
+      }
+      filter {
+        grok {
+          match => { "message" => "%{HTTPD_COMMONLOG}"}
+        }
+        geoip {
+          source => "[source][address]"
+          target => "[source]"
+        }
+      }
+      output {
+        pipeline {
+          send_to => 'prod'
+        }
+        pipeline {
+          send_to => 'qa'
+        }
+      }
+  - pipeline.id: production
+    config.string: |
+      input {
+        pipeline {
+          address => 'prod'
+        }
+      }
+      output {
+        elasticsearch {
+          hosts => [ "${PROD_ES_ES_HOSTS}" ]
+          user => "${PROD_ES_ES_USER}"
+          password => "${PROD_ES_ES_PASSWORD}"
+          ssl_certificate_authorities => "${PROD_ES_ES_SSL_CERTIFICATE_AUTHORITY}"
+        }
+      }
+  - pipeline.id: qa
+    config.string: |
+      input {
+        pipeline {
+          address => 'qa'
+        }
+      }
+      output {
+        elasticsearch {
+          hosts => [ "${QA_ES_ES_HOSTS}" ]
+          user => "${QA_ES_ES_USER}"
+          password => "${QA_ES_ES_PASSWORD}"
+          ssl_certificate_authorities => "${QA_ES_ES_SSL_CERTIFICATE_AUTHORITY}"
+        }
+      }
+
+elasticsearchRefs:
+  - clusterName: prod-es
+    name: production
+  - clusterName: qa-es
+    name: qa
+    namespace: qa
+
+services:
+  - name: beats
+    service:
+      spec:
+        type: ClusterIP
+        ports:
+          - port: 5044
+            name: "filebeat"
+            protocol: TCP
+            targetPort: 5044
+