[stable/filebeat] Allow volumes, env vars, and http metrics to be configurable

[rifelpet]

What this PR does / why we need it:
This PR allows additional volumes to be mounted along with additional prospectors configured.

This way we can watch logs from the host's /var/log that aren't from container logs themselves.

Additional environment variables can be specified as a key-value map in order to support interpolation in the prospector configurations.

Example values.yaml for capturing kube-proxy logs on a Kops cluster:

volumes:
  - hostPath:
      path: /var/log
    name: varlog
volumeMounts:
  - name: varlog
    mountPath: /host/var/log
    readOnly: true

extraEnv:
  CLUSTER_NAME: set-me-at-runtime

config:
  kube-proxy.yml: |-
    - type: log
      paths:
        - /host/var/log/kube-proxy.log
      tags: ["proxy"]
      processors:
        - add_kubernetes_metadata:
            in_cluster: true
            namespace: kube-system
        - add_cloud_metadata:
      fields:
        kubernetes.cluster: ${CLUSTER_NAME}
      fields_under_root: true

I also cleaned up some trailing whitespace and added checksums of the Secret and ConfigMap to the Daemonset annotations so pods will be rolled whenever either the Secret or ConfigMap change.

Which issue this PR fixes: N/A

Special notes for your reviewer:

Default behavior should be unchanged, but pods will be rolled upon upgrading due to the annotation change.

[rifelpet]

/ok-to-test

[rifelpet]

@sstarcher I've rebased and added some additional functionality - NODE_NAME env var for adding as a field to host logs not originating from docker containers (syslog in my case), as well as exposing the http metrics endpoint for monitoring. I also added a home to Chart.yaml to pass the stricter linting test that was added today.

[sstarcher]

Is http.enabled a filebeat configuration? If not I would separate it out into it's own thing. Also the dot in the string may cause issues with people trying to set it on the command line as I believe it will be treated as a map.

[sstarcher]

Just looked at the docs, I see this is a filebeat configuration. LGTM

[rifelpet]

Agreed, having a period in the key makes it a bit tricky to use with --set and the usage of the key in the template is not the most intuitive. We could try to add a separate, more helm-friendly key value pair that enables the containerPort but I'm not sure how we could conditionally merge a yaml block into the filebeat.yml Secret as its setup now. Any ideas?

[sstarcher]

I would be fine with keeping it as it is, but I would add a comment around it saying that it can't be set on the command line.

[unguiculus]

In an effort to standardize things across charts, we tend to go with extraContainers, extraVars, extraSecrets, extraVolumes , etc. Would you mind following this pattern?

[unguiculus]

/assign

[rifelpet]

@unguiculus done 👍

[unguiculus]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rifelpet, unguiculus

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [unguiculus]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment