Improve setup of ClusterRole and ClusterRoleBinding for autodiscover

[david-kow]

Autodiscover is a feature available in a number of Beats that enables driving configuration from Pod configuration (via annotations) as new Pods are appearing instead of driving it from Beats configuration directly. We want to support autodiscover out-of-the-box in Beat CRD.

To perform a discovery, Beat pod needs to have clusterwide permissions to get, list and watch pods and namespaces.

Similarly to #3133, we can leave it up to the user to configure, do it as a part of the installation or do it in the controller.

So far we agreed to do it in the controller (behind a flag) but it does have downsides:

• it requires the operator to have additional permissions (to create clusterroles/clusterrolebindings)
• delete story is not great as all our CRs are namespaced and cluster-wide resource can't have namespaced owner. This means that if someone creates and deletes Beat CR they will be left with these autodiscover role and binding. If someone applies and deletes all-in-one.yaml and create Beat CR in between it will be also left behind. To do it properly we would need to make sure that no other Beat is using it, ie. list all Beats and check if they are using the default service account.

I think I'd be for putting it in the manifest generation code. But if we decide otherwise, it seems to make sense to align this decision with the decision for #3133.

[david-kow]

To address the concerns above, the following is currently implemented as a part of #3041:

• ClusterRole with correct autodiscover permissions is created as a part of the ECK installation
• The operator gets a permission to create bindings to only that ClusterRole
• ClusterRoleBinding and ServiceAccount is created for each Beat that doesn't have ServiceAccount set by the user
• The operator filters the events for ClusterRoleBindings that don't concern it
• The operator removes ClusterRoleBinding when Beat resource is deleted based on well-known name template
• Default ServiceAccount has a well-known name template that allows users to bind their own roles to it, independently from the operator

[david-kow]

This is implemented with #3041. Docs issue will come separately.