Default node allocation awareness

[anyasabo]

We should consider adding allocation awareness for the node the Elasticsearch pod is on by default. We have some documentation here, and we set the default antiaffinity on the pods to prefer to not be scheduled on the same node. But pods still might be scheduled on the same pod, and this would give Elasticsearch the information it needs to try and spread shards out across nodes in that situation.

We can set it to just try and spread out shards (i.e. not force), and users can override it, so I do not see a downside to adding this by default.