Dealing with proposed http.enabled changes on ES's side

[ppf2]

This is to track the outcome of an active ES ticket intended to remove http.enabled from the product.

This will directly affect how we do sniffing today in the LS ES output.

We already have users today using a combination of sniffing and http.enabled: false (as recommended in the LS documentation) on the ES side to influence the ES output so that it doesn't send the events to the master nodes or other specific nodes of interest (eg. warm nodes).

If ES removes the ability to disable http, it will be breaking for LS ES output's sniffing feature.

[jordansissel]

it will be breaking for LS ES output's sniffing feature

Sounds like we need to update the documentation to indicate that the http.enabled recommendation only works with Elasticsearch versions 5.x and older, not 6.0 or beyond.

[jasontedor]

Disabling HTTP on master-eligible nodes is not something I would recommend to begin with, sometimes the cluster is misbehaving and you need to send requests directly to a master. If the master is not enabled for HTTP then it can not receive said requests, this is undesirable. I think instead the sniffing should be enhanced to exclude master nodes if needed.

An option for now is to bind HTTP on master-eligible nodes to localhost only, but this has problems too:

• without looking at the code, I would guess this means the output would turn around and try to send a request its localhost (because it will see localhost addresses in the sniffing)
• it doesn't prevent requests from hitting a master-eligible if it's running on the same box (not a configuration I would recommend)
• it means a user has to SSH to a master-eligible node to send requests directly to a master

[jordansissel]

@jasontedor I agree with your concerns about disabling http.

I tried to find some history. Sniffing was added to our http client in 6e95c72 (squashed from #216, 10595f8) added the following text to the existing sniffing feature:

Will return ALL nodes with HTTP enabled (including master nodes!). If you use this with master nodes, you probably want to disable HTTP on them by setting http.enabled to false in their elasticsearch.yml.

Relevant tickets: #240, #216, #150.

I was not able to find history behind the decision to recommend http.enabled: false on masters.

---

I am +1 on removing the http.enabled: false recommendation from our docs, and we can also start work on improving sniffing for users who want such feature.

[ppf2]

and we can also start work on improving sniffing for users who want such feature.

+1 , the ES clients eg. TransportClient, Python, etc.. already filter out master nodes by default when sniffing is used, so LS can do the same without relying on the http.enabled setting.

[tomsommer]

+1 #303

[andrewvc]

@tomsommer I agree. I've created elastic/elasticsearch#24871 to see if we can standardize that across all clients (along with some other stuff)

[andrewvc]

Fixed in #694