Disable fielddata for all analyzed string fields in LS template

[ppf2]

Have come across this multiple times in the past with various users in the field. ES does not support doc values for analyzed string fields today (even in ES 2.0). Kibana warns about, but does not adequately prevent users from hurting themselves by running aggs/sorting/scripting against an analyzed string field (elastic/kibana#5189).

We recently added doc_values: true to the latest LS template in master which is a great addition (esp. for users on 1.x). I will suggest taking this further by disabling fielddata for all analyzed string fields by default (adding the following to all analyzed string fields):

"fielddata": {
                "format": "disabled"
              }

For those who need fielddata for analyzed string and don't care about issues that stem from loading analyzed string field values into the heap, they can change the template themselves.

From what I have seen, 90%+ of the use cases out there do not require agg/sorting/scripting against analyzed string fields and these are operations are done against not_analyzed string fields.

[PhaedrusTheGreek]

Although this is a closed issue, it might be worth pointing out that the Kibana Discover page has sort widgets that don't work by default because of this change. You get Field Data Loading is Forbidden on [FieldName].

While this protects the average use case, this error can be confusing (at first) when received in Kibana.

[ppf2]

The idea of disabling fd in the LS template is to prevent it from causing stability issues in Elasticsearch. The Kibana UI will have to handle this in a more intuitive way for users who do not know what fielddata is. Some of this will be improved in 5.0: elastic/kibana#4230 -> elastic/kibana#3335 .

[andrewvc]

I'm tentatively OK with this. I'm a bit worried that it will just confuse people.

I mean, field data is harmless if not used..... I'm still not quite sure. @acchen97 @jordansissel @jsvd @untergeek what do you guys think?

[andrewvc]

I think ideally the Kibana UI would better warn people about sorting on field data.

[untergeek]

Disabling field data by default may be a good thing in some cases, but less so in others. For the time being, I prefer to leave our template as a good default, for the out of the box experience. We make it pretty plain that it's only a starting point.

[untergeek]

And by that I mean, we shouldn't try to solve every problem for every user just getting started.

[jsvd]

@ppf2 is this still a common need you see in our users?

[ppf2]

@jsvd This one is obsolete now since 5.0+'s text datatype in Elasticsearch defaults to fielddata: false by default. So I am closing :) thx!