Disable scripted fields when dynamic scripting is disabled

[w33ble]

Assuming we can check the cluster state for dynamic scripting support, we should at the very least disable the scripted fields interface when it's disabled.

Ideally, we should scrub all requests of any scripted fields that may be defined, possibly previous to the dynamic scripting being disabled.

[martinhynar]

Using Kibana 4-b3 with disabled dynamic scripting in ES goes directly to huge exception after index is defined in settings. This is quite frustrating if you do not expect this. Also, some installations (like mine is) dynamic scripting is wanted to remain disabled.

If someone wants to just experiment with ES/Kibana, and reads "Download, Extract, Awesome" on download page, then this is not where one wants to end on its first journey.

The exception goes like this

Error: SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; ....

... lots of details ...

nested: ScriptException[dynamic scripting for [groovy] disabled]; }]
    at respond (http://localhost:5601/index.js?_b=4673:78854:15)
    at checkRespForFailure (http://localhost:5601/index.js?_b=4673:78822:7)
    at http://localhost:5601/index.js?_b=4673:77509:7
    at wrappedErrback (http://localhost:5601/index.js?_b=4673:20773:78)
    at wrappedErrback (http://localhost:5601/index.js?_b=4673:20773:78)
    at wrappedErrback (http://localhost:5601/index.js?_b=4673:20773:78)
    at http://localhost:5601/index.js?_b=4673:20906:76
    at Scope.$eval (http://localhost:5601/index.js?_b=4673:21893:28)
    at Scope.$digest (http://localhost:5601/index.js?_b=4673:21705:31)
    at Scope.$apply (http://localhost:5601/index.js?_b=4673:21997:24)

[peuh]

New to the ELK, stack it took me several hours to figure out the cause of this exception. It was that I had (as advised by many how to) disabled dynamic scripting.
I'm running kibana 4.0 b3 on elasticsearch 1.4.3.

If it can help here is the complete Exception :

Segmented Fetch: SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[NdVK6wEMQ5imlYGKVo-8Zw][logstash-2015.02.12][0]: SearchParseException[[logstash-2015.02.12][0]: query[ConstantScore(BooleanFilter(+cache(@timestamp:[1423740305000 TO 1423740306000])))],from[-1],size[500],sort[<custom:"@timestamp": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@33afc9b8>!]: Parse Failure [Failed to parse source [{"size":500,"highlight":{"pre_tags":["@kibana-highlighted-field@"],"post_tags":["@/kibana-highlighted-field@"],"fields":{"*":{}}},"query":{"filtered":{"query":{"query_string":{"query":"*"}},"filter":{"bool":{"must":[{"range":{"@timestamp":{"gte":1423740305000,"lte":1423740306000}}}],"must_not":[]}}}},"sort":{"@timestamp":"desc"},"aggs":{"2":{"date_histogram":{"field":"@timestamp","interval":"100ms","min_doc_count":0,"extended_bounds":{"min":1423740305000,"max":1423740306000}}}},"fields":["*","_source"],"script_fields":{"received_at":{"script":"if (doc[\"received_at\"].value == 0) { null } else { doc[\"received_at\"].value }"},"@timestamp":{"script":"if (doc[\"@timestamp\"].value == 0) { null } else { doc[\"@timestamp\"].value }"}}}]]]; nested: ScriptException[dynamic scripting for [groovy] disabled]; }{[NdVK6wEMQ5imlYGKVo-8Zw][logstash-2015.02.12][1]: SearchParseException[[logstash-2015.02.12][1]: query[ConstantScore(BooleanFilter(+cache(@timestamp:[1423740305000 TO 1423740306000])))],from[-1],size[500],sort[<custom:"@timestamp": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@62fd9d76>!]: Parse Failure [Failed to parse source [{"size":500,"highlight":{"pre_tags":["@kibana-highlighted-field@"],"post_tags":["@/kibana-highlighted-field@"],"fields":{"*":{}}},"query":{"filtered":{"query":{"query_string":{"query":"*"}},"filter":{"bool":{"must":[{"range":{"@timestamp":{"gte":1423740305000,"lte":1423740306000}}}],"must_not":[]}}}},"sort":{"@timestamp":"desc"},"aggs":{"2":{"date_histogram":{"field":"@timestamp","interval":"100ms","min_doc_count":0,"extended_bounds":{"min":1423740305000,"max":1423740306000}}}},"fields":["*","_source"],"script_fields":{"received_at":{"script":"if (doc[\"received_at\"].value == 0) { null } else { doc[\"received_at\"].value }"},"@timestamp":{"script":"if (doc[\"@timestamp\"].value == 0) { null } else { doc[\"@timestamp\"].value }"}}}]]]; nested: ScriptException[dynamic scripting for [groovy] disabled]; }{[NdVK6wEMQ5imlYGKVo-8Zw][logstash-2015.02.12][2]: SearchParseException[[logstash-2015.02.12][2]: query[ConstantScore(BooleanFilter(+cache(@timestamp:[1423740305000 TO 1423740306000])))],from[-1],size[500],sort[<custom:"@timestamp": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@786ce0bf>!]: Parse Failure [Failed to parse source [{"size":500,"highlight":{"pre_tags":["@kibana-highlighted-field@"],"post_tags":["@/kibana-highlighted-field@"],"fields":{"*":{}}},"query":{"filtered":{"query":{"query_string":{"query":"*"}},"filter":{"bool":{"must":[{"range":{"@timestamp":{"gte":1423740305000,"lte":1423740306000}}}],"must_not":[]}}}},"sort":{"@timestamp":"desc"},"aggs":{"2":{"date_histogram":{"field":"@timestamp","interval":"100ms","min_doc_count":0,"extended_bounds":{"min":1423740305000,"max":1423740306000}}}},"fields":["*","_source"],"script_fields":{"received_at":{"script":"if (doc[\"received_at\"].value == 0) { null } else { doc[\"received_at\"].value }"},"@timestamp":{"script":"if (doc[\"@timestamp\"].value == 0) { null } else { doc[\"@timestamp\"].value }"}}}]]]; nested: ScriptException[dynamic scripting for [groovy] disabled]; }{[NdVK6wEMQ5imlYGKVo-8Zw][logstash-2015.02.12][3]: SearchParseException[[logstash-2015.02.12][3]: query[ConstantScore(BooleanFilter(+cache(@timestamp:[1423740305000 TO 1423740306000])))],from[-1],size[500],sort[<custom:"@timestamp": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@3417595d>!]: Parse Failure [Failed to parse source [{"size":500,"highlight":{"pre_tags":["@kibana-highlighted-field@"],"post_tags":["@/kibana-highlighted-field@"],"fields":{"*":{}}},"query":{"filtered":{"query":{"query_string":{"query":"*"}},"filter":{"bool":{"must":[{"range":{"@timestamp":{"gte":1423740305000,"lte":1423740306000}}}],"must_not":[]}}}},"sort":{"@timestamp":"desc"},"aggs":{"2":{"date_histogram":{"field":"@timestamp","interval":"100ms","min_doc_count":0,"extended_bounds":{"min":1423740305000,"max":1423740306000}}}},"fields":["*","_source"],"script_fields":{"received_at":{"script":"if (doc[\"received_at\"].value == 0) { null } else { doc[\"received_at\"].value }"},"@timestamp":{"script":"if (doc[\"@timestamp\"].value == 0) { null } else { doc[\"@timestamp\"].value }"}}}]]]; nested: ScriptException[dynamic scripting for [groovy] disabled]; }{[NdVK6wEMQ5imlYGKVo-8Zw][logstash-2015.02.12][4]: SearchParseException[[logstash-2015.02.12][4]: query[ConstantScore(BooleanFilter(+cache(@timestamp:[1423740305000 TO 1423740306000])))],from[-1],size[500],sort[<custom:"@timestamp": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@f4a7f52>!]: Parse Failure [Failed to parse source [{"size":500,"highlight":{"pre_tags":["@kibana-highlighted-field@"],"post_tags":["@/kibana-highlighted-field@"],"fields":{"*":{}}},"query":{"filtered":{"query":{"query_string":{"query":"*"}},"filter":{"bool":{"must":[{"range":{"@timestamp":{"gte":1423740305000,"lte":1423740306000}}}],"must_not":[]}}}},"sort":{"@timestamp":"desc"},"aggs":{"2":{"date_histogram":{"field":"@timestamp","interval":"100ms","min_doc_count":0,"extended_bounds":{"min":1423740305000,"max":1423740306000}}}},"fields":["*","_source"],"script_fields":{"received_at":{"script":"if (doc[\"received_at\"].value == 0) { null } else { doc[\"received_at\"].value }"},"@timestamp":{"script":"if (doc[\"@timestamp\"].value == 0) { null } else { doc[\"@timestamp\"].value }"}}}]]]; nested: ScriptException[dynamic scripting for [groovy] disabled]; }]

[tbragin]

As part of #7700 we added a check to disable creation of new scripted fields in the interface if dynamic scripting is disabled: 9563f26

However, keeping this ticket open for this part:

Ideally, we should scrub all requests of any scripted fields that may be defined, possibly previous to the dynamic scripting being disabled.

[LeeDr]

It's been over 2 years since the last comment on this issue. We've switched to Painless scripting language. I suppose you could disable that in Elasticsearch? I'm going to close but if this is still an issue please reopen and elaborate.