Field [stats.value] used in expression does not exist in mappings -- but the query does not contain that

Elastic Stack Kibana
1

I get this error for any use of kibana:

Field [stats.value] used in expression does not exist in mappings

But my kibana query is simply '*'. What does this mean? What expression is being referred to?

2

Check your ES logs, there should be something with a bit more info.

3

Lots of the below errors, but I still don't understand where the stats.value comes from. The log message seems to indicate that stats.value may be scripted, but we don't use any scripting.

{
  "exception": {
    "stacktrace": "RemoteTransportException[[ops-elk-2][10.0.197.130:9300][indices:data\/read\/search[phase\/query]]]; nested: SearchParseException[failed to parse search source [{\"size\":500,\"sort\":[{\"@timestamp\":{\"order\":\"desc\",\"unmapped_type\":\"boolean\"}}],\"highlight\":{\"pre_tags\":[\"@kibana-highlighted-field@\"],\"post_tags\":[\"@\/kibana-highlighted-field@\"],\"fields\":{\"*\":{}},\"require_field_match\":false,\"fragment_size\":2147483647},\"query\":{\"filtered\":{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":{\"bool\":{\"must\":[{\"range\":{\"@timestamp\":{\"gte\":1467700583897,\"lte\":1467786983897,\"format\":\"epoch_millis\"}}}],\"must_not\":[]}}}},\"aggs\":{\"2\":{\"date_histogram\":{\"field\":\"@timestamp\",\"interval\":\"30m\",\"time_zone\":\"Asia\/Kolkata\",\"min_doc_count\":0,\"extended_bounds\":{\"min\":1467700583897,\"max\":1467786983897}}}},\"fields\":[\"*\",\"_source\"],\"script_fields\":{\"ResponseBYRequest\":{\"script\":\"doc['stats.value'].value\/doc['stats.requestCount'].value\",\"lang\":\"expression\"}},\"fielddata_fields\":[\"@timestamp\",\"time\",\"state.lastTimestamp\",\"updatedState.lastTimestamp\",\"newState.last_timestamp\",\"probe.timestamp\",\"probe.received_at\",\"probe.creation_time\"]}]]; nested: ScriptException[Error during search with inline script [doc['stats.value'].value\/doc['stats.requestCount'].value] using lang [expression]]; nested: ScriptException[Field [stats.value] used in expression does not exist in mappings];\nCaused by: SearchParseException[failed to parse search source [{\"size\":500,\"sort\":[{\"@timestamp\":{\"order\":\"desc\",\"unmapped_type\":\"boolean\"}}],\"highlight\":{\"pre_tags\":[\"@kibana-highlighted-field@\"],\"post_tags\":[\"@\/kibana-highlighted-field@\"],\"fields\":{\"*\":{}},\"require_field_match\":false,\"fragment_size\":2147483647},\"query\":{\"filtered\":{\"query\":{\"query_string\":{\"query\":\"*\",\"analyze_wildcard\":true}},\"filter\":{\"bool\":{\"must\":[{\"range\":{\"@timestamp\":{\"gte\":1467700583897,\"lte\":1467786983897,\"format\":\"epoch_millis\"}}}],\"must_not\":[]}}}},\"aggs\":{\"2\":{\"date_histogram\":{\"field\":\"@timestamp\",\"interval\":\"30m\",\"time_zone\":\"Asia\/Kolkata\",\"min_doc_count\":0,\"extended_bounds\":{\"min\":1467700583897,\"max\":1467786983897}}}},\"fields\":[\"*\",\"_source\"],\"script_fields\":{\"ResponseBYRequest\":{\"script\":\"doc['stats.value'].value\/doc['stats.requestCount'].value\",\"lang\":\"expression\"}},\"fielddata_fields\":[\"@timestamp\",\"time\",\"state.lastTimestamp\",\"updatedState.lastTimestamp\",\"newState.last_timestamp\",\"probe.timestamp\",\"probe.received_at\",\"probe.creation_time\"]}]]; nested: ScriptException[Error during search with inline script [doc['stats.value'].value\/doc['stats.requestCount'].value] using lang [expression]]; nested: ScriptException[Field [stats.value] used in expression does not exist in mappings];\n\tat org.elasticsearch.search.SearchService.parseSource(SearchService.java:855)\n\tat org.elasticsearch.search.SearchService.createContext(SearchService.java:654)\n\tat org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:620)\n\tat org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:371)\n\tat org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:368)\n\tat org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:365)\n\tat 
...
    "exception_class": "org.elasticsearch.transport.RemoteTransportException",
    "exception_message": "[ops-elk-2][10.0.197.130:9300][indices:data\/read\/search[phase\/query]]"
  },
  "source_host": "ops-elk-1",
  "method": "onFirstPhaseResult",
  "level": "DEBUG",
  "message": "[ops-elk-1] [logstash-2016.07.06][0], node[d2qEmDULQzyXxCmRyIhSuw], [R], v[3], s[STARTED], a[id=P--qsxm0SWOBm8O8uhAGFQ]: Failed to execute [org.elasticsearch.action.search.SearchRequest@17b78bf8]",
  "mdc": {},
  "@timestamp": "2016-07-06T06:36:24.108Z",
  "file": "AbstractSearchAsyncAction.java",
  "line_number": "193",
  "thread_name": "elasticsearch[ops-elk-1][transport_client_worker][T#2]{New I\/O worker #2}",
  "@version": 1,
  "logger_name": "action.search",
  "class": "org.elasticsearch.action.search.AbstractSearchAsyncAction"
}
4

Well, I deleted and recreated the index in Kibana and the error cleared up. But I still am no closer to knowing the cause.

5

It could have been a scripted field that someone created.

6

That field is output by some of our services. And it's certainly used in kibana aggregations for visualizations. But in the kibana mappings page it did not show any scripted fields. Nor did GET /logstash-xxxx/_mapping show any scripted fields.