I've been using ES and Kibana for quite a while now, but this morning ES started failing with "Courier Fetch: 5 of 5 shards failed". I started digging into this, and there's nothing in the logs, but if I query older records everything works fine. If I query recent records, looking at the request actually going from Kibana I see:
{
"size": 500,
"sort": [{"@timestamp": {"order": "desc", "unmapped_type": "boolean"}}],
"query": {...},
"highlight": {...},
"_source": {"excludes": []},
"aggs": {...},
"stored_fields": ["*"],
"script_fields": {},
"docvalue_fields": [
...
"data.schedule",
...
]
}
And then in the reply:
"reason":{
"type":"illegal_argument_exception",
"reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [data.schedule] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory."
}
Wat? Why is Kibana sending "data.schedule" in "docvalue_fields"? And why is this suddenly breaking now? Kibana/ES version is 5.1.1.