I have a lot of daily indices, each with two shards. Yet a dashboard that predominantly searches two named indices (both of which are daily) tells me I'm search 350 shards (and some are failing to respond) with the last 15m selected.
My question is why? I can understand the queries involved being cast across a handful of shards but not 300+!
Is there any way I can debug what might be going on? This is with Elasticsearch 2.4 and Kibana 4.6.6.
I've found the problem: the templates created each index with a static alias for the use of a separate application and some of the saved queries used by the dashboard were referencing this static alias.
I went into Kibana Objects and updated them. My dashboard now loads without shard failures being reported.
Of course, multi-month queries will still show the fault but at least that's a different problem.
Incidentally the way to diagnose this (for others reading) was to use the web developer tools in the browser to find the request made of Kibana. The Request body has the query including the index names involved.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.