Some indices failing with "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed]"

Elastic Stack Elasticsearch
1

Everything was working fine when all of a sudden some indices started
failing.

GET localhost:9200/logstash-2014.09.11/_search
yields response:
{"error":"SearchPhaseExecutionException[Failed to execute phase [query],
all shards failed]","status":503}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/67143a50-7cd9-4836-a19f-a2911bd05e1f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

On Friday, September 12, 2014 at 08:53 CEST,
Kevin DeLand kevin.deland@gmail.com wrote:

Everything was working fine when all of a sudden some indices started
failing.
GET localhost:9200/logstash-2014.09.11/_search
yields response:
{"error":"SearchPhaseExecutionException[Failed to execute phase
[query], all shards failed]","status":503}

How's the cluster's health? Anything interesting in the Elasticsearch
logs?

--
Magnus Bäck | Software Engineer, Development Tools
magnus.back@sonymobile.com | Sony Mobile Communications

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/20140912065653.GC3212%40seldlx20533.corpusers.net.
For more options, visit https://groups.google.com/d/optout.

3

Cluster health is red:

On Friday, September 12, 2014 2:57:06 AM UTC-4, Magnus Bäck wrote:

On Friday, September 12, 2014 at 08:53 CEST,
Kevin DeLand <kevin....@gmail.com <javascript:>> wrote:

Everything was working fine when all of a sudden some indices started
failing.
GET localhost:9200/logstash-2014.09.11/_search
yields response:
{"error":"SearchPhaseExecutionException[Failed to execute phase
[query], all shards failed]","status":503}

How's the cluster's health? Anything interesting in the Elasticsearch
logs?

--
Magnus Bäck | Software Engineer, Development Tools
magnu...@sonymobile.com <javascript:> | Sony Mobile Communications

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6849f141-64f0-4a15-99ca-ba43192f7064%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

Which logs should I look in?

From
*~/elasticsearch-1.3.1/logs/elasticsearch.log *
[2014-09-12 07:17:26,047][DEBUG][action.bulk ] [Frankie and
Victoria] observer: timeout notification from cluster service. timeout
setting [1m], time since start [1m]
[2014-09-12 07:18:04,466][DEBUG][action.search.type ] [Frankie and
Victoria] All shards failed for phase: [query]
[2014-09-12 07:18:27,103][DEBUG][action.bulk ] [Frankie and
Victoria] observer: timeout notification from cluster service. timeout
setting [1m], time since start [1m]
[2014-09-12 07:18:27,103][DEBUG][action.bulk ] [Frankie and
Victoria] observer: timeout notification from cluster service. timeout
setting [1m], time since start [1m]

On Friday, September 12, 2014 2:57:06 AM UTC-4, Magnus Bäck wrote:

On Friday, September 12, 2014 at 08:53 CEST,
Kevin DeLand <kevin....@gmail.com <javascript:>> wrote:

Everything was working fine when all of a sudden some indices started
failing.
GET localhost:9200/logstash-2014.09.11/_search
yields response:
{"error":"SearchPhaseExecutionException[Failed to execute phase
[query], all shards failed]","status":503}

How's the cluster's health? Anything interesting in the Elasticsearch
logs?

--
Magnus Bäck | Software Engineer, Development Tools
magnu...@sonymobile.com <javascript:> | Sony Mobile Communications

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/06391bb2-2ab4-4168-9ced-0ebadd6f08ba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

There are six indices with a red cluster status, but only two fail... any
advice on what to check?

On Friday, September 12, 2014 2:57:06 AM UTC-4, Magnus Bäck wrote:

On Friday, September 12, 2014 at 08:53 CEST,
Kevin DeLand <kevin....@gmail.com <javascript:>> wrote:

Everything was working fine when all of a sudden some indices started
failing.
GET localhost:9200/logstash-2014.09.11/_search
yields response:
{"error":"SearchPhaseExecutionException[Failed to execute phase
[query], all shards failed]","status":503}

How's the cluster's health? Anything interesting in the Elasticsearch
logs?

--
Magnus Bäck | Software Engineer, Development Tools
magnu...@sonymobile.com <javascript:> | Sony Mobile Communications

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4f70cae6-8155-40b7-b458-0e6687000ce7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.