High CPU usage on slave -- org.elasticsearch.index.IndexShardMissingException: [verbose][3] missing

Elastic Stack Elasticsearch
1

Hi,

The master is running fine. But the slave is using high cpu ~80% constantly
for the past couple of hours. The following log fragement repeats. Any idea
why?

[2013-11-06 06:40:45,421][DEBUG][action.search.type ] [Slave]
[verbose][3], node[SwHOK0KEQ_eCIy-rAf80Zw], relocating
[hEPpftBGR2SK5WC1UK-CpA], [P], s[RELOCATING]: Failed to execute
[org.elasticsearch.action.search.SearchRequest@15df311b]
org.elasticsearch.transport.RemoteTransportException:
[Slave][inet[/10.223.12.32:9300]][search/phase/query]
Caused by: org.elasticsearch.index.IndexShardMissingException: [verbose][3]
missing
at
org.elasticsearch.index.service.InternalIndexService.shardSafe(InternalIndexService.java:188)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:474)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:469)
at
org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:462)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:234)
at
org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:529)
at
org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:518)
at
org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:268)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)

/Krishna

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

Hey,

this exception basically only tells you, that an expected shard seems not
be on this node. The question is merely, why this is the case. Do you have
any other exceptions in the logfile of that node, which might give a hint
for further problems?

--Alex

On Wed, Nov 6, 2013 at 6:47 AM, Krishna Kumar Perumalla <
krishnakumar85@gmail.com> wrote:

Hi,

The master is running fine. But the slave is using high cpu ~80%
constantly for the past couple of hours. The following log fragement
repeats. Any idea why?

[2013-11-06 06:40:45,421][DEBUG][action.search.type ] [Slave]
[verbose][3], node[SwHOK0KEQ_eCIy-rAf80Zw], relocating
[hEPpftBGR2SK5WC1UK-CpA], [P], s[RELOCATING]: Failed to execute
[org.elasticsearch.action.search.SearchRequest@15df311b]
org.elasticsearch.transport.RemoteTransportException:
[Slave][inet[/10.223.12.32:9300]][search/phase/query]
Caused by: org.elasticsearch.index.IndexShardMissingException:
[verbose][3] missing
at
org.elasticsearch.index.service.InternalIndexService.shardSafe(InternalIndexService.java:188)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:474)
at
org.elasticsearch.search.SearchService.createContext(SearchService.java:469)
at
org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:462)
at
org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:234)
at
org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:529)
at
org.elasticsearch.search.action.SearchServiceTransportAction$SearchQueryTransportHandler.messageReceived(SearchServiceTransportAction.java:518)
at
org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:268)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)

/Krishna

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.