[action.admin.indices.stats] failed to execute operation for shard

Elastic Stack Elasticsearch
1

I saw a lot of that error on my Elasticsearch logs:

[2015-12-01 17:27:03,663][DEBUG][action.admin.indices.stats] [crawler_service_001] [indices:monitor/stats] failed to execute operation for shard [[carrier-2015.12.01][2], node[rCqrCf10SGuAVqW_YYiDng], [P], v[7], s[INITIALIZING], a[id=qHbJSlmPRhCLRc7lrpO_Nw], unassigned_info[[reason=ALLOCATION_FAILED], at[2015-12-01T09:25:09.830Z], details[engine failure, reason [refresh failed], failure NullPointerException[null]]]]
[carrier-2015.12.01][[carrier-2015.12.01][2]] BroadcastShardOperationFailedException[operation indices:monitor/stats failed]; nested: IllegalIndexShardStateException[CurrentState[RECOVERING] operations only allowed when shard state is one of [POST_RECOVERY, STARTED, RELOCATED]];
	at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.onShardOperation(TransportBroadcastByNodeAction.java:405)
	at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.messageReceived(TransportBroadcastByNodeAction.java:382)
	at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.messageReceived(TransportBroadcastByNodeAction.java:371)
	at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:350)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: [carrier-2015.12.01][[carrier-2015.12.01][2]] IllegalIndexShardStateException[CurrentState[RECOVERING] operations only allowed when shard state is one of [POST_RECOVERY, STARTED, RELOCATED]]
	at org.elasticsearch.index.shard.IndexShard.readAllowed(IndexShard.java:974)
	at org.elasticsearch.index.shard.IndexShard.acquireSearcher(IndexShard.java:808)
	at org.elasticsearch.index.shard.IndexShard.docStats(IndexShard.java:628)
	at org.elasticsearch.action.admin.indices.stats.CommonStats.<init>(CommonStats.java:131)
	at org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.shardOperation(TransportIndicesStatsAction.java:165)
	at org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.shardOperation(TransportIndicesStatsAction.java:47)
	at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.onShardOperation(TransportBroadcastByNodeAction.java:401)
	... 7 more

Then I curl -XGET http://localhost:9200/_cat/shards | grep carrier:

carrier-2015.12.01    4 p STARTED      215337 120.8mb 127.0.0.1 crawler_service_001 
carrier-2015.12.01    4 r UNASSIGNED                                                
carrier-2015.12.01    2 p INITIALIZING                127.0.0.1 crawler_service_001 
carrier-2015.12.01    2 r UNASSIGNED                                                
carrier-2015.12.01    1 p STARTED      215626 120.8mb 127.0.0.1 crawler_service_001 
carrier-2015.12.01    1 r UNASSIGNED                                                
carrier-2015.12.01    3 p STARTED      212904  97.1mb 127.0.0.1 crawler_service_001 
carrier-2015.12.01    3 r UNASSIGNED                                                
carrier-2015.12.01    0 p STARTED      214469 120.8mb 127.0.0.1 crawler_service_001 
carrier-2015.12.01    0 r UNASSIGNED

So, what should I do now?

2

I guess you have to wait.
Here we can see that shard 2 of your index is still INITIALIZING. So wait until it goes to STARTED.

You can have a look at pending tasks with the _cat API.

3

Hey hi i too ran into same issue and my output for
curl -XGET http://localhost:9200/_cat/shards | grep carrier:
is as below.

{
"cluster_name" : "elasticsearch",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 11,
"active_shards" : 11,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 11,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 50.0
}

4

And output for UNASSIGNED resulted me as below
curl -XGET http://localhost:9200/_cat/shards | grep UNASSIGNED | awk '{print $2}'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1628 100 1628 0 0 63828 0 --:--:-- --:--:-- --:--:-- 65120
0
1
3
2
4
0
1
3
2
4
0

5

You'd better open a new question.
This thread is old.

Nothing looks bad here. What is your problem?