# ElasticSearch error - org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed **URL:** \ **Category:** Elasticsearch\ **Created:** [April 17, 2019, 6:01pm UTC](https://discuss.elastic.co/t/elasticsearch-error-org-elasticsearch-action-search-searchphaseexecutionexception-all-shards-failed/177350 "2019-04-17T18:01:47Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![robp21751](https://avatars.discourse-cdn.com/v4/letter/r/ed655f/32.png) [@robp21751](https://discuss.elastic.co/u/robp21751)\ **Post date:** [April 17, 2019, 6:01pm UTC](https://discuss.elastic.co/t/elasticsearch-error-org-elasticsearch-action-search-searchphaseexecutionexception-all-shards-failed/177350/1 "2019-04-17T18:01:47Z")
I am not collecting any logs. The drive became 85% full which is what started this issue I believe. It is now at 60% I ran the following, which did not help ``` curl -X PUT "localhost:9200/.kibana/_settings" -H 'Content-Type: application/json' -d' { "index.blocks.read_only_allow_delete": null } ``` ' elasticsearch.log ``` [2019-04-17T13:56:16,890][WARN][r.suppressed] [abEucW7] path: /.kibana/doc/_count, params: {index=.kibana, type=doc} org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:291) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:133) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:254) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:100) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.action.search.InitialSearchPhase.access$100(InitialSearchPhase.java:48) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.action.search.InitialSearchPhase$2.lambda$onFailure$1(InitialSearchPhase.java:220) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.action.search.InitialSearchPhase.maybeFork(InitialSearchPhase.java:174) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.action.search.InitialSearchPhase.access$000(InitialSearchPhase.java:48) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.action.search.InitialSearchPhase$2.onFailure(InitialSearchPhase.java:220) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:73) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:59) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:463) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1108) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1220) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1194) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:60) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.action.support.ChannelActionListener.onFailure(ChannelActionListener.java:56) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.search.SearchService$2.onFailure(SearchService.java:366) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:360) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:354) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.search.SearchService$4.doRun(SearchService.java:1085) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:751) [elasticsearch-6.7.1.jar:6.7.1] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.7.1.jar:6.7.1] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:834) [?:?] ``` logstash-plain.log `[2019-04-17T13:58:36,696][INFO][logstash.outputs.elasticsearch] retrying failed action with response code: 403 ({"type"=>"cluster_block_exception", "reason"=>"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"})` ---
**Author:** ![robp21751](https://avatars.discourse-cdn.com/v4/letter/r/ed655f/32.png) [@robp21751](https://discuss.elastic.co/u/robp21751)\ **Post date:** [April 17, 2019, 6:03pm UTC](https://discuss.elastic.co/t/elasticsearch-error-org-elasticsearch-action-search-searchphaseexecutionexception-all-shards-failed/177350/2 "2019-04-17T18:03:32Z")
More info, if this helps ``` curl -XGET 'localhost:9200/_cluster/health?pretty' { "cluster_name" : "elasticsearch", "status" : "yellow", "timed_out" : false, "number_of_nodes" : 1, "number_of_data_nodes" : 1, "active_primary_shards" : 223, "active_shards" : 223, "relocating_shards" : 0, "initializing_shards" : 0, "unassigned_shards" : 205, "delayed_unassigned_shards" : 0, "number_of_pending_tasks" : 0, "number_of_in_flight_fetch" : 0, "task_max_waiting_in_queue_millis" : 0, "active_shards_percent_as_number" : 52.10280373831776 } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 15, 2019, 6:03pm UTC](https://discuss.elastic.co/t/elasticsearch-error-org-elasticsearch-action-search-searchphaseexecutionexception-all-shards-failed/177350/3 "2019-05-15T18:03:46Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.