# Red status: blocked by: \[SERVICE\_UNAVAILABLE/1/state not recovered / initialized\]

**URL:** <https://discuss.elastic.co/t/red-status-blocked-by-service-unavailable-1-state-not-recovered-initialized/201520>\
**Category:** Elasticsearch\
**Created:** [September 29, 2019, 6:35am UTC](https://discuss.elastic.co/t/red-status-blocked-by-service-unavailable-1-state-not-recovered-initialized/201520 "2019-09-29T06:35:06Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Behzad\_Rezaie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/behzad_rezaie/32/48588_2.png) [@Behzad\_Rezaie](https://discuss.elastic.co/u/Behzad_Rezaie)\
**Post date:** [September 29, 2019, 6:35am UTC](https://discuss.elastic.co/t/red-status-blocked-by-service-unavailable-1-state-not-recovered-initialized/201520/1 "2019-09-29T06:35:06Z")

</div>

Hi pals,

A few days ago we got the following error on our elastic node:

```auto
[FORBIDDEN/12/index read-only / allow delete (api)]

```

We found out that more disk space is needed, since only 5% of the space had remained for the elastic node. Next, we added more disk space and now the elastic node has occupied only 35% of the space. With this in mind, we are still facing the same error message! After googling, I disabled the searchguard plugin and ran the following command:

```auto
POST /_settings
{
  "index.blocks.read_only_allow_delete": null
}

```

But faced the following error and the elastic search status is RED!

```auto
{
    "error": {
        "root_cause": [
            {
                "type": "cluster_block_exception",
                "reason": "blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];"
            }
        ],
        "type": "master_not_discovered_exception",
        "reason": "ClusterBlockException[blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];]",
        "caused_by": {
            "type": "cluster_block_exception",
            "reason": "blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];"
        }
    },
    "status": 503
}

```

Here is `elasticsearch.yml`:

```auto
bootstrap.memory_lock: false
cluster.name: elasticsearch
http.port: 9200
node.data: true
node.ingest: true
node.master: true
node.max_local_storage_nodes: 1
node.name: ELASTIC-SRV1
path.data: D:\Program\ELK\data
path.logs: D:\Program\ELK\logs
transport.tcp.port: 9300
xpack.license.self_generated.type: basic
xpack.security.enabled: false

searchguard.disabled: true
logger.org.elasticsearch.gateway: TRACE

network.host: 0.0.0.0
http.cors.allow-origin: "*"
http.cors.enabled: true

```

And the following are the status of the elastic node:

```auto
{
  "name" : "ELASTIC-SRV1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "_na_",
  "version" : {
    "number" : "6.5.4",
    "build_flavor" : "unknown",
    "build_type" : "unknown",
    "build_hash" : "d2ef93d",
    "build_date" : "2018-12-17T21:17:40.758843Z",
    "build_snapshot" : false,
    "lucene_version" : "7.5.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

```

```auto
# GET /_cat/master
Pbra1KbPRzGKWxoIxPg0Lw <IP Address> <IP Address> ELASTIC-SRV1

```

```auto
# GET /_cat/health
1569738758 06:32:38 elasticsearch red 1 1 223 223 0 4 11954 5 2.5s 1.8%

```

Could you please help me with the problem?!

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [September 29, 2019, 6:47am UTC](https://discuss.elastic.co/t/red-status-blocked-by-service-unavailable-1-state-not-recovered-initialized/201520/2 "2019-09-29T06:47:47Z")

</div>

> [@Behzad\_Rezaie](#):
>
> But faced the following error and the Elasticsearch status is RED!
> 
> ```auto
> ... blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];...
> 
> ```

Did you run that command very soon after starting Elasticsearch? If you try again does it report the same error?

You should expect that message if Elasticsearch is still starting up, but the response to `GET _cat/health` you shared suggests it should work now.

---

<div class="post-metadata">

**Author:** ![Behzad\_Rezaie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/behzad_rezaie/32/48588_2.png) [@Behzad\_Rezaie](https://discuss.elastic.co/u/Behzad_Rezaie)\
**Post date:** [September 29, 2019, 6:52am UTC](https://discuss.elastic.co/t/red-status-blocked-by-service-unavailable-1-state-not-recovered-initialized/201520/3 "2019-09-29T06:52:39Z")

</div>

> [@DavidTurner](#):
>
> Did you run that command very soon after starting Elasticsearch? If you try again does it report the same error?

I guess it was not soon. It was around 1 minute after starting the Elasticsearch. I am still getting the same error:

```auto
[2019-09-29T10:17:23,284][ERROR][o.e.x.m.c.i.IndexRecoveryCollector] [ELASTIC-SRV1] collector [index_recovery] failed to collect data
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
	at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:166) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.action.admin.indices.recovery.TransportRecoveryAction.checkGlobalBlock(TransportRecoveryAction.java:114) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.action.admin.indices.recovery.TransportRecoveryAction.checkGlobalBlock(TransportRecoveryAction.java:52) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$AsyncAction.<init>(TransportBroadcastByNodeAction.java:255) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction.doExecute(TransportBroadcastByNodeAction.java:233) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction.doExecute(TransportBroadcastByNodeAction.java:78) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:81) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:87) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:76) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:395) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:384) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1242) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:46) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:60) ~[elasticsearch-6.5.4.jar:6.5.4]
	at org.elasticsearch.xpack.monitoring.collector.indices.IndexRecoveryCollector.doCollect(IndexRecoveryCollector.java:76) ~[x-pack-monitoring-6.5.4.jar:6.5.4]
	at org.elasticsearch.xpack.monitoring.collector.Collector.collect(Collector.java:87) [x-pack-monitoring-6.5.4.jar:6.5.4]
	at org.elasticsearch.xpack.monitoring.MonitoringService$MonitoringExecution$1.doRun(MonitoringService.java:278) [x-pack-monitoring-6.5.4.jar:6.5.4]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.5.4.jar:6.5.4]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_202]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_202]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:624) [elasticsearch-6.5.4.jar:6.5.4]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_202]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_202]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_202]

```

> [@DavidTurner](#):
>
> You should expect that message if Elasticsearch is still starting up, but the response to `GET _cat/health` you shared suggests it should work now.

You mean, the `red` status is not important for now?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [September 29, 2019, 7:01am UTC](https://discuss.elastic.co/t/red-status-blocked-by-service-unavailable-1-state-not-recovered-initialized/201520/4 "2019-09-29T07:01:47Z")

</div>

The error you shared in your second message doesn't look like it came from an API call. What happens if you try the `POST /_settings` command again?

> [@Behzad\_Rezaie](#):
>
> You mean, the `red` status is not important for now?

I mean that your cluster reports 4 initializing shards, which indicates it is no longer in the `state not recovered` state.

---

<div class="post-metadata">

**Author:** ![Behzad\_Rezaie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/behzad_rezaie/32/48588_2.png) [@Behzad\_Rezaie](https://discuss.elastic.co/u/Behzad_Rezaie)\
**Post date:** [September 29, 2019, 8:10am UTC](https://discuss.elastic.co/t/red-status-blocked-by-service-unavailable-1-state-not-recovered-initialized/201520/5 "2019-09-29T08:10:44Z")

</div>

> [@DavidTurner](#):
>
> The error you shared in your second message doesn't look like it came from an API call. What happens if you try the `POST /_settings` command again?

The second message was extracted from the log file. When calling the

```auto
POST /_settings
{
  "index.blocks.read_only_allow_delete": null
}

```

I get

```auto
{
    "acknowledged": false
}

```

What does that mean?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [September 29, 2019, 9:18am UTC](https://discuss.elastic.co/t/red-status-blocked-by-service-unavailable-1-state-not-recovered-initialized/201520/6 "2019-09-29T09:18:43Z")

</div>

It means something went wrong processing that setting update, perhaps a timeout, but maybe it went through in the end. There will be more information in the logs.

---

<div class="post-metadata">

**Author:** ![Behzad\_Rezaie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/behzad_rezaie/32/48588_2.png) [@Behzad\_Rezaie](https://discuss.elastic.co/u/Behzad_Rezaie)\
**Post date:** [September 29, 2019, 11:45am UTC](https://discuss.elastic.co/t/red-status-blocked-by-service-unavailable-1-state-not-recovered-initialized/201520/7 "2019-09-29T11:45:58Z")

</div>

I removed all the indices and restarted the elasticsearch and now it works

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [September 29, 2019, 1:55pm UTC](https://discuss.elastic.co/t/red-status-blocked-by-service-unavailable-1-state-not-recovered-initialized/201520/8 "2019-09-29T13:55:01Z")

</div>

Seems a bit drastic. I'm pretty sure there was a solution that didn't involve deleting everything.

---

<div class="post-metadata">

**Author:** ![pestevao](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pestevao/32/44138_2.png) [@pestevao](https://discuss.elastic.co/u/pestevao)\
**Post date:** [October 1, 2019, 3:33pm UTC](https://discuss.elastic.co/t/red-status-blocked-by-service-unavailable-1-state-not-recovered-initialized/201520/9 "2019-10-01T15:33:41Z")

</div>

We have had a similar issue.  
Deleting all indices and all data folder does not solved it.  
It was some network NIC configuration that have changed - don't know why but it enabled jumbo frames and some network packets stop flowing - reset to it original configuration and everything start working smoothly as always.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 29, 2019, 3:33pm UTC](https://discuss.elastic.co/t/red-status-blocked-by-service-unavailable-1-state-not-recovered-initialized/201520/10 "2019-10-29T15:33:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.