# Why Cluster Status is Red, please help **URL:** \ **Category:** Elasticsearch\ **Created:** [January 20, 2016, 1:36pm UTC](https://discuss.elastic.co/t/why-cluster-status-is-red-please-help/39659 "2016-01-20T13:36:47Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![thyfere](https://avatars.discourse-cdn.com/v4/letter/t/48db29/32.png) [@thyfere](https://discuss.elastic.co/u/thyfere)\ **Post date:** [January 20, 2016, 1:36pm UTC](https://discuss.elastic.co/t/why-cluster-status-is-red-please-help/39659/1 "2016-01-20T13:36:47Z")
Hi, I ran this command curl -XGET localhost:9200/\_cluster//health?pretty=true and got the following result: "cluster\_name" : "elasticsearch", "status" : "red", "timed\_out" : false, "number\_of\_nodes" : 1, "number\_of\_data\_nodes" : 1, "active\_primary\_shards" : 504, "active\_shards" : 504, "relocating\_shards" : 0, "initializing\_shards" : 0, "unassigned\_shards" : 688, "delayed\_unassigned\_shards" : 0, "number\_of\_pending\_tasks" : 1, "number\_of\_in\_flight\_fetch" : 0 I am running Elasticsearch and Kibana on Windows box. The following are my current log entries: [2016-01-20 16:30:51,130][WARN][cluster.action.shard] [ABC-Elastic1] [logstash-2016.01.01][2] received shard failed for [logstash-2016.01.01][2], node[j\_ZQyQsvRRGBvPo1NYifeA], [P], s[INITIALIZING], unassigned\_info[[reason=ALLOCATION\_FAILED], at[2016-01-20T13:30:45.254Z], details[shard failure [failed recovery][IndexShardGatewayRecoveryException[[logstash-2016.01.01][2] failed recovery]; nested: ElasticsearchException[failed to obtain write log pre translog recovery]; nested: LockObtainFailedException[Lock obtain timed out: NativeFSLock@\[xyz-synology1.abc.xyz.edu](http://xyz-synology1.abc.xyz.edu)\syslog\elasticsearch\nodes\0\indices\logstash-2016.01.01\2\index\write.lock]; ]]], indexUUID [XnyU0jKKRJ6vsz7j3xdRrA], reason [shard failure [failed recovery][IndexShardGatewayRecoveryException[[logstash-2016.01.01][2] failed recovery]; nested: ElasticsearchException[failed to obtain write log pre translog recovery]; nested: LockObtainFailedException[Lock obtain timed out: NativeFSLock@\[xyz-synology1.abc.xyz.edu](http://xyz-synology1.abc.xyz.edu)\syslog\elasticsearch\nodes\0\indices\logstash-2016.01.01\2\index\write.lock]; ]] [2016-01-20 16:30:57,928][INFO][monitor.jvm] [ABC-Elastic1] [gc][old][1343][222] duration [6.5s], collections [1]/[7s], total [6.5s]/[5.2m], memory [2.8gb]-\>[2.8gb]/[2.9gb], all\_pools {[young] [146.4kb]-\>[8mb]/[133.1mb]}{[survivor] [0b]-\>[0b]/[16.6mb]}{[old] [2.8gb]-\>[2.8gb]/[2.8gb]} [2016-01-20 16:30:58,459][INFO][cluster.metadata] [ABC-Elastic1] [logstash-2016.01.20] update\_mapping [nxlog-json] (dynamic) [2016-01-20 16:31:04,835][INFO][monitor.jvm] [ABC-Elastic1] [gc][old][1345][223] duration [5.5s], collections [1]/[5.9s], total [5.5s]/[5.3m], memory [2.9gb]-\>[2.8gb]/[2.9gb], all\_pools {[young] [133.1mb]-\>[12.6mb]/[133.1mb]}{[survivor] [15mb]-\>[0b]/[16.6mb]}{[old] [2.8gb]-\>[2.8gb]/[2.8gb]} [2016-01-20 16:31:05,194][INFO][cluster.metadata] [ABC-Elastic1] [logstash-2016.01.20] update\_mapping [nxlog-json] (dynamic) [2016-01-20 16:31:06,788][WARN][indices.cluster] [ABC-Elastic1] [[logstash-2016.01.01][2]] marking and sending shard failed due to [failed recovery] org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [logstash-2016.01.01][2] failed recovery at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:162) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.elasticsearch.ElasticsearchException: failed to obtain write log pre translog recovery at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:228) at org.elasticsearch.index.gateway.IndexShardGatewayService$1.run(IndexShardGatewayService.java:112) ... 3 more Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: NativeFSLock@\[xyz-synology1.abc.xyz.edu](http://xyz-synology1.abc.xyz.edu)\syslog\elasticsearch\nodes\0\indices\logstash-2016.01.01\2\index\write.lock at org.apache.lucene.store.Lock.obtain(Lock.java:89) at org.elasticsearch.common.lucene.Lucene.acquireLock(Lucene.java:149) at org.elasticsearch.common.lucene.Lucene.acquireWriteLock(Lucene.java:140) at org.elasticsearch.index.gateway.local.LocalIndexShardGateway.recover(LocalIndexShardGateway.java:183) ... 4 more ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 20, 2016, 9:17pm UTC](https://discuss.elastic.co/t/why-cluster-status-is-red-please-help/39659/2 "2016-01-20T21:17:34Z")
Basically it's red cause you have unassigned primaries. It looks like you are using NFS and it's timing out. ---
**Author:** ![thyfere](https://avatars.discourse-cdn.com/v4/letter/t/48db29/32.png) [@thyfere](https://discuss.elastic.co/u/thyfere)\ **Post date:** [January 20, 2016, 10:01pm UTC](https://discuss.elastic.co/t/why-cluster-status-is-red-please-help/39659/3 "2016-01-20T22:01:31Z")
Thanks. What could be the reason for timing out? I am a newbie to elk stack. Btw, what are primaries. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 20, 2016, 10:31pm UTC](https://discuss.elastic.co/t/why-cluster-status-is-red-please-help/39659/4 "2016-01-20T22:31:07Z")
NFS is slow by design, but then you are on a consumer NAS which is slow again. Also take a read of [https://www.elastic.co/guide/en/elasticsearch/guide/master/\_how\_primary\_and\_replica\_shards\_interact.html](https://www.elastic.co/guide/en/elasticsearch/guide/master/_how_primary_and_replica_shards_interact.html) ---
**Author:** ![thyfere](https://avatars.discourse-cdn.com/v4/letter/t/48db29/32.png) [@thyfere](https://discuss.elastic.co/u/thyfere)\ **Post date:** [January 21, 2016, 5:30am UTC](https://discuss.elastic.co/t/why-cluster-status-is-red-please-help/39659/5 "2016-01-21T05:30:34Z")
But if we need any reliable external storage, what should we use, SAN Vol? So, cluster is in red status due to slow NAS? I am using only one node, should I turn off the replica? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 21, 2016, 5:31am UTC](https://discuss.elastic.co/t/why-cluster-status-is-red-please-help/39659/6 "2016-01-21T05:31:36Z")
Turning off replicas won't help, it just looks like the NAS is slow. ---
**Author:** ![thyfere](https://avatars.discourse-cdn.com/v4/letter/t/48db29/32.png) [@thyfere](https://discuss.elastic.co/u/thyfere)\ **Post date:** [January 21, 2016, 6:20am UTC](https://discuss.elastic.co/t/why-cluster-status-is-red-please-help/39659/7 "2016-01-21T06:20:48Z")
So, if you have to suggest someone a solution for a mounted volume, will you suggest NAS or SAN volume or any other solution? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 21, 2016, 6:23am UTC](https://discuss.elastic.co/t/why-cluster-status-is-red-please-help/39659/8 "2016-01-21T06:23:45Z")
Local is king. NAS and SAN can work, they just need to be decent quality. ---
**Author:** ![thyfere](https://avatars.discourse-cdn.com/v4/letter/t/48db29/32.png) [@thyfere](https://discuss.elastic.co/u/thyfere)\ **Post date:** [January 21, 2016, 6:25am UTC](https://discuss.elastic.co/t/why-cluster-status-is-red-please-help/39659/9 "2016-01-21T06:25:43Z")
In fact, it's a VMware virtual machine; therefore, we want to use any reliable data storage source. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:22pm UTC](https://discuss.elastic.co/t/why-cluster-status-is-red-please-help/39659/10 "2017-07-05T23:22:42Z")