# Elasticsearch status:red

**URL:** <https://discuss.elastic.co/t/elasticsearch-status-red/109932>\
**Category:** Elasticsearch\
**Created:** [December 1, 2017, 1:16pm UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932 "2017-12-01T13:16:24Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![minidan](https://avatars.discourse-cdn.com/v4/letter/m/439d5e/32.png) [@minidan](https://discuss.elastic.co/u/minidan)\
**Post date:** [December 1, 2017, 1:16pm UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/1 "2017-12-01T13:16:25Z")

</div>

Hi team and all,

Newbie to ELK and evaluating to put live soon, was all working until it stopped... no changes made just adding servers in one by one using winbeatlog direct to elasticsearch.

Can someone tell me what I need to do to resolve this in simple language please?

Bitnami ELK Stack 5.6.3 on Windows Server 2012 R2

C:\Users\administrator\>curl -XGET [http://localhost:9200/\_cluster/health?pretty=t](http://localhost:9200/_cluster/health?pretty=t)  
rue  
{  
"cluster\_name" : "elasticsearch",  
"status" : "red",  
"timed\_out" : false,  
"number\_of\_nodes" : 1,  
"number\_of\_data\_nodes" : 1,  
"active\_primary\_shards" : 1594,  
"active\_shards" : 1594,  
"relocating\_shards" : 0,  
"initializing\_shards" : 4,  
"unassigned\_shards" : 1943,  
"delayed\_unassigned\_shards" : 0,  
"number\_of\_pending\_tasks" : 206,  
"number\_of\_in\_flight\_fetch" : 0,  
"task\_max\_waiting\_in\_queue\_millis" : 742331,  
"active\_shards\_percent\_as\_number" : 45.01553233549845  
}

---

<div class="post-metadata">

**Author:** ![Fram\_Souza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fram_souza/32/24833_2.png) [@Fram\_Souza](https://discuss.elastic.co/u/Fram_Souza)\
**Post date:** [December 1, 2017, 1:20pm UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/2 "2017-12-01T13:20:45Z")

</div>

Hi @minidan,

Run GET \_cluster/allocation/explain?pretty to see why the shards are not allocated, from then on, the resolution will become easier. 😉

Some primary shard has been deallocated, but there are 4 shards that are being initialized

---

<div class="post-metadata">

**Author:** ![minidan](https://avatars.discourse-cdn.com/v4/letter/m/439d5e/32.png) [@minidan](https://discuss.elastic.co/u/minidan)\
**Post date:** [December 1, 2017, 1:32pm UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/3 "2017-12-01T13:32:34Z")

</div>

Thank you, output below, not being a db or programming guy this stuff is quite alien to me....

C:\Users\administrator\>curl -XGET [http://localhost:9200/](http://localhost:9200/)_cluster/allocation/expl  
ain?pretty  
{  
"index" : "winlogbeat-2017.11.15",  
"shard" : 3,  
"primary" : true,  
"current\_state" : "unassigned",  
"unassigned\_info" : {  
"reason" : "CLUSTER\_RECOVERED",  
"at" : "2017-12-01T13:25:05.325Z",  
"last\_allocation\_status" : "throttled"  
},  
"can\_allocate" : "throttled",  
"allocate\_explanation" : "allocation temporarily throttled",  
"node\_allocation\_decisions" : [  
{  
"node\_id" : "ymeD-hFgRkilIoUS3xSgBA",  
"node\_name" : "ymeD-hF",  
"transport\_address" : "172.16.3.33:9300",  
"node\_decision" : "throttled",  
"store" : {  
"in\_sync" : true,  
"allocation\_id" : "WkEMU_-6SXGSX12YIbmq-g"  
},  
"deciders" : [  
{  
"decider" : "throttling",  
"decision" : "THROTTLE",  
"explanation" : "reached the limit of ongoing initial primary recoveri  
es [4], cluster setting [cluster.routing.allocation.node\_initial\_primaries\_recov  
eries=4]"  
}  
]  
}  
]  
}

---

<div class="post-metadata">

**Author:** ![Fram\_Souza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fram_souza/32/24833_2.png) [@Fram\_Souza](https://discuss.elastic.co/u/Fram_Souza)\
**Post date:** [December 1, 2017, 1:35pm UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/4 "2017-12-01T13:35:05Z")

</div>

Oh, you have many shards for a single node. I recommend that you add another node in the cluster for the shards to be redistributed.

---

<div class="post-metadata">

**Author:** ![minidan](https://avatars.discourse-cdn.com/v4/letter/m/439d5e/32.png) [@minidan](https://discuss.elastic.co/u/minidan)\
**Post date:** [December 1, 2017, 1:44pm UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/5 "2017-12-01T13:44:39Z")

</div>

apologies for my total ignorance, but by "node" do you mean I need to spread this load across another server now?

I sense I have a lot more learning to do......

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 1, 2017, 2:08pm UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/8 "2017-12-01T14:08:08Z")

</div>

You probably don’t need 5 shards per index.  
So change the index template and use 1 shard only.

Also may be you don’t need to keep all data around but just the last 7 days?

In which case, remove old indices. You can use elasticsearch curator for that.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 1, 2017, 2:13pm UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/9 "2017-12-01T14:13:20Z")

</div>

As already stated, you need to reduce the shard count. This is a reasonably common mistake, which is why I created [a blog post containing guidance and best practices around sharding](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster).

---

<div class="post-metadata">

**Author:** ![minidan](https://avatars.discourse-cdn.com/v4/letter/m/439d5e/32.png) [@minidan](https://discuss.elastic.co/u/minidan)\
**Post date:** [December 1, 2017, 2:23pm UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/10 "2017-12-01T14:23:19Z")

</div>

Hi both, unfortunately this project is to provide a central logging server with a full historical record, so I cannot trim the data down this way.

I will give the "What is a shard?" page immediately as I have lots to learn. Just to mention, the current configuration is absolutely "out of the box" so just hoping to ascertain the simplest, most reliable way forward really 🙂

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 1, 2017, 2:27pm UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/11 "2017-12-01T14:27:49Z")

</div>

You can reduce the shard count by using the [shrink index API](https://www.elastic.co/guide/en/elasticsearch/reference/6.0/indices-shrink-index.html) to go from 5 to 1 shard per index. Given the number of indices you have and the fact that you want to keep data for a very long period, you probably need to reindex into monthly indices and/or scale out the cluster.

---

<div class="post-metadata">

**Author:** ![minidan](https://avatars.discourse-cdn.com/v4/letter/m/439d5e/32.png) [@minidan](https://discuss.elastic.co/u/minidan)\
**Post date:** [December 1, 2017, 4:03pm UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/12 "2017-12-01T16:03:06Z")

</div>

okay, will have to come back to this later... I'm not even sure I can see where the "5" shards are being referenced now I'm looking back at my outputs? 🤨

Scaling out sounds like it would require less maintenance going forward (this needs to be as close to "set and forget" as possible, very small business with no staff to "maintain" the db going forward.

---

<div class="post-metadata">

**Author:** ![Fram\_Souza](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/fram_souza/32/24833_2.png) [@Fram\_Souza](https://discuss.elastic.co/u/Fram_Souza)\
**Post date:** [December 1, 2017, 8:14pm UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/13 "2017-12-01T20:14:08Z")

</div>

@minidian

5 shards and 1 replica is the default amount set by Elasticsearch, I assume you have not changed that amount. If you have 5 shards, and 1 replica for each shard you will have 10 shards in all. So you have 10 index shards that are being stored within your node

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 2, 2017, 2:11pm UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/14 "2017-12-02T14:11:42Z")

</div>

Leaving it as it is will just make it even more difficult to fix the problems later on. You can set the number of shards to be used through an [index template](https://www.elastic.co/guide/en/elasticsearch/reference/6.0/indices-templates.html). The following will set it to 1 for all newly created indices (assuming you do not have any other index template that overrides it):

```auto
curl -XPUT 'localhost:9200/_template/single_primary_shard?pretty' -H 'Content-Type: application/json' -d'
{
  "index_patterns": ["*"],
  "settings": {
    "number_of_shards": 1
  }
}'

```

If you are using Logstash or Beats to ingest data, I would also recommend changing to weekly or monthly indices (depending on data volume). These measure will slow down the shard growth in the cluster.

---

<div class="post-metadata">

**Author:** ![minidan](https://avatars.discourse-cdn.com/v4/letter/m/439d5e/32.png) [@minidan](https://discuss.elastic.co/u/minidan)\
**Post date:** [December 6, 2017, 11:57am UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/15 "2017-12-06T11:57:56Z")

</div>

Hi again, keep getting pulled away from this... hard to run anything with those variables while the whole thing is stuck. I presume needs subtly reformatting for curl under windows.

Really I'd just like to be able to delete any accrued data and indexes to date and start again at this point.

---

<div class="post-metadata">

**Author:** ![minidan](https://avatars.discourse-cdn.com/v4/letter/m/439d5e/32.png) [@minidan](https://discuss.elastic.co/u/minidan)\
**Post date:** [December 6, 2017, 5:00pm UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/16 "2017-12-06T17:00:06Z")

</div>

Hooray! I have working install again after some cleaning up as above... struggling a bit with something still though.

right now, I only have the winlogbeat-\* template installed, and verified it can be used (200 OK)

I get the following output every time, it's not clear how I use the command above, have tried many variations. Going to investigate support contact options here at some point I think

{  
"error": {  
"root\_cause": [  
{  
"type": "action\_request\_validation\_exception",  
"reason": "Validation Failed: 1: template is missing;"  
}  
],  
"type": "action\_request\_validation\_exception",  
"reason": "Validation Failed: 1: template is missing;"  
},  
"status": 400  
}

---

<div class="post-metadata">

**Author:** ![minidan](https://avatars.discourse-cdn.com/v4/letter/m/439d5e/32.png) [@minidan](https://discuss.elastic.co/u/minidan)\
**Post date:** [December 8, 2017, 9:49am UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/17 "2017-12-08T09:49:21Z")

</div>

Is there a more general resource around here for index templates then?

I've got what I'm supposed to be doing now I think with GET / PUT from my original template, but struggling to insert the section above into my existing template without a syntax error.... MY server is still up and running 3 days in but I'd like to ensure robustness going froward.

Also not really sure how to change to weekly or monthly indices as mentioned. Again, really not coming from a dev/programmer background here so the learning curve is quite steep on assumed knowledge.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 5, 2018, 9:49am UTC](https://discuss.elastic.co/t/elasticsearch-status-red/109932/18 "2018-01-05T09:49:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
