# Elasticsearch Cluster turns Red - Is JVM Heap main culprit?
**URL:** https://discuss.elastic.co/t/elasticsearch-cluster-turns-red-is-jvm-heap-main-culprit/168865
**Category:** Elasticsearch
**Created:** [February 18, 2019, 3:52pm UTC](https://discuss.elastic.co/t/elasticsearch-cluster-turns-red-is-jvm-heap-main-culprit/168865 "2019-02-18T15:52:51Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@Nikesh](https://discuss.elastic.co/u/Nikesh)
#### Post date: [February 18, 2019, 3:52pm UTC](https://discuss.elastic.co/t/elasticsearch-cluster-turns-red-is-jvm-heap-main-culprit/168865/1 "2019-02-18T15:52:51Z")
Hi,
I am indexing at a decent rate.
20 indices each with 10000 fields and 50000 documents continuously indexing through 9 threads.
I have a cluster with a dedicated master node and two data nodes. Each node has 16GB RAM and 8GB Heap size.
No issues were found when only indexing the above mentioned scenario although it was at peak(let's say 6.5 to 7.5 GB). JVM Heap crossed the upper limit when more indexing and search requests were performed. Cluster went to Red Status and OOM error was thrown in logs.
My doubts are :
1. What contributes to JVM Heap? As I have both text and keyword for a single field.
**fielddata** stays in in-memory but is not enabled by default and I have not changed this behaviour.
**Stored\_fields** also contributes to JVM Heap.
I have attached Kibana Screenshot at the time of indexing (partly).
2. What measures can be taken to bring down Heap-Size or rather prevent Heap-Size to reach its maximum?
3.Even though I restarted my cluster, JVM Heap didn't drop down after giving it some time. What can be the causes of this?
---
### Author:  [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)
#### Post date: [February 18, 2019, 6:58pm UTC](https://discuss.elastic.co/t/elasticsearch-cluster-turns-red-is-jvm-heap-main-culprit/168865/2 "2019-02-18T18:58:50Z")
> [@Nikesh](#):
>
> 10000 fields
Is that a typo? If not, that seems excessive and is likely to cause issues.
How many shards do you have?
---
### Author:  [@Nikesh](https://discuss.elastic.co/u/Nikesh)
#### Post date: [February 19, 2019, 4:53am UTC](https://discuss.elastic.co/t/elasticsearch-cluster-turns-red-is-jvm-heap-main-culprit/168865/3 "2019-02-19T04:53:20Z")
Thanks David for the response.
Unfortunately, it's not an typo. It is common case for my users to have 5000-7000 fields per index.
I have configured 2 primary shards and 1 replica per index.
---
### Author:  [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)
#### Post date: [February 19, 2019, 9:06am UTC](https://discuss.elastic.co/t/elasticsearch-cluster-turns-red-is-jvm-heap-main-culprit/168865/4 "2019-02-19T09:06:06Z")
> [@Nikesh](#):
>
> Unfortunately, it's not an typo. It is common case for my users to have 5000-7000 fields per index.
There's a good reason that the default limit in Elasticsearch is 1000 fields per index. I recommend working towards respecting that limit.
> [@Nikesh](#):
>
> I have configured 2 primary shards and 1 replica per index.
I meant in total. How many shards do you have in total? As in, what does `GET _cluster/health` report?
---
### Author:  [@Nikesh](https://discuss.elastic.co/u/Nikesh)
#### Post date: [February 19, 2019, 10:05am UTC](https://discuss.elastic.co/t/elasticsearch-cluster-turns-red-is-jvm-heap-main-culprit/168865/5 "2019-02-19T10:05:23Z")
My Cluster consists of 1300 shards!
---
### Author:  [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)
#### Post date: [February 19, 2019, 12:17pm UTC](https://discuss.elastic.co/t/elasticsearch-cluster-turns-red-is-jvm-heap-main-culprit/168865/6 "2019-02-19T12:17:18Z")
This also sounds like too many for your cluster size. See this article for more detail:
> **[How many shards should I have in my Elasticsearch cluster?](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster)**
In particular:
> _A good rule-of-thumb is to ensure you keep the number of shards per node below 20 per GB heap it has configured._
Thus with 8GB of heap you should aim to limit yourself to 160 shards per node.
---
### Author:  [@Nikesh](https://discuss.elastic.co/u/Nikesh)
#### Post date: [February 28, 2019, 12:09pm UTC](https://discuss.elastic.co/t/elasticsearch-cluster-turns-red-is-jvm-heap-main-culprit/168865/7 "2019-02-28T12:09:16Z")
Thanks David for your continuous help!
Is there a similar thumb rule as to how much GB of data a particular shard can hold?
---
### Author:  [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)
#### Post date: [February 28, 2019, 12:23pm UTC](https://discuss.elastic.co/t/elasticsearch-cluster-turns-red-is-jvm-heap-main-culprit/168865/8 "2019-02-28T12:23:35Z")
From the very same article:
> _Aim to keep the average shard size between at least a few GB and a few tens of GB. For use-cases with time-based data, it is common to see shards between 20GB and 40GB in size._
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 28, 2019, 12:23pm UTC](https://discuss.elastic.co/t/elasticsearch-cluster-turns-red-is-jvm-heap-main-culprit/168865/9 "2019-03-28T12:23:41Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.