# Heap Problem

**URL:** https://discuss.elastic.co/t/heap-problem/61250
**Category:** Elasticsearch
**Created:** [September 22, 2016, 12:12pm UTC](https://discuss.elastic.co/t/heap-problem/61250 "2016-09-22T12:12:42Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Buddhi\_Rathore](https://avatars.discourse-cdn.com/v4/letter/b/6f9a4e/32.png) [@Buddhi\_Rathore](https://discuss.elastic.co/u/Buddhi_Rathore)
#### Post date: [September 22, 2016, 12:12pm UTC](https://discuss.elastic.co/t/heap-problem/61250/1 "2016-09-22T12:12:42Z")

</div>

Hello,  
we using elastic search to process analytic data, we got daily basis analytic dump from our system, we need to do some analysis on analytic data, for this we push analytic data from log files to elastic search using log stash. log stash is working fine but on elastic search when doc's indexed heap is continually grow and after some hours nodes stopped due to memory issue, so we have to restart the cluster nodes, log stash using update groovy script to increase count += some values, so if docs with a id is already exist then it will update otherwise new doc is index,

right now we are not doing any kind of query from the ES cluster,  
our cluster has 4 nodes, 3 data nodes with 4 cpu and 30 GB memory,  
and 1 master node

and index configuration is :

10 shards,  
0 replication,  
60s refresh interval

please help me to resolve the heap issue......

---

<div class="post-metadata">

### Author: ![mainec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mainec/32/5557_2.png) [@mainec](https://discuss.elastic.co/u/mainec)
#### Post date: [September 22, 2016, 12:46pm UTC](https://discuss.elastic.co/t/heap-problem/61250/2 "2016-09-22T12:46:02Z")

</div>

From your description it's not entirely clear to me what's going wrong. You could try and take a look at

> **[Performance Considerations for Elasticsearch Indexing
	  	 | Elastic](https://www.elastic.co/blog/performance-considerations-elasticsearch-indexing)**
>
> Running Elasticsearch 2.0? Check out this updated post about performance considerations for Elasticsearch 2.0 indexing. Elasticsearch users have delightfully diverse use cases, ranging from appen...

for general advice on indexing optimisation.

Hope this helps,  
Isabel

---

<div class="post-metadata">

### Author: ![Buddhi\_Rathore](https://avatars.discourse-cdn.com/v4/letter/b/6f9a4e/32.png) [@Buddhi\_Rathore](https://discuss.elastic.co/u/Buddhi_Rathore)
#### Post date: [September 22, 2016, 1:16pm UTC](https://discuss.elastic.co/t/heap-problem/61250/3 "2016-09-22T13:16:05Z")

</div>

my problem is nodes heap is continuously grow, GC is not cleaning heap.  
and i am just indexing docs

---

<div class="post-metadata">

### Author: ![Kim-Kruse-Hansen](https://avatars.discourse-cdn.com/v4/letter/k/f1d935/32.png) [@Kim-Kruse-Hansen](https://discuss.elastic.co/u/Kim-Kruse-Hansen)
#### Post date: [September 22, 2016, 2:47pm UTC](https://discuss.elastic.co/t/heap-problem/61250/4 "2016-09-22T14:47:18Z")

</div>

I am experiencing the same problem. I am indexing 400 million log lines per day on a 2 node 2.4 cluster and the heap is also slowly growing.Typically at around 95% full , I am hit by a old gc , that can take minutes to complete. This of course causing the the node to fail and potentially a cluster restart.

I have been unable to get any good advice on here so far. So I have upped my heap close to max (29 GB) and now waiting to see the results of this-

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 10:18pm UTC](https://discuss.elastic.co/t/heap-problem/61250/5 "2017-07-05T22:18:04Z")

</div>


