# ElasticSearch 2.2.0 memory dump and node goes out of cluster

**URL:** <https://discuss.elastic.co/t/elasticsearch-2-2-0-memory-dump-and-node-goes-out-of-cluster/46297>\
**Category:** Elasticsearch\
**Created:** [April 4, 2016, 9:39pm UTC](https://discuss.elastic.co/t/elasticsearch-2-2-0-memory-dump-and-node-goes-out-of-cluster/46297 "2016-04-04T21:39:35Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![lokeshhctm](https://avatars.discourse-cdn.com/v4/letter/l/a4c791/32.png) [@lokeshhctm](https://discuss.elastic.co/u/lokeshhctm)\
**Post date:** [April 4, 2016, 9:39pm UTC](https://discuss.elastic.co/t/elasticsearch-2-2-0-memory-dump-and-node-goes-out-of-cluster/46297/1 "2016-04-04T21:39:35Z")

</div>

ElasticSearch version: 2.2.0

JVM version: 1.8

OS version: AWS Linux C3.4xlarge

Description of the problem including expected versus actual behavior:

After we restart ElasticSearch run fine like for 2-3 days and then suddenly one of node in cluster goes out of cluster and on node it show Elasticsearch running but leaves a memory dump on disk.  
It doesn't come back to cluster unless we restart this.

Please find the memory dump location:  
[http://admarvel.s3.amazonaws.com/uploads/lokesh1/java\_pid26509.hprof.zip](http://admarvel.s3.amazonaws.com/uploads/lokesh1/java_pid26509.hprof.zip)

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 4, 2016, 9:41pm UTC](https://discuss.elastic.co/t/elasticsearch-2-2-0-memory-dump-and-node-goes-out-of-cluster/46297/2 "2016-04-04T21:41:42Z")

</div>

Are you asking us to analyse your heap dump?

---

<div class="post-metadata">

**Author:** ![lokeshhctm](https://avatars.discourse-cdn.com/v4/letter/l/a4c791/32.png) [@lokeshhctm](https://discuss.elastic.co/u/lokeshhctm)\
**Post date:** [April 4, 2016, 9:45pm UTC](https://discuss.elastic.co/t/elasticsearch-2-2-0-memory-dump-and-node-goes-out-of-cluster/46297/3 "2016-04-04T21:45:30Z")

</div>

I tried to look into dump and was not able to figure out whether the problem was with filter cache or not.  
I suspected that the filter cache just keep going up regardless of boundary limits. and hence it goes out of memory and generating memory dump.  
I post on git hub as a bug and i am being asked to post it here.  
Can you please help here ?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 4, 2016, 10:04pm UTC](https://discuss.elastic.co/t/elasticsearch-2-2-0-memory-dump-and-node-goes-out-of-cluster/46297/4 "2016-04-04T22:04:07Z")

</div>

I don't have time to do a heap dump analysis sorry, you probably won't find many here that will be able to either.

Are you running Marvel to keep track of what is happening in your cluster?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:02pm UTC](https://discuss.elastic.co/t/elasticsearch-2-2-0-memory-dump-and-node-goes-out-of-cluster/46297/5 "2017-07-05T23:02:28Z")

</div>