# Querying Elasticsearch 2.3.0 Causes node to exit cluster

**URL:** <https://discuss.elastic.co/t/querying-elasticsearch-2-3-0-causes-node-to-exit-cluster/48841>\
**Category:** Elasticsearch\
**Created:** [April 29, 2016, 9:35pm UTC](https://discuss.elastic.co/t/querying-elasticsearch-2-3-0-causes-node-to-exit-cluster/48841 "2016-04-29T21:35:16Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![AvantGardeDreams](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@AvantGardeDreams](https://discuss.elastic.co/u/AvantGardeDreams)\
**Post date:** [April 29, 2016, 9:35pm UTC](https://discuss.elastic.co/t/querying-elasticsearch-2-3-0-causes-node-to-exit-cluster/48841/1 "2016-04-29T21:35:16Z")

</div>

I have a five node cluster recently upgraded from ES 1.7 to ES 2.3.0.

Each node has 13 gig of memory dedicated to ES. I have observed that making a large search query will cause a node to exit the cluster. Error logs make reference to zen ping being unable to reach the missing node. Restarting the missing node will allow it to rejoin the cluster, but it subsequently hangs on re-sharding.

Any help in troubleshooting and understanding this issue would be greatly appreciated.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 30, 2016, 1:32am UTC](https://discuss.elastic.co/t/querying-elasticsearch-2-3-0-causes-node-to-exit-cluster/48841/2 "2016-04-30T01:32:58Z")

</div>

Did you check the logs on the node that disconnected?

---

<div class="post-metadata">

**Author:** ![AvantGardeDreams](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@AvantGardeDreams](https://discuss.elastic.co/u/AvantGardeDreams)\
**Post date:** [May 2, 2016, 12:28pm UTC](https://discuss.elastic.co/t/querying-elasticsearch-2-3-0-causes-node-to-exit-cluster/48841/3 "2016-05-02T12:28:47Z")

</div>

Yes, the following entries appear in the log:

[2016-05-02 08:17:30,447][WARN][index.translog] [Node1] [aggblip][3] unexpected error while checking whether the translog needs a flush. rescheduling  
java.lang.OutOfMemoryError: Java heap space  
[2016-05-02 08:17:30,447][DEBUG][action.search] [Node1] [120] Failed to execute fetch phase  
RemoteTransportException[[Failed to deserialize response of type [org.elasticsearch.search.fetch.FetchSearchResult]]]; nested: TransportSerializationException[Failed to deserialize response of type [org.elasticsearch.search.fetch.FetchSearchResult]]; nested: OutOfMemoryError[Java heap space];  
Caused by: TransportSerializationException[Failed to deserialize response of type [org.elasticsearch.search.fetch.FetchSearchResult]]; nested: OutOfMemoryError[Java heap space];

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 2, 2016, 10:05pm UTC](https://discuss.elastic.co/t/querying-elasticsearch-2-3-0-causes-node-to-exit-cluster/48841/4 "2016-05-02T22:05:45Z")

</div>

> [@AvantGardeDreams](#):
>
> OutOfMemoryError[Java heap space];

That's why.  
What is the query, what are your node specs, what is the config (heap etc).

---

<div class="post-metadata">

**Author:** ![AvantGardeDreams](https://avatars.discourse-cdn.com/v4/letter/a/d07c76/32.png) [@AvantGardeDreams](https://discuss.elastic.co/u/AvantGardeDreams)\
**Post date:** [May 16, 2016, 12:35pm UTC](https://discuss.elastic.co/t/querying-elasticsearch-2-3-0-causes-node-to-exit-cluster/48841/5 "2016-05-16T12:35:29Z")

</div>

Sorry for the late reply.

The heap space was the issue. Elasticsearch had been started with the following arguments:

./elasticsearch -d --Xms=20g --Xmx=20g

The Xms/Xmx args were not utilized when the JVM started, so it used the system defaults - 256M/1g

I set the ES\_HEAP\_SIZE environment variable to the desired value and the issue has been resolved. Thanks for the input.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:51pm UTC](https://discuss.elastic.co/t/querying-elasticsearch-2-3-0-causes-node-to-exit-cluster/48841/6 "2017-07-05T22:51:22Z")

</div>