# Elasticsearch Over Heap Any times

**URL:** <https://discuss.elastic.co/t/elasticsearch-over-heap-any-times/124812>\
**Category:** Elasticsearch\
**Created:** [March 20, 2018, 3:58pm UTC](https://discuss.elastic.co/t/elasticsearch-over-heap-any-times/124812 "2018-03-20T15:58:10Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ben96](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ben96/32/28334_2.png) [@Ben96](https://discuss.elastic.co/u/Ben96)\
**Post date:** [March 20, 2018, 3:58pm UTC](https://discuss.elastic.co/t/elasticsearch-over-heap-any-times/124812/1 "2018-03-20T15:58:10Z")

</div>

Hi guys,

Here we are for a new topic again 😛

I'm in trouble with mu Elastic Cluster :

Got on 2 nodes in particular, heap usage in red at all times. (Around 19.7 min & 19.8 Max)

We jus lost one node few minutes ago, and 2 mores few days ago too.

HEAP : XMS/XMX 20G / Node

128G Set on Each Node ( Only 20 for ES)

i don't realy what is the real cause of this..

Some logs here :

```
> in the last [17.3s]
> [2018-03-19T11:55:29,329][WARN][o.e.m.j.JvmGcMonitorService] XXX] [gc][old][657803][29914] duration [18.1s], collections [1]/[18.4s], total [18.1s]/[1h], memory [19.7gb]->[19.7gb]/[19.8gb], all_pools {[young] [1.4gb]->[1.4gb]/[1.4gb]}{[survivor] [138.3mb]->[152.9mb]/[191.3mb]}{[old] [18.1gb]->[18.1gb]/[18.1gb]}
> [2018-03-19T11:55:29,329][WARN][o.e.m.j.JvmGcMonitorService] XXX] [gc][657803] overhead, spent [18.1s] collecting in the last [18.4s]
> [2018-03-19T11:55:49,159][WARN][o.e.m.j.JvmGcMonitorService] [XXX [gc][old][657804][29915] duration [19.7s], collections [1]/[19.9s], total [19.7s]/[1h], memory [19.7gb]->[19.7gb]/[19.8gb], all_pools {[young] [1.4gb]->[1.4gb]/[1.4gb]}{[survivor] [152.9mb]->[159.7mb]/[191.3mb]}{[old] [18.1gb]->[18.1gb]/[18.1gb]}
> [2018-03-19T11:55:49,164][WARN][o.e.m.j.JvmGcMonitorService] [XXX] [gc][657804] overhead, spent [19.7s] collecting in the last [19.9s]
> [2018-03-19T12:46:38,130][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] XXX] fatal error in thread [I/O dispatcher 28], exiting
> java.lang.OutOfMemoryError: Java heap space
> [2018-03-19T12:46:21,040][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [xxx] fatal error in thread [elasticsearch[XXX][bulk][T#25]], exiting
> java.lang.OutOfMemoryError: Java heap space
> [2018-03-19T12:46:38,132][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [xxx] fatal error in thread [elasticsearch[XXX][bulk][T#24]], exiting
> java.lang.OutOfMemoryError: Java heap space
> [2018-03-19T12:46:52,659][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [xxx] fatal error in thread [elasticsearch[xxx][bulk][T#28]], exiting
> java.lang.OutOfMemoryError: Java heap space
> [2018-03-19T12:46:21,042][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [xxx] fatal error in thread [I/O dispatcher 31], exiting
> java.lang.OutOfMemoryError: Java heap space
> at java.util.HashMap$KeySet.iterator(Unknown Source) ~[?:1.8.0_73]
> at java.util.HashSet.iterator(Unknown Source) ~[?:1.8.0_73]
> at java.util.Collections$UnmodifiableCollection$1.<init>(Unknown Source) ~[?:1.8.0_73]
> at java.util.Collections$UnmodifiableCollection.iterator(Unknown Source) ~[?:1.8.0_73]
> at org.apache.http.impl.nio.reactor.BaseIOReactor.validate(BaseIOReactor.java:212) ~[?:?]
> at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280) ~[?:?]
> at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[?:?]
> at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) ~[?:?]
> at java.lang.Thread.run(Unknown Source) [?:1.8.0_73]

```

If any one get any advice or idea about this problem i'm open to suggestion.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 20, 2018, 4:06pm UTC](https://discuss.elastic.co/t/elasticsearch-over-heap-any-times/124812/2 "2018-03-20T16:06:49Z")

</div>

Which version of Elasticsearch are you using? What is the output of the [cluster stats API](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/cluster-stats.html)?

---

<div class="post-metadata">

**Author:** ![Ben96](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ben96/32/28334_2.png) [@Ben96](https://discuss.elastic.co/u/Ben96)\
**Post date:** [March 20, 2018, 4:25pm UTC](https://discuss.elastic.co/t/elasticsearch-over-heap-any-times/124812/3 "2018-03-20T16:25:16Z")

</div>

I'm running V5.5.1

Nothing special, cluster jsut backed green so all fine for now

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 17, 2018, 4:26pm UTC](https://discuss.elastic.co/t/elasticsearch-over-heap-any-times/124812/4 "2018-04-17T16:26:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.