# High Heap utilization in Client Node

**URL:** <https://discuss.elastic.co/t/high-heap-utilization-in-client-node/111464>\
**Category:** Elasticsearch\
**Created:** [December 13, 2017, 12:27am UTC](https://discuss.elastic.co/t/high-heap-utilization-in-client-node/111464 "2017-12-13T00:27:30Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Manikanth\_Reddy](https://avatars.discourse-cdn.com/v4/letter/m/5fc32e/32.png) [@Manikanth\_Reddy](https://discuss.elastic.co/u/Manikanth_Reddy)\
**Post date:** [December 13, 2017, 12:27am UTC](https://discuss.elastic.co/t/high-heap-utilization-in-client-node/111464/1 "2017-12-13T00:27:30Z")

</div>

We are seeing high heap utilization in two of the client nodes. Its configured as 4GB RAM and its hitting around 98%. There is nothing showing in elasticsearch logs.  
ES LOG:

```
[2017-12-12T19:23:48,005][WARN][o.e.m.j.JvmGcMonitorService] [zlt10363.vci.att.com-01] [gc][old][6778036][135594] duration [17.8s], collections [1]/[18.3s], total [17.8s]/[5.4h], memory [3.9gb]->[3.6gb]/[3.9gb], all_pools {[young] [266.2mb]->[14.4mb]/[266.2mb]}{[survivor] [32.6mb]->[0b]/[33.2mb]}{[old] [3.6gb]->[3.6gb]/[3.6gb]}
[2017-12-12T19:23:48,005][WARN][o.e.m.j.JvmGcMonitorService] [zlt10363.vci.att.com-01] [gc][6778036] overhead, spent [17.8s] collecting in the last [18.3s]

```

But I can see something related to it in Kibana logs.

```
{"type":"log","@timestamp":"2017-12-12T23:25:59Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
{"type":"log","@timestamp":"2017-12-12T23:25:59Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2017-12-12T23:28:53Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
{"type":"log","@timestamp":"2017-12-12T23:28:53Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2017-12-12T23:30:59Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
{"type":"log","@timestamp":"2017-12-12T23:30:59Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2017-12-12T23:39:59Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
{"type":"log","@timestamp":"2017-12-12T23:39:59Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2017-12-12T23:42:58Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
{"type":"log","@timestamp":"2017-12-12T23:42:58Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2017-12-12T23:57:47Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
{"type":"log","@timestamp":"2017-12-12T23:57:47Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2017-12-13T00:03:30Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
{"type":"log","@timestamp":"2017-12-13T00:03:30Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2017-12-13T00:06:22Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
{"type":"log","@timestamp":"2017-12-13T00:06:22Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}

```

Could you please help me out in fixing this issue. Thanks.

---

<div class="post-metadata">

**Author:** ![mujtabahussain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mujtabahussain/32/17514_2.png) [@mujtabahussain](https://discuss.elastic.co/u/mujtabahussain)\
**Post date:** [December 13, 2017, 12:31am UTC](https://discuss.elastic.co/t/high-heap-utilization-in-client-node/111464/2 "2017-12-13T00:31:13Z")

</div>

This may be of some help [https://www.linuxatemyram.com/](https://www.linuxatemyram.com/) 🙂

---

<div class="post-metadata">

**Author:** ![Manikanth\_Reddy](https://avatars.discourse-cdn.com/v4/letter/m/5fc32e/32.png) [@Manikanth\_Reddy](https://discuss.elastic.co/u/Manikanth_Reddy)\
**Post date:** [December 13, 2017, 3:09pm UTC](https://discuss.elastic.co/t/high-heap-utilization-in-client-node/111464/3 "2017-12-13T15:09:40Z")

</div>

Thanks for sharing the link. But this is happening from last 2 days. Previously, it used to work fine. But now, its reaching 99% and I can see that elasticsearch plugin is red in kibana logs. I think its not normal as Elasticsearch plugin is becoming red often.

```
     {"type":"log","@timestamp":"2017-12-13T10:55:46Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
    {"type":"log","@timestamp":"2017-12-13T11:12:10Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
    {"type":"log","@timestamp":"2017-12-13T11:12:10Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
    {"type":"log","@timestamp":"2017-12-13T11:32:50Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
    {"type":"log","@timestamp":"2017-12-13T11:32:50Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
    {"type":"log","@timestamp":"2017-12-13T11:57:35Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
    {"type":"log","@timestamp":"2017-12-13T11:57:35Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
    {"type":"log","@timestamp":"2017-12-13T12:00:16Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
    {"type":"log","@timestamp":"2017-12-13T12:00:16Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
    {"type":"log","@timestamp":"2017-12-13T12:34:13Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
    {"type":"log","@timestamp":"2017-12-13T12:34:13Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
    {"type":"log","@timestamp":"2017-12-13T14:12:44Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
    {"type":"log","@timestamp":"2017-12-13T14:12:44Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
    {"type":"log","@timestamp":"2017-12-13T14:52:07Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
    {"type":"log","@timestamp":"2017-12-13T14:52:07Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
    {"type":"log","@timestamp":"2017-12-13T14:54:56Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
    {"type":"log","@timestamp":"2017-12-13T14:54:56Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
    {"type":"log","@timestamp":"2017-12-13T15:00:28Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
    {"type":"log","@timestamp":"2017-12-13T15:00:28Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
    {"type":"log","@timestamp":"2017-12-13T15:02:30Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
    {"type":"log","@timestamp":"2017-12-13T15:02:30Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
    {"type":"log","@timestamp":"2017-12-13T15:03:02Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
    {"type":"log","@timestamp":"2017-12-13T15:03:02Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
    {"type":"log","@timestamp":"2017-12-13T15:05:10Z","tags":["status","plugin:elasticsearch@5.2.1","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Request Timeout after 3000ms","prevState":"green","prevMsg":"Kibana index ready"}
    {"type":"log","@timestamp":"2017-12-13T15:05:10Z","tags":["status","ui settings","error"],"pid":20308,"state":"red","message":"Status changed from green to red - Elasticsearch plugin is red","prevState":"green","prevMsg":"Ready"}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 10, 2018, 3:09pm UTC](https://discuss.elastic.co/t/high-heap-utilization-in-client-node/111464/4 "2018-01-10T15:09:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.