# How to reduce elasticsearch client node memory usage

**URL:** <https://discuss.elastic.co/t/how-to-reduce-elasticsearch-client-node-memory-usage/208004>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-alerting\
**Created:** [November 15, 2019, 7:39am UTC](https://discuss.elastic.co/t/how-to-reduce-elasticsearch-client-node-memory-usage/208004 "2019-11-15T07:39:19Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![lovelly](https://avatars.discourse-cdn.com/v4/letter/l/4af34b/32.png) [@lovelly](https://discuss.elastic.co/u/lovelly)\
**Post date:** [November 15, 2019, 7:39am UTC](https://discuss.elastic.co/t/how-to-reduce-elasticsearch-client-node-memory-usage/208004/1 "2019-11-15T07:39:19Z")

</div>

I have a client node.  
Node.master: false  
Node.data: false

I allocated memory to jvm:  
-Xms10g  
-Xmx10g

I use the bulk api to insert data for a long time.  
The elasticsearch client node runs for about 10 minutes, and the memory usage becomes 100%. It does not decrease.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/7/f70e9ea24b260e94d7f9c8ffe85c4ff78a698002.png)  
After a while, the clinet node crashes.

```auto
java.lang.OutOfMemoryError: Java heap space
Dumping heap to data/java_pid3851.hprof ...
Heap dump file created [11703693630 bytes in 97.763 secs]
[2019-11-15T16:34:56,625][ERROR][o.e.ExceptionsHelper] [es] fatal error
	at org.elasticsearch.ExceptionsHelper.lambda$maybeDieOnAnotherThread$4(ExceptionsHelper.java:300)
	at java.base/java.util.Optional.ifPresent(Optional.java:176)
	at org.elasticsearch.ExceptionsHelper.maybeDieOnAnotherThread(ExceptionsHelper.java:290)
	at org.elasticsearch.transport.netty4.Netty4TcpChannel.lambda$addPromise$1(Netty4TcpChannel.java:88)
	at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:500)
	at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:474)
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:413)
	at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:538)
	at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:531)

```

**Plugins installed** : [none]

**JVM version** (`java -version`):  
JVM: 13.0.1 ES: 7.4.2

**OS version** (`uname -a` if on a Unix-like system):  
Linux es 3.10.0-1062.4.1.el7.x86\_64 #1 SMP Fri Oct 18 17:15:30 UTC 2019 x86\_64 x86\_64 x86\_64 GNU/Linux

elasticsearch log:

```auto
[2019-11-15T16:20:33,317][WARN][o.e.m.j.JvmGcMonitorService] [es] [gc][2331] overhead, spent [1.4s] collecting in the last [2.3s]
[2019-11-15T16:20:40,712][WARN][o.e.m.j.JvmGcMonitorService] [es] [gc][2337] overhead, spent [1.7s] collecting in the last [2.3s]
[2019-11-15T16:20:48,607][WARN][o.e.m.j.JvmGcMonitorService] [es] [gc][2344] overhead, spent [1.5s] collecting in the last [1.8s]
[2019-11-15T16:20:55,414][WARN][o.e.m.j.JvmGcMonitorService] [es] [gc][2350] overhead, spent [1.6s] collecting in the last [1.8s]
[2019-11-15T16:21:04,547][WARN][o.e.m.j.JvmGcMonitorService] [es] [gc][2358] overhead, spent [1.7s] collecting in the last [2.1s]
[2019-11-15T16:21:10,891][WARN][o.e.m.j.JvmGcMonitorService] [es] [gc][2363] overhead, spent [1.6s] collecting in the last [2.3s]
[2019-11-15T16:21:37,683][WARN][o.e.m.j.JvmGcMonitorService] [es] [gc][2389] overhead, spent [1.6s] collecting in the last [1.6s]
[2019-11-15T16:21:48,849][WARN][o.e.m.j.JvmGcMonitorService] [es] [gc][2399] overhead, spent [1.5s] collecting in the last [2.1s]
[2019-11-15T16:22:00,279][WARN][o.e.m.j.JvmGcMonitorService] [es] [gc][2410] overhead, spent [1.3s] collecting in the last [1.3s]

```

Elasticseach go sdk client A lot of the following errors:

```auto
error:map[bytes_limit:1.0200547328e+10 bytes_wanted:1.0648510022e+10 durability:TRANSIENT
reason:[parent] Data too large, data for [<http_request>] would be [10648510022/9.9gb], which is 
larger than the limit of [10200547328/9.5gb], real usage: [10638953944/9.9gb], new bytes 
reserved: [9556078/9.1mb], usages [request=6040780800/5.6gb, fielddata=0/0b, 
in_flight_requests=212138460/202.3mb, accounting=0/0b] root_cause:
[map[bytes_limit:1.0200547328e+10 bytes_wanted:1.0648510022e+10 durability:TRANSIENT 
reason:[parent] Data too large, data for [<http_request>] would be [10648510022/9.9gb], which is
larger than the limit of [10200547328/9.5gb], real usage: [10638953944/9.9gb], new bytes 
reserved: [9556078/9.1mb], usages [request=6040780800/5.6gb, fielddata=0/0b, 
in_flight_requests=212138460/202.3mb, accounting=0/0b] type:circuit_breaking_exception]] 
type:circuit_breaking_exception] status:429

```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [November 15, 2019, 7:50am UTC](https://discuss.elastic.co/t/how-to-reduce-elasticsearch-client-node-memory-usage/208004/2 "2019-11-15T07:50:08Z")

</div>

I would recommend sending smaller bulk requests. A common recommendation is that each bulk request should not exceed around 5MB in size.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 13, 2019, 7:50am UTC](https://discuss.elastic.co/t/how-to-reduce-elasticsearch-client-node-memory-usage/208004/3 "2019-12-13T07:50:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.