# Elasticsearch 6.8.23 happen OOM **URL:** \ **Category:** Elasticsearch\ **Created:** [April 26, 2023, 8:16am UTC](https://discuss.elastic.co/t/elasticsearch-6-8-23-happen-oom/330802 "2023-04-26T08:16:02Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![yunpeng.jiangyp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yunpeng.jiangyp/32/120230_2.png) [@yunpeng.jiangyp](https://discuss.elastic.co/u/yunpeng.jiangyp)\ **Post date:** [April 26, 2023, 8:16am UTC](https://discuss.elastic.co/t/elasticsearch-6-8-23-happen-oom/330802/1 "2023-04-26T08:16:02Z")
Hi, We have a 8core/16GB ( 4 nodes cluster ) for the Elasticsearch and the Elasticsearch process is getting killed. JDK settings ```auto -Xms8g -Xmx8g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false ``` Elasticsearch settings: ```auto node.master: true node.data: true thread_pool.bulk.queue_size: 50000 thread_pool.search.size: 200 ``` There are error logs: ```auto [2023-04-24T18:59:16,595][WARN][o.e.m.j.JvmGcMonitorService] [web-49-102-hzifc.node.hzifc.wacai.sdc] [gc][2095859] overhead, spent [14.3s] collecting in the last [15.1s] [2023-04-24T18:59:33,643][WARN][o.e.m.j.JvmGcMonitorService] [web-49-102-hzifc.node.hzifc.wacai.sdc] [gc][old][2095860][2952] duration [16.8s], collections [1]/[17s], total [16.8s]/[22.2m], memory [7.6gb]->[7.7gb]/[7.9gb], all_pools {[young] [353.9mb]->[420.5mb]/[532.5mb]}{[survivor] [0b]->[0b]/[66.5mb]}{[old] [7.3gb]->[7.3gb]/[7.3gb]} [2023-04-24T19:01:24,919][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [web-49-102-hzifc.node.hzifc.wacai.sdc] fatal error in thread [Thread-1136062], exiting java.lang.OutOfMemoryError: Java heap space at io.netty.buffer.UnpooledHeapByteBuf.allocateArray(UnpooledHeapByteBuf.java:88) ~[?:?] at io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledHeapByteBuf.allocateArray(UnpooledByteBufAllocator.java:164) ~[?:?] at io.netty.buffer.UnpooledHeapByteBuf.(UnpooledHeapByteBuf.java:61) ~[?:?] at io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledHeapByteBuf.(UnpooledByteBufAllocator.java:159) ~[?:?] at io.netty.buffer.UnpooledByteBufAllocator.newHeapBuffer(UnpooledByteBufAllocator.java:82) ~[?:?] at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:166) ~[?:?] at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:157) ~[?:?] at io.netty.buffer.Unpooled.buffer(Unpooled.java:116) ~[?:?] at io.netty.buffer.Unpooled.copiedBuffer(Unpooled.java:409) ~[?:?] at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:73) ~[?:?] at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?] at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?] at org.elasticsearch.http.netty4.cors.Netty4CorsHandler.channelRead(Netty4CorsHandler.java:86) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) ~[?:?] at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[?:?] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) ~[?:?] ``` pass by heap file , we find there is a lot of memory footprint by netty's io.netty.buffer.PoolChunk object ![image-20230426102358873](https://us1.discourse-cdn.com/elastic/original/3X/4/3/4323c1dd423c2a681d7b5cc6960f49dface7c596.png) ![1](https://us1.discourse-cdn.com/elastic/original/3X/6/e/6ee5af43b26bbf4c35d51e10613baa79a50aa77a.jpeg) ![2](https://us1.discourse-cdn.com/elastic/original/3X/1/f/1fa1b9c0b952775bfd3fe3cdc5df1be9a455554d.jpeg) Can you advice whether any other settings to be made to avoid the OOM killer to crash the elasticsearch process. Thanks ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 26, 2023, 8:16am UTC](https://discuss.elastic.co/t/elasticsearch-6-8-23-happen-oom/330802/2 "2023-04-26T08:16:02Z")
Elasticsearch 6.8.23 is [EOL](https://www.elastic.co/support/eol) and no longer supported. Please upgrade ASAP. (This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: ) ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [April 26, 2023, 8:51am UTC](https://discuss.elastic.co/t/elasticsearch-6-8-23-happen-oom/330802/3 "2023-04-26T08:51:08Z")
> [@yunpeng.jiangyp](#): > > `thread_pool.bulk.queue_size: 50000` What is the point in potentially queueing up a huge amount of documents to index in memory if the cluster can not keep up? This will drive heap usage. I would recommend setting this back to default and let the clients handle backpressure. You may also want to look into how you are sharding and indexing data to make it more efficient. ---
**Author:** ![yunpeng.jiangyp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yunpeng.jiangyp/32/120230_2.png) [@yunpeng.jiangyp](https://discuss.elastic.co/u/yunpeng.jiangyp)\ **Post date:** [April 26, 2023, 11:37am UTC](https://discuss.elastic.co/t/elasticsearch-6-8-23-happen-oom/330802/4 "2023-04-26T11:37:05Z")
Thanks Christian . good suggestion , i will remove it ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 24, 2023, 11:37am UTC](https://discuss.elastic.co/t/elasticsearch-6-8-23-happen-oom/330802/5 "2023-05-24T11:37:08Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.