# ElasticSearch crashes with 5.3.1 Client

**URL:** <https://discuss.elastic.co/t/elasticsearch-crashes-with-5-3-1-client/86008>\
**Category:** Elasticsearch\
**Created:** [May 16, 2017, 8:45pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-5-3-1-client/86008 "2017-05-16T20:45:40Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ninad\_Pradhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ninad_pradhan/32/33548_2.png) [@Ninad\_Pradhan](https://discuss.elastic.co/u/Ninad_Pradhan)\
**Post date:** [May 16, 2017, 8:45pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-5-3-1-client/86008/1 "2017-05-16T20:45:40Z")

</div>

Im hitting the issue which is being discussed here. Anyone know how to fix it?

> <https://github.com/elastic/elasticsearch/issues/22406>

Ive tried the above options with es 5.3.1 ... Im still getting the same issue, when I run our test cases 6 times, every 6th time I get the same error, its a single node cluster with the below settings...  
[cluster.name](http://cluster.name): test  
[node.name](http://node.name): test-0  
node.master: true  
node.data: true  
path.data: /tmp/elasticsearch  
path.logs: /tmp/eslogs

If I increase the java heap size it runs for a few more times and still crashes eventually...

12:49:31.022 INFO [PluginsService]: no modules loaded  
12:49:31.025 INFO [PluginsService]: loaded plugin [org.elasticsearch.index.reindex.ReindexPlugin]  
12:49:31.025 INFO [PluginsService]: loaded plugin [org.elasticsearch.percolator.PercolatorPlugin]  
12:49:31.025 INFO [PluginsService]: loaded plugin [org.elasticsearch.script.mustache.MustachePlugin]  
12:49:31.025 INFO [PluginsService]: loaded plugin [org.elasticsearch.transport.Netty3Plugin]  
12:49:31.025 INFO [PluginsService]: loaded plugin [org.elasticsearch.transport.Netty4Plugin]  
12:49:34.874 ERROR [Netty4Utils]: fatal error on the network layer  
at org.elasticsearch.transport.netty4.Netty4Utils.maybeDie(Netty4Utils.java:140)  
at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:344)  
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:242)  
at org.elasticsearch.client.transport.TransportProxyClient.execute(TransportProxyClient.java:59)  
at org.elasticsearch.client.transport.TransportClient.doExecute(Transp

---

<div class="post-metadata">

**Author:** ![jovanmal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jovanmal/32/17330_2.png) [@jovanmal](https://discuss.elastic.co/u/jovanmal)\
**Post date:** [May 16, 2017, 8:51pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-5-3-1-client/86008/2 "2017-05-16T20:51:25Z")

</div>

Upgrade to 5.4?

---

<div class="post-metadata">

**Author:** ![Ninad\_Pradhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ninad_pradhan/32/33548_2.png) [@Ninad\_Pradhan](https://discuss.elastic.co/u/Ninad_Pradhan)\
**Post date:** [May 16, 2017, 9:22pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-5-3-1-client/86008/3 "2017-05-16T21:22:39Z")

</div>

Have you seen anything in particular in 5.4 that fixes this issue, 5.4 requires a code changes for histogram builder, hence Im holding off on it @jovanmal

---

<div class="post-metadata">

**Author:** ![jovanmal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jovanmal/32/17330_2.png) [@jovanmal](https://discuss.elastic.co/u/jovanmal)\
**Post date:** [May 16, 2017, 10:34pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-5-3-1-client/86008/4 "2017-05-16T22:34:36Z")

</div>

No, just an idea to make a change that could resolve this issue.

You can setup a new test environment to try this.

I would have started dealing with the problem on this way, when I was in your position

---

<div class="post-metadata">

**Author:** ![Ninad\_Pradhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ninad_pradhan/32/33548_2.png) [@Ninad\_Pradhan](https://discuss.elastic.co/u/Ninad_Pradhan)\
**Post date:** [May 16, 2017, 10:49pm UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-5-3-1-client/86008/5 "2017-05-16T22:49:56Z")

</div>

Took that advice tried 5.4.0 no joy, same exact issue.

---

<div class="post-metadata">

**Author:** ![jovanmal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jovanmal/32/17330_2.png) [@jovanmal](https://discuss.elastic.co/u/jovanmal)\
**Post date:** [May 17, 2017, 12:12am UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-5-3-1-client/86008/6 "2017-05-17T00:12:29Z")

</div>

hmm, network layer... TCP/IP protocol...

Are there configured more than one ip addresses on your server? Or just one?

---

<div class="post-metadata">

**Author:** ![Ninad\_Pradhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ninad_pradhan/32/33548_2.png) [@Ninad\_Pradhan](https://discuss.elastic.co/u/Ninad_Pradhan)\
**Post date:** [May 17, 2017, 12:50am UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-5-3-1-client/86008/7 "2017-05-17T00:50:33Z")

</div>

> [@jovanmal](#):
>
> ne

just one

---

<div class="post-metadata">

**Author:** ![Ninad\_Pradhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ninad_pradhan/32/33548_2.png) [@Ninad\_Pradhan](https://discuss.elastic.co/u/Ninad_Pradhan)\
**Post date:** [May 19, 2017, 5:06am UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-5-3-1-client/86008/8 "2017-05-19T05:06:59Z")

</div>

Is there anyone else facing this issue. posting the error again....

1:59:05.817 ERROR [Netty4Utils]: fatal error on the network layer  
at org.elasticsearch.transport.netty4.Netty4Utils.maybeDie(Netty4Utils.java:140)  
at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.exceptionCaught(Netty4MessageChannelHandler.java:83)  
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:286)  
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:265)  
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:257)  
at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:131)  
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:286)  
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:265)  
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:257)  
at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1301)  
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:286)  
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:265)  
at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:914)  
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:99)  
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:140)  
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:642)  
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:527)  
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:481)  
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:441)  
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)  
at java.lang.Thread.run(Thread.java:745)  
Exception in thread "Thread-22" java.lang.OutOfMemoryError: Java heap space  
at io.netty.buffer.PoolArena$HeapArena.newChunk(PoolArena.java:656)  
at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:237)  
at io.netty.buffer.PoolArena.allocate(PoolArena.java:221)  
at io.netty.buffer.PoolArena.allocate(PoolArena.java:141)  
at io.netty.buffer.PooledByteBufAllocator.newHeapBuffer(PooledByteBufAllocator.java:272)  
at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:160)  
at io.netty.buffer.AbstractByteBufAllocator.heapBuffer(AbstractByteBufAllocator.java:151)  
at io.netty.buffer.AbstractByteBufAllocator.ioBuffer(AbstractByteBufAllocator.java:133)  
at io.netty.channel.DefaultMaxMessagesRecvByteBufAllocator$MaxMessageHandle.allocate(DefaultMaxMessagesRecvByteBufAllocator.java:73)  
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:117)  
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:642)  
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:527)  
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:481)  
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:441)  
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)  
at java.lang.Thread.run(Thread.java:745)  
^C^C^CException in thread "elasticsearch[_client_][generic][T#2]" java.lang.OutOfMemoryError: GC overhead limit exceeded  
Exception in thread "elasticsearch[_client_][generic][T#1]" java.lang.OutOfMemoryError: GC overhead limit exceeded

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 16, 2017, 5:07am UTC](https://discuss.elastic.co/t/elasticsearch-crashes-with-5-3-1-client/86008/9 "2017-06-16T05:07:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
