java.lang.OutOfMemoryError: Java heap space on 5.1.2

tramchamploo · January 17, 2017, 8:46am

I formerly use 5.1.1 and encountered to this OOM, so I update to 5.1.2 it still happens.

magnusbaeck · January 17, 2017, 9:02am

That should not be surprising as OOM problems are rarely fixed by minor upgrades. Sounds like you might have to increase the heap or decrease the amount of data. If you supply additional details about your setup we can say more.

jasontedor · January 17, 2017, 10:37am

I agree in general. In this case there is one major stability change incorporated into 5.1.2 that should address some of the OOMs that we have been seeing.

Yes, this. A stack trace of where the OOM is coming from would be helpful to know whether or not what you're seeing should have been addressed by the aforementioned change.

tramchamploo · January 17, 2017, 10:38am

2017-01-17T16:27:29,548][ERROR][o.e.t.n.Netty4Utils      ] fatal error on the network layer
        at org.elasticsearch.transport.netty4.Netty4Utils.maybeDie(Netty4Utils.java:140)
        at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.exceptionCaught(Netty4HttpRequestHandler.java:76)
        at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:296)
        at io.netty.channel.AbstractChannelHandlerContext.notifyHandlerException(AbstractChannelHandlerContext.java:861)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:375)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:66)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
        at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)

[2017-01-17T16:27:25,886][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-172.16.105.125] fatal error in thread [Thread-117], exiting
java.lang.OutOfMemoryError: Java heap space

jasontedor · January 17, 2017, 10:51am

Please show the stacktrace under what you just pasted. Namely the stacktrace that is under:

[2017-01-17T16:27:25,886][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-172.16.105.125] fatal error in thread [Thread-117], exiting
java.lang.OutOfMemoryError: Java heap space

tramchamploo · January 18, 2017, 3:00am

There's nothing under, and ES process went down.

jasontedor · January 18, 2017, 3:42am

That the node went down is not unexpected, but there should be a stack trace there (we print it first before shutting the node down).

tramchamploo · January 18, 2017, 7:06am

I'm sure there's nothing below.

jasontedor · January 18, 2017, 8:09am

Then there is no information to be gleaned here.

tramchamploo · January 18, 2017, 10:03am

But version 2 doesn't have this problem with the same amount of data.

system · February 15, 2017, 10:03am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.