Accidentally sending a HTTP request to TCP port during recovery results in OutOfMemoryError in Netty

Michael_Snell · May 10, 2012, 12:14pm

Hi - Accidentally sending a HTTP request to the TCP port (eg curl
'http://localhost:9300/') results in an OutOfMemoryError being logged. This
appears to be reproducible in 0.19.3 during recovery - the stacktrace
logged is:

52,258][WARN ][transport.netty ] [nodename] Exception caught on
netty layer [[id: 0x00c1df71, /127.0.0.1:50268 => /127.0.0.1:9300]]
java.lang.OutOfMemoryError: Java heap space
at
org.elasticsearch.common.netty.buffer.HeapChannelBuffer.(HeapChannelBuffer.java:42)
at
org.elasticsearch.common.netty.buffer.BigEndianHeapChannelBuffer.(BigEndianHeapChannelBuffer.java:34)
at
org.elasticsearch.common.netty.buffer.ChannelBuffers.buffer(ChannelBuffers.java:134)
at
org.elasticsearch.common.netty.buffer.HeapChannelBufferFactory.getBuffer(HeapChannelBufferFactory.java:69)
at
org.elasticsearch.common.netty.buffer.DynamicChannelBuffer.(DynamicChannelBuffer.java:58)
at
org.elasticsearch.common.netty.buffer.ChannelBuffers.dynamicBuffer(ChannelBuffers.java:221)
at
org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:98)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:792)
at
org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.lang.Thread.run(Unknown Source)

kimchy · May 10, 2012, 8:36pm

Yea, I've noticed it as well, requites some hardening on the internal
transport layer. Open an issue so we won't miss it?

On Thu, May 10, 2012 at 3:14 PM, Michael Snell michael@snell.com wrote:

Hi - Accidentally sending a HTTP request to the TCP port (eg curl '
http://localhost:9300/') results in an OutOfMemoryError being logged.
This appears to be reproducible in 0.19.3 during recovery - the stacktrace
logged is:

52,258][WARN ][transport.netty ] [nodename] Exception caught on
netty layer [[id: 0x00c1df71, /127.0.0.1:50268 => /127.0.0.1:9300]]
java.lang.OutOfMemoryError: Java heap space
at
org.elasticsearch.common.netty.buffer.HeapChannelBuffer.(HeapChannelBuffer.java:42)
at
org.elasticsearch.common.netty.buffer.BigEndianHeapChannelBuffer.(BigEndianHeapChannelBuffer.java:34)
at
org.elasticsearch.common.netty.buffer.ChannelBuffers.buffer(ChannelBuffers.java:134)
at
org.elasticsearch.common.netty.buffer.HeapChannelBufferFactory.getBuffer(HeapChannelBufferFactory.java:69)
at
org.elasticsearch.common.netty.buffer.DynamicChannelBuffer.(DynamicChannelBuffer.java:58)
at
org.elasticsearch.common.netty.buffer.ChannelBuffers.dynamicBuffer(ChannelBuffers.java:221)
at
org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:98)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:792)
at
org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.lang.Thread.run(Unknown Source)

Michael_Snell · May 16, 2012, 9:00am

Opened Sending a HTTP request to TCP port during recovery results in OutOfMemoryError in Netty · Issue #1955 · elastic/elasticsearch · GitHub

On Thursday, 10 May 2012 21:36:27 UTC+1, kimchy wrote:

Yea, I've noticed it as well, requites some hardening on the internal
transport layer. Open an issue so we won't miss it?

On Thu, May 10, 2012 at 3:14 PM, Michael Snell michael@snell.com wrote:

Hi - Accidentally sending a HTTP request to the TCP port (eg curl '
http://localhost:9300/') results in an OutOfMemoryError being logged.
This appears to be reproducible in 0.19.3 during recovery - the stacktrace
logged is:

52,258][WARN ][transport.netty ] [nodename] Exception caught on
netty layer [[id: 0x00c1df71, /127.0.0.1:50268 => /127.0.0.1:9300]]
java.lang.OutOfMemoryError: Java heap space
at
org.elasticsearch.common.netty.buffer.HeapChannelBuffer.(HeapChannelBuffer.java:42)
at
org.elasticsearch.common.netty.buffer.BigEndianHeapChannelBuffer.(BigEndianHeapChannelBuffer.java:34)
at
org.elasticsearch.common.netty.buffer.ChannelBuffers.buffer(ChannelBuffers.java:134)
at
org.elasticsearch.common.netty.buffer.HeapChannelBufferFactory.getBuffer(HeapChannelBufferFactory.java:69)
at
org.elasticsearch.common.netty.buffer.DynamicChannelBuffer.(DynamicChannelBuffer.java:58)
at
org.elasticsearch.common.netty.buffer.ChannelBuffers.dynamicBuffer(ChannelBuffers.java:221)
at
org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:98)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:75)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:792)
at
org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:74)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.lang.Thread.run(Unknown Source)

Topic		Replies	Views
What is the work-around before this is fixed....Sending a HTTP request to TCP port during recovery results in OutOfMemoryError in Netty Issue#1955 Elasticsearch	2	271	July 6, 2017
Out of memory/heap Elasticsearch	3	963	July 6, 2017
Exception in netty while indexing, :( Elasticsearch	4	1215	July 6, 2017
Restarting ES Service frequently for Memory Issue Elasticsearch	3	481	July 6, 2017
Java heap space issues Elasticsearch elastic-stack-monitoring , elastic-stack-alerting	2	809	April 5, 2021

Accidentally sending a HTTP request to TCP port during recovery results in OutOfMemoryError in Netty

Related topics