java.io.IOException: An established connection was aborted by the software in your host machine

chelawatankit · July 26, 2020, 2:16pm

Hi Team,

We are using single node cluster of elasticsearch 7.1.1 version on one of our machine. System details as follow
OS : windows server 2012 R2
Xmx : 6GB
System RAM : 64 GB

We are getting following error frequently.

[2020-07-24T03:08:09,644][WARN ][o.e.t.OutboundHandler    ] [cinode_win-sab371f96sl] send message failed [channel: Netty4TcpChannel{localAddress=0.0.0.0/0.0.0.0:9300, remoteAddress=/127.0.0.1:51958}]
java.io.IOException: An established connection was aborted by the software in your host machine
	at sun.nio.ch.SocketDispatcher.write0(Native Method) ~[?:?]
	at sun.nio.ch.SocketDispatcher.write(Unknown Source) ~[?:?]
	at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source) ~[?:?]
	at sun.nio.ch.IOUtil.write(Unknown Source) ~[?:?]
	at sun.nio.ch.SocketChannelImpl.write(Unknown Source) ~[?:?]
	at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:405) ~[?:?]
	at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:938) ~[?:?]
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:360) ~[?:?]
	at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:905) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1396) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749) ~[?:?]
	at io.netty.handler.logging.LoggingHandler.flush(LoggingHandler.java:265) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:768) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:749) ~[?:?]
	at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:776) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:802) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:814) ~[?:?]
	at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:794) ~[?:?]
	at io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:1066) ~[?:?]
	at io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:309) ~[?:?]
	at org.elasticsearch.transport.netty4.Netty4TcpChannel.sendMessage(Netty4TcpChannel.java:139) ~[?:?]
	at org.elasticsearch.transport.OutboundHandler.internalSendMessage(OutboundHandler.java:80) ~[elasticsearch-7.1.1.jar:7.1.1]
	at org.elasticsearch.transport.OutboundHandler.sendBytes(OutboundHandler.java:59) ~[elasticsearch-7.1.1.jar:7.1.1]
	at org.elasticsearch.transport.TransportKeepAlive.sendPing(TransportKeepAlive.java:118) [elasticsearch-7.1.1.jar:7.1.1]
	at org.elasticsearch.transport.TransportKeepAlive.receiveKeepAlive(TransportKeepAlive.java:105) [elasticsearch-7.1.1.jar:7.1.1]
	at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:755) [elasticsearch-7.1.1.jar:7.1.1]
	at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:53) [transport-netty4-client-7.1.1.jar:7.1.1]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) [netty-codec-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297) [netty-codec-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:241) [netty-handler-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:656) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:556) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:510) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:470) [netty-transport-4.1.32.Final.jar:4.1.32.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:909) [netty-common-4.1.32.Final.jar:4.1.32.Final]
	at java.lang.Thread.run(Unknown Source) [?:1.8.0_201].

We are using transport client to connect with elasticsearch server. and both client and server are running on same machine.

Can someone guide me to identify the problem. What is the possible cause of above issue.

Thanks
Ankit Chelawat

chelawatankit · July 27, 2020, 1:59pm

Hi,

Any suggestion or direction to identify the issue.

Thanks

chelawatankit · July 28, 2020, 10:08am

One observation to add that we saw this error message so many times and most of the time no new document was ingesting or updating.

How I can solve this problem.

Thanks in advance.

chelawatankit · July 30, 2020, 7:49am

Hi,

Please suggest any direction to solve this problem. it is impacting us on one of our system.

spinscale · August 3, 2020, 10:35am

can you share the code you are using to initialize? Also, do you have TLS enabled and if so, on client and server side?

system · August 31, 2020, 10:35am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.