We have 3 dedicated ingest nodes in a cluster. They are functioning and data is coming into ES, but they are logging constantly "send messsage failed". No idea why ... logs about every few seconds from each ingest node and each master. Data nodes don't log it and coordinator nodes don't log it. Just the ingest nodes.
Full log output is:
2019-09-06T16:52:57,089][WARN ][o.e.t.OutboundHandler ] [ingest.clustername] send message failed [channel: Netty4TcpChannel{localAddress=/10.10.10.5:9300, remoteAddress=/10.10.10.1:44628}]
java.lang.NullPointerException: null
at org.elasticsearch.common.io.stream.StreamOutput.writeString(StreamOutput.java:403) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.action.DocWriteResponse.writeTo(DocWriteResponse.java:283) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.action.update.UpdateResponse.writeTo(UpdateResponse.java:82) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.action.bulk.BulkItemResponse.writeTo(BulkItemResponse.java:510) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.action.bulk.BulkResponse.writeTo(BulkResponse.java:148) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.transport.OutboundMessage.writeMessage(OutboundMessage.java:70) ~[elasticsearch-7.3.1.jar:7.3.1]
at org.elasticsearch.transport.OutboundMessage.serialize(OutboundMessage.java:53) ~[elasticsearch-7.3.1.jar:7.3.1]