I'm facing an issue I cant dig down. I have setup filebeat to ship logs from different hosts to logstash. I found that some logs were missing and I found an out of memory error in the logs:
[2020-07-17T09:37:03,328][INFO ][org.logstash.beats.BeatsHandler][main][1a7b6b60de606e72fe759d281b5a4fa727bde4c285f36adad433cb80083db7d8] [local: 10.8.1.1:5044, remote: 10.4.5.1:47792] Handling exception: Direct buffer memory [2020-07-17T09:37:03,330][WARN ][io.netty.channel.DefaultChannelPipeline][main][1a7b6b60de606e72fe759d281b5a4fa727bde4c285f36adad433cb80083db7d8] An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception. java.lang.OutOfMemoryError: Direct buffer memory at java.nio.Bits.reserveMemory(Bits.java:175) ~[?:?]
This happens only on one specifc host (which does not have a higher log volume than the others). I also increased heap from one to four GB of heap, but no change.
How to debug this?
Thanks for any help and suggestions.