# Memory leak in Logstash 6.0.0 **URL:** \ **Category:** Logstash\ **Created:** [February 16, 2018, 4:03am UTC](https://discuss.elastic.co/t/memory-leak-in-logstash-6-0-0/120095 "2018-02-16T04:03:44Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![vineet](https://avatars.discourse-cdn.com/v4/letter/v/8e8cbc/32.png) [@vineet](https://discuss.elastic.co/u/vineet)\ **Post date:** [February 16, 2018, 4:03am UTC](https://discuss.elastic.co/t/memory-leak-in-logstash-6-0-0/120095/1 "2018-02-16T04:03:44Z")
Hello, I am looking for help with Logstash 6.0.0. Winlogbeat is sending logs to Logstash 6.0.0 which is running on a 8 GB machine with 4 GB min/max heap size. Over a period of a few hours, Logstash uses up all the memory and dies. Till the time Logstash is running; it is processing all incoming logs in time. Pls assist ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 16, 2018, 4:09am UTC](https://discuss.elastic.co/t/memory-leak-in-logstash-6-0-0/120095/2 "2018-02-16T04:09:17Z")
Please post the logs and your config. ---
**Author:** ![vineet](https://avatars.discourse-cdn.com/v4/letter/v/8e8cbc/32.png) [@vineet](https://discuss.elastic.co/u/vineet)\ **Post date:** [February 16, 2018, 3:37pm UTC](https://discuss.elastic.co/t/memory-leak-in-logstash-6-0-0/120095/3 "2018-02-16T15:37:22Z")
Thank you for responding. The log file is 2.5 GB (after compression it is about 15 MB) and it mostly contains this message: [2018-02-15T23:59:28,078][WARN][io.netty.channel.AbstractChannelHandlerContext] An exception 'java.lang.NullPointerException' [enable DEBUG level for full stacktrace] was thrown by a user handler's exceptionCaught() method while handling the following exception: java.io.IOException: Connection reset by peer Pls advise how I can send the log file Here is the config file: input { beats { port =\> 5056 } } filter { if [level] == "Information" { drop {} } mutate { remove\_field =\> ["[event\_data][Binary]", "[user\_data][binaryData]", "[user\_data][binaryDataSize]"] } } output { elasticsearch { hosts =\> ["[http://xx.xx.xx.xx:9200](http://xx.xx.xx.xx:9200)"] index =\> "winlogbeat-6.0.0-%{+YYYY.MM.dd}" document\_type =\> "doc" user =\> "xxxxxxxx" password =\> "xxxxxxxx" manage\_template =\> false } } Here is another thing: I set heap size to 2GB, after which memory utilization of Logstash is at ~ 55% on a 8 GB machine for several hours now. There is no backlog in event processing. It there a ratio of Logstash heap size vs memory used? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 16, 2018, 11:52pm UTC](https://discuss.elastic.co/t/memory-leak-in-logstash-6-0-0/120095/4 "2018-02-16T23:52:52Z")
If there is a log showing an OOM or crash of the Logstash process, please show that. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [February 17, 2018, 5:35pm UTC](https://discuss.elastic.co/t/memory-leak-in-logstash-6-0-0/120095/5 "2018-02-17T17:35:36Z")
Do you have any non-default settings in your logstash.yml file? How are you securing the Elasticsearch cluster? ---
**Author:** ![vineet](https://avatars.discourse-cdn.com/v4/letter/v/8e8cbc/32.png) [@vineet](https://discuss.elastic.co/u/vineet)\ **Post date:** [February 17, 2018, 11:38pm UTC](https://discuss.elastic.co/t/memory-leak-in-logstash-6-0-0/120095/6 "2018-02-17T23:38:50Z")
The log file shows nothing related to OOM or crash of Logstash process. Logstash was running in console mode, and it had just a single line extra than the log file which said 'Killed': [2018-02-17T07:26:23,715][WARN][io.netty.channel.AbstractChannelHandlerContext] An exception 'java.lang.NullPointerException' [enable DEBUG level for full stacktrace] was thrown by a user handler's exceptionCaught() method while handling the following exception: java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[?:1.8.0\_151] at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[?:1.8.0\_151] at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) ~[?:1.8.0\_151] at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[?:1.8.0\_151] at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380) ~[?:1.8.0\_151] at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:288) ~[netty-all-4.1.3.Final.jar:4.1.3.Final] at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1100) ~[netty-all-4.1.3.Final.jar:4.1.3.Final] at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:349) ~[netty-all-4.1.3.Final.jar:4.1.3.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:112) ~[netty-all-4.1.3.Final.jar:4.1.3.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:571) ~[netty-all-4.1.3.Final.jar:4.1.3.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:512) ~[netty-all-4.1.3.Final.jar:4.1.3.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:426) ~[netty-all-4.1.3.Final.jar:4.1.3.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:398) [netty-all-4.1.3.Final.jar:4.1.3.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:877) [netty-all-4.1.3.Final.jar:4.1.3.Final] at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144) [netty-all-4.1.3.Final.jar:4.1.3.Final] at java.lang.Thread.run(Thread.java:748) [?:1.8.0\_151] Killed ---
**Author:** ![vineet](https://avatars.discourse-cdn.com/v4/letter/v/8e8cbc/32.png) [@vineet](https://discuss.elastic.co/u/vineet)\ **Post date:** [February 17, 2018, 11:39pm UTC](https://discuss.elastic.co/t/memory-leak-in-logstash-6-0-0/120095/7 "2018-02-17T23:39:41Z")
I have the following non-default setting in pipelines.yml: pipeline.workers: 12 ---
**Author:** ![vineet](https://avatars.discourse-cdn.com/v4/letter/v/8e8cbc/32.png) [@vineet](https://discuss.elastic.co/u/vineet)\ **Post date:** [February 18, 2018, 4:50am UTC](https://discuss.elastic.co/t/memory-leak-in-logstash-6-0-0/120095/8 "2018-02-18T04:50:37Z")
It is a 4 core machine. The only way ElasticSearch is secured is by x-pack ---
**Author:** ![vineet](https://avatars.discourse-cdn.com/v4/letter/v/8e8cbc/32.png) [@vineet](https://discuss.elastic.co/u/vineet)\ **Post date:** [February 18, 2018, 6:02pm UTC](https://discuss.elastic.co/t/memory-leak-in-logstash-6-0-0/120095/9 "2018-02-18T18:02:54Z")
I ran it again this time got the following message in the console: java.lang.OutOfMemoryError: Java heap space Dumping heap to java\_pid19661.hprof ... # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007f2cb328b0bb, pid=19661, tid=0x00007f2cb01b3700 # # JRE version: Java(TM) SE Runtime Environment (8.0\_151-b12) (build 1.8.0\_151-b12) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.151-b12 mixed mode linux-amd64 compressed oops) # Problematic frame: # V [libjvm.so+0x6960bb] java\_lang\_Class::signers(oopDesc\*)+0x1b # # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # # An error report file with more information is saved as: # /opt/elk/hs\_err\_pid19661.log # # If you would like to submit a bug report, please visit: # [http://bugreport.java.com/bugreport/crash.jsp](http://bugreport.java.com/bugreport/crash.jsp) # Aborted (core dumped) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 18, 2018, 6:03pm UTC](https://discuss.elastic.co/t/memory-leak-in-logstash-6-0-0/120095/10 "2018-03-18T18:03:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.