# Logstash Memory Requirements and JVM Sizing

**URL:** https://discuss.elastic.co/t/logstash-memory-requirements-and-jvm-sizing/227072
**Category:** Logstash
**Created:** [April 8, 2020, 7:49am UTC](https://discuss.elastic.co/t/logstash-memory-requirements-and-jvm-sizing/227072 "2020-04-08T07:49:26Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![shrikantgulia](https://avatars.discourse-cdn.com/v4/letter/s/c68b51/32.png) [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)
#### Post date: [April 8, 2020, 7:49am UTC](https://discuss.elastic.co/t/logstash-memory-requirements-and-jvm-sizing/227072/1 "2020-04-08T07:49:26Z")

</div>

Hello

I am getting approx 5000 events per second and a lot of Grok, KV, DNS filters are being used.  
My current memory is 16 GB, 8 Core thread per core is 2.  
I have set JVM size to 8GB.  
But My JVM is getting full and Logstash is getting shut down and many events are getting dropped.

Please advice me

Stay Safe and Search!

Regards

---

<div class="post-metadata">

### Author: ![shrikantgulia](https://avatars.discourse-cdn.com/v4/letter/s/c68b51/32.png) [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)
#### Post date: [April 9, 2020, 8:30am UTC](https://discuss.elastic.co/t/logstash-memory-requirements-and-jvm-sizing/227072/2 "2020-04-09T08:30:40Z")

</div>

Can Someone from elastic guide me Please

Regards

---

<div class="post-metadata">

### Author: ![shrikantgulia](https://avatars.discourse-cdn.com/v4/letter/s/c68b51/32.png) [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)
#### Post date: [April 9, 2020, 9:25am UTC](https://discuss.elastic.co/t/logstash-memory-requirements-and-jvm-sizing/227072/3 "2020-04-09T09:25:49Z")

</div>

@Christian_Dahlqvist

Can You Please help me  
I am stuck at this

Regards

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 9, 2020, 9:30am UTC](https://discuss.elastic.co/t/logstash-memory-requirements-and-jvm-sizing/227072/4 "2020-04-09T09:30:26Z")

</div>

This forum is manned by volunteers. There is therefore no guaranteed SLAs and nor guarantee you will get an answer. Please do not ping people who are not already involved in the thread. If you have not received any response after 2-3 days (excluding weekends) I would recommend you bump the issue, but please give people time to respond.

I do not have an answer to this question, so will leave it for others. It would probably help if you provided as much information as possible though, e.g. version used, settings and information about the pipeline(s)/configuration used.

---

<div class="post-metadata">

### Author: ![shrikantgulia](https://avatars.discourse-cdn.com/v4/letter/s/c68b51/32.png) [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)
#### Post date: [April 9, 2020, 9:35am UTC](https://discuss.elastic.co/t/logstash-memory-requirements-and-jvm-sizing/227072/5 "2020-04-09T09:35:42Z")

</div>

@Christian_Dahlqvist I apologize for this behaviour  
I am using 7.3 version of Logstash  
I am having 3 Pipelines and I am receiving data from firewalls and Applications on UDP Port  
I am receiving data on UDP Port for the firewalls and It is dropping the Packets

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/b/ab7949f6f9091cef958d7943271cca12d4e1ecbd.png)

I am having 16 GB Memory 14 GB JVM  
This is my input file Config  
udp {  
port =\> 5600  
queue\_size =\> 163840  
workers =\> 8  
codec =\> multiline {  
pattern =\> "^\s"  
what =\> "next"  
}  
type =\> "udp"  
}

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 9, 2020, 9:47am UTC](https://discuss.elastic.co/t/logstash-memory-requirements-and-jvm-sizing/227072/6 "2020-04-09T09:47:51Z")

</div>

Given that UDP does not guarantee ordering nor delivery I do not think it makes any sense to have a multiline filter in place.

There have been some threads on tuning UDP performance in Logstash in the forum, e.g. [this one](https://discuss.elastic.co/t/dropped-udp-packets-on-logstash-inputs-3-different-udp-inputs/146922) and [this one](https://discuss.elastic.co/t/increase-udp-input-plugin-performance/130798/3), so I would recommend reviewing these.

---

<div class="post-metadata">

### Author: ![shrikantgulia](https://avatars.discourse-cdn.com/v4/letter/s/c68b51/32.png) [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)
#### Post date: [April 9, 2020, 11:08am UTC](https://discuss.elastic.co/t/logstash-memory-requirements-and-jvm-sizing/227072/7 "2020-04-09T11:08:18Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> ultiline filter in place

Not Successful

These are the Logs which i am getting  
[2020-04-09T12:16:48,652][ERROR][logstash.agent] Failed to execute action {:id=\>:globalfilter, :action\_type=\>LogStash::ConvergeResult::FailedAction, :message=\>"Something is wrong with your configuration.", :backtrace=\>["/usr/share/logstash/logstash-core/lib/logstash/config/mixin.rb:87:in `config_init'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:126:in `initialize'", "org/logstash/plugins/PluginFactoryExt.java:81:in `filter_delegator'", "org/logstash/plugins/PluginFactoryExt.java:251:in `plugin'", "org/logstash/execution/JavaBasePipelineExt.java:50:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/reload.rb:37:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:325:in `block in converge\_state'"]}  
[2020-04-09T12:40:04,281][ERROR][org.logstash.execution.ShutdownWatcherExt] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.  
[2020-04-09T13:33:37,090][ERROR][org.logstash.execution.ShutdownWatcherExt] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.  
[2020-04-09T13:33:55,346][ERROR][logstash.inputs.udp] UDP listener died {:exception=\>#\<IOError: closed stream\>, :backtrace=\>["org/jruby/ext/socket/RubyUDPSocket.java:294:in `recvfrom_nonblock'", "org/jruby/ext/socket/RubyUDPSocket.java:286:in `recvfrom\_nonblock'", "org/jruby/ext/socket/RubyUDPSocket.java:269:in `recvfrom_nonblock'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-udp-3.3.4/lib/logstash/inputs/udp.rb:125:in `block in udp\_listener'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-udp-3.3.4/lib/logstash/inputs/udp.rb:123:in `udp_listener'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-udp-3.3.4/lib/logstash/inputs/udp.rb:68:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java\_pipeline.rb:309:in `inputworker'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:302:in `block in start\_input'"]}  
[2020-04-09T14:49:32,340][ERROR][org.logstash.execution.ShutdownWatcherExt] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 9, 2020, 11:15am UTC](https://discuss.elastic.co/t/logstash-memory-requirements-and-jvm-sizing/227072/8 "2020-04-09T11:15:29Z")

</div>

If you rely on lines being processed in order so you can assemble them into complete messages you should use TCP, not UDP.

---

<div class="post-metadata">

### Author: ![shrikantgulia](https://avatars.discourse-cdn.com/v4/letter/s/c68b51/32.png) [@shrikantgulia](https://discuss.elastic.co/u/shrikantgulia)
#### Post date: [April 16, 2020, 5:02am UTC](https://discuss.elastic.co/t/logstash-memory-requirements-and-jvm-sizing/227072/9 "2020-04-16T05:02:01Z")

</div>

Hello @Christian_Dahlqvist,

I have increased the JVM of the machine

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/2/b2f9a09a5aed3be42bf25389be23f5a7169ca1c0.png)  
It is Consuming the Full JVM .  
currently I am having the 48 GB and 16 core machine

Regards

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 14, 2020, 5:02am UTC](https://discuss.elastic.co/t/logstash-memory-requirements-and-jvm-sizing/227072/10 "2020-05-14T05:02:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
