What is the recommend memory size to run logstash?

Elastic Stack Logstash
1

Hi all,

Please explain What is the recommend memory size to run logstash.

Thanks

2

That really depends on your use case. I routinely run logstash with a 400 MB heap on a server with 2 GB of memory. Some folk use 2GB or 4GB heap, some much more.

3

can it be 256 mb as per my need without OOM? Yes how can I do?

4

I doubt that you could run with a heap as small as 256MB, but you could try it.

5

In my case, logstash taking 550MB memory, so how can I reduce memory size upto some mb like 350 MB.

I have set 256m minimum and maximum in jvm.option file

6

Java uses a lot of memory that is not heap. The mx and sx options only control the heap size. The rest is not adjustable. With a minimal configuration I can take the heap size down to 128 MB. The virtual size of the java process is then 2.4 GB, and the RSS is 390 MB.

7

@Badger, when I down to 128 MB heap, logstash is giving OOM error. i.e outofmemory error.

is there any way to reduce logstash plugins size which also taking more memory. I mean install that plugins only which I am using in configuration. Like logstash-file-input, logstash-output-elasticsearch.

8

Not that I know of. logstash only loads the plugins that the configuration uses.

9

Because of ruby is dynamic, so this statement is true.

10

One thing, I am not getting understand that I am setting heap size maximum and minimum with 256m, but it is going upto 500m.

is there any concepts behind that?

11

How are you measuring the memory size?

12

Using linux command top -p <logstash-pid>

and checking column mem%, it is taking around 3.2% of 16gb RAM = ~500MB

And defining as in jvm.option,

-Xms256m -Xmx256m -Xss1m

13

The memory used by the heap can be a small percentage of the memory used by the JVM. The JVM requires hundreds of megabytes of non-heap memory. See this post, or Google "java non-heap memory".

Some JVMs have options to size some of the non-heap components, but typically this kind of tuning is more work than it is worth.

14

Thank you, @Badger for reply

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.