Cluster ID "5d1f1d" - Logstash startup is super slow

Elastic Stack Logstash
1

Hi,

Logstash is taking hours to execute as the filter size is growing, and CPU usage goes to max.

It works fine for a parser of around 2000 lines, but then it would slow down, almost stops, as it grows.

CentOS machine, Logstash version 5.3.2, Java version Open JDK 1.8; and haveged is installed and enabled.

Any suggestions on what can be the cause of it, is highly appreciated.

Thanks,
Vida

2

https://github.com/jruby/jruby/wiki/Improving-startup-time may be helpful.

3

Thank you @warkolm, have tried all this but it is not helping! I assume the nature of the Logstash is to run large amount of data!

4

How slow are you talking?
What's your config look like?

5

The config file is a grok filter which normalises data, followed by an extensive list of if conditions to add more fields to the normalised data. The if conditions are let's say around 4000 in number for a single config file and it would take up to 5, 6 for Logstash to ingest the data and starts successfully.

6

That's pretty massive, so not surprising it takes a while.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.