High CPU usage for logstash-agent


(Joice Joseph) #1

The logstash-agent is consuming more than 100% of CPU in our nodes. Is there any solution to fix this issue .
The logstash-agent.conf is as follows


input {
file {
type => "hybris_console"
add_field => { "environment" => "PROD" }
add_field => { "application" => "hybris" }
path => [ "<path_of_logfile>-" ]
codec => multiline {
pattern => "(.+|.+|.+|.+|[ ]{3}.
)|(.+|.+|.+|.+| ])|(.+|.+|.+|.+| SEVERE)|(.+|.+|.+|.+| de.hybris.platform)|(.+|.+|.+|.+| java.lang.NullPointerException)|(.+|.+|.+|.+|[ ]\t.)|(.+|.+|.+|.+|[ ]{9}.)"
negate => false
what => "previous"
charset => "UTF-8"
}
}
}
output {
stdout { }
redis {
host => ""
data_type => "list"
key => "logstash"
}
}

*********************************`


(Magnus B├Ąck) #2

You're not running Logstash 1.4.x and having trouble with the Logstash Web process, are you? Because it had a tendency to eat CPU once installed.

If it indeed is the agent that's consuming all that CPU, simplifying that regexp should help. It looks like you're trying to match continuation lines to make them meld with the previous line. I usually find it easier to craft an expression that matches non-continuation lines and set negate => true.

What kind of message rate are you seeing?


(system) #3