Logstash use 100% CPU in lazy mode


(Max) #1

Good day!
I have Logstash 6.2 with ES 6.2.
Logstash use 100% of one core in lazy mode without any load.
Is it normal?

[elog@srv7-owltank current]$ ps aux | grep logstash
elog 21660 110 3.4 5867628 1118488 pts/3 Sl 15:00 10:56 /usr/java/jdk1.8.0_181-amd64/bin/java -Xss2048k -Xmx2g -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=9123 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.local.only=true -Djava.io.tmpdir=/data/elog/logstash/current -cp /data/elog/logstash/current/logstash-core/lib/jars/animal-sniffer-annotations-1.14.jar:/data/elog/logstash/current/logstash-core/lib/jars/commons-compiler-3.0.8.jar:/data/elog/logstash/current/logstash-core/lib/jars/error_prone_annotations-2.0.18.jar:/data/elog/logstash/current/logstash-core/lib/jars/google-java-format-1.5.jar:/data/elog/logstash/current/logstash-core/lib/jars/guava-22.0.jar:/data/elog/logstash/current/logstash-core/lib/jars/j2objc-annotations-1.1.jar:/data/elog logstash/current/logstash-core/lib/jars/jackson-annotations-2.9.1.jar:/data/elog/logstash/current/logstash-core/lib/jars/jackson-core-2.9.1.jar:/data/elog/logstash/current/logstash-core/lib/jars/jackson-databind-2.9.1.jar:/data/elog/logstash/current/logstash-core/lib/jars/jackson-dataformat-cbor-2.9.1.jar:/data/elog/logstash/current/logstash-core/lib/jars/janino-3.0.8.jar:/data/elog/logstash/current/logstash-core/lib/jars/javac-shaded-9-dev-r4023-3.jar:/data/elog logstash/current/logstash-core/lib/jars/jruby-complete-9.1.13.0.jar:/data/elog/logstash/current/logstash-core/lib/jars/jsr305-1.3.9.jar:/data/elog/logstash/current/logstash-core/lib/jars/log4j-api-2.9.1.jar:/data/elog/logstash/current/logstash-core/lib/jars/log4j-core-2.9.1.jar:/data/elog/logstash/current/logstash-core/lib/jars/log4j-slf4j-impl-2.9.1.jar:/data/elog/logstash/current/logstash-core/lib/jars/logstash-core.jar:/data/elog/logstash/current/logstash-core/lib/jars/slf4j-api-1.7.25.jar org.logstash.Logstash -l /data/logs/elog/logstash --path.settings /data/elog/logstash/current/config --path.config /data/elog logstash/current/config/conf.d --http.host 0.0.0.0 --http.port 9600

Thank you!!!


(Max) #2

Logstash config:
input {

beats {
host => "0.0.0.0"
port => 5044
tags => ["filebeat"]
}
http {
host => "0.0.0.0"
port => 31311
}
..
}

filter {

if "jsonl" in [type] or ([fields] and [fields][document_type] == "jsonl") {
....
} else {
grok {
patterns_dir => ["/data/elog/logstash/current/config/patterns"]
match => { "message" => "%{LOG_CODE:log_code} %{DATESTAMP:date_time} - <%{SESSION_ID:session_id}>,%{BASE10NUM:call_time},%{SERVICE_CODE},%{BASE10NUM:
add_tag => ["brt_reject"]
}
date {
match => ["date_time", "dd/MM/yy HH:mm:ss"]
}
if [message] =~ /^(I:[A-Z]{3}|[F,D,S,P]:[A-Z]{2,3})/ {
# Alternative: [log_code] ~= ^I:CC
drop {}
}
}
mutate{
add_field => { "logstash" => "$(HOSTNAME)"}
}

}
output {
if ![fields]{
elasticsearch {
document_type => "filebeat"
hosts => ["srv7-lipoglav:9200"]
index => "logstash-main-filebeat-elog-%{+YYYY.MM.dd}"
}
} else {
elasticsearch {
document_type => "filebeat"
hosts => ["srv7-lipoglav:9200"]
index => "logstash-main-%{[fields][index_prefix]}-elog-%{+YYYY.MM.dd}"
}
}
}


(Max) #3

Solution: Logstash 6.2.2 high CPU usage


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.