Very low cpu usage in logstash server

(Xiaochuan Lin) #1

My logstash server has 8 CPUs and 8g Men, and 4 pipeline-workers(
4 pipeline-workers is the best performance after testing), but no matter how i test it, the CPU usage has been 20-30%.
The logstash input data comes from kafka, and kafka has 8 partitions in a topic,so my logstash has 8 threads to retrieve input data。
The logstash version is 6.4.3.
The logstash input config is:
input {
kafka {
bootstrap_servers => ",,"
topics => "nginx"
group_id => "nginx-group"
codec => "json"
consumer_threads => 8
decorate_events => true

At first, The logstash input data comes from filebeat, and the cpu uasge of logstash has been 70%,even more then 80%.

I don't understand why this is happening. please help me, thinks

(Christian Dahlqvist) #2

Logstash is only able to process data as fast as the slowest downstream system can accept it. Where are you sending data? Have you verified there is no bottleneck downstream? What does your full config look like?

(Xiaochuan Lin) #3

I have found the problem, the CPU is restricted on the server, thank you Christian_Dahlqvist

(system) closed #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.