My logstash server has 8 CPUs and 8g Men, and 4 pipeline-workers(
4 pipeline-workers is the best performance after testing), but no matter how i test it, the CPU usage has been 20-30%.
The logstash input data comes from kafka, and kafka has 8 partitions in a topic，so my logstash has 8 threads to retrieve input data。
The logstash version is 6.4.3.
The logstash input config is:
bootstrap_servers => "192.168.23.1:9092,192.168.23.2:9092,192.168.23.6:9092"
topics => "nginx"
group_id => "nginx-group"
codec => "json"
consumer_threads => 8
decorate_events => true
At first, The logstash input data comes from filebeat, and the cpu uasge of logstash has been 70%,even more then 80%.
I don't understand why this is happening. please help me, thinks