Logstash cpu usage is very high, and there are a large number of thread GC

[1065916870 (tialias)]

Using 3-node logstash to consume kafka data, logstash cpu usage is very high, and there are a large number of thread GC

logstash configuration: 3 node , 16C 32G
jvm -Xms16g -Xmx16g

two pipeline:
pipeline.workers: 16
pipeline.batch.size: 2000
pipeline.batch.delay: 20

[warkolm (Mark Walkom)]

Please don't post pictures of text, logs or code. They are difficult to read, impossible to search and replicate (if it's code), and some people may not be even able to see them

What do the Logstash logs show?

[1065916870 (tialias)]

logstash shows normal, no error

[Badger]

Up to a certain point (and I don't think 16 cores is it), a parallel garbage collector can use one thread per core, so having 16 GC threads looks normal to me. Having the GC threads use 3.7% of the CPU seems high. You would have to review the GC logs to see what is happening (which I used to do for a living, but that was years ago, with collectors that are not much used now).

[leandrojmp (Leandro Pereira)]

What does your pipelines configuration looks like?

[1065916870 (tialias)]

I added consumer_threads in logstash input kafka，but now output Elasticsearch has many rejected

This is my pipelines

- pipeline.id: case_logs
 path.config: /work/logstash/logstash_case_logs.conf
 queue.type: memory
 # =cpu
 pipeline.workers: 16
 pipeline.batch.size: 1000
 pipeline.batch.delay: 10

- pipeline.id: other_logs
 path.config: /work/logstash/logstash_other_logs.conf
 queue.type: memory
 # =cpu
 pipeline.workers: 16
 pipeline.batch.size: 2000
 pipeline.batch.delay: 5

[leandrojmp (Leandro Pereira)]

You need to share the content of the .conf files, if your logstash is using a lot of cpu the issue could be in one of your filters for example.