# Logstash cpu usage is very high, and there are a large number of thread GC

**URL:** <https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high-and-there-are-a-large-number-of-thread-gc/305666>\
**Category:** Logstash\
**Created:** [May 26, 2022, 6:42am UTC](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high-and-there-are-a-large-number-of-thread-gc/305666 "2022-05-26T06:42:29Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![1065916870](https://avatars.discourse-cdn.com/v4/letter/1/e47774/32.png) [@1065916870](https://discuss.elastic.co/u/1065916870)\
**Post date:** [May 26, 2022, 6:42am UTC](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high-and-there-are-a-large-number-of-thread-gc/305666/1 "2022-05-26T06:42:30Z")

</div>

Using 3-node logstash to consume kafka data, logstash cpu usage is very high, and there are a large number of thread GC

logstash configuration: 3 node , 16C 32G  
jvm -Xms16g -Xmx16g

two pipeline:  
pipeline.workers: 16  
pipeline.batch.size: 2000  
pipeline.batch.delay: 20

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/0/101271510f3919099865491bbec6314c93a7cdbb.png)

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 26, 2022, 7:34am UTC](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high-and-there-are-a-large-number-of-thread-gc/305666/2 "2022-05-26T07:34:23Z")

</div>

Please don't post pictures of text, logs or code. They are difficult to read, impossible to search and replicate (if it's code), and some people may not be even able to see them 🙂

What do the Logstash logs show?

---

<div class="post-metadata">

**Author:** ![1065916870](https://avatars.discourse-cdn.com/v4/letter/1/e47774/32.png) [@1065916870](https://discuss.elastic.co/u/1065916870)\
**Post date:** [May 26, 2022, 8:13am UTC](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high-and-there-are-a-large-number-of-thread-gc/305666/3 "2022-05-26T08:13:18Z")

</div>

logstash shows normal, no error

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 26, 2022, 4:19pm UTC](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high-and-there-are-a-large-number-of-thread-gc/305666/4 "2022-05-26T16:19:23Z")

</div>

Up to a certain point (and I don't think 16 cores is it), a parallel garbage collector can use one thread per core, so having 16 GC threads looks normal to me. Having the GC threads use 3.7% of the CPU seems high. You would have to review the GC logs to see what is happening (which I used to do for a living, but that was years ago, with collectors that are not much used now).

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [May 26, 2022, 5:05pm UTC](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high-and-there-are-a-large-number-of-thread-gc/305666/5 "2022-05-26T17:05:05Z")

</div>

What does your pipelines configuration looks like?

---

<div class="post-metadata">

**Author:** ![1065916870](https://avatars.discourse-cdn.com/v4/letter/1/e47774/32.png) [@1065916870](https://discuss.elastic.co/u/1065916870)\
**Post date:** [May 27, 2022, 3:06am UTC](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high-and-there-are-a-large-number-of-thread-gc/305666/6 "2022-05-27T03:06:26Z")

</div>

I added consumer\_threads in logstash input kafka，but now output Elasticsearch has many rejected

This is my pipelines

```auto
- pipeline.id: case_logs
 path.config: /work/logstash/logstash_case_logs.conf
 queue.type: memory
 # =cpu
 pipeline.workers: 16
 pipeline.batch.size: 1000
 pipeline.batch.delay: 10

- pipeline.id: other_logs
 path.config: /work/logstash/logstash_other_logs.conf
 queue.type: memory
 # =cpu
 pipeline.workers: 16
 pipeline.batch.size: 2000
 pipeline.batch.delay: 5

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [May 27, 2022, 12:16pm UTC](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high-and-there-are-a-large-number-of-thread-gc/305666/7 "2022-05-27T12:16:39Z")

</div>

You need to share the content of the `.conf` files, if your logstash is using a lot of cpu the issue could be in one of your filters for example.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 24, 2022, 12:17pm UTC](https://discuss.elastic.co/t/logstash-cpu-usage-is-very-high-and-there-are-a-large-number-of-thread-gc/305666/8 "2022-06-24T12:17:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
