# Logstash-input-rabbitmq: Over 8000 unused threads on a 24-cores machine

**URL:** <https://discuss.elastic.co/t/logstash-input-rabbitmq-over-8000-unused-threads-on-a-24-cores-machine/70458>\
**Category:** Logstash\
**Created:** [January 3, 2017, 3:51pm UTC](https://discuss.elastic.co/t/logstash-input-rabbitmq-over-8000-unused-threads-on-a-24-cores-machine/70458 "2017-01-03T15:51:27Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![LeMic](https://avatars.discourse-cdn.com/v4/letter/l/91b2a8/32.png) [@LeMic](https://discuss.elastic.co/u/LeMic)\
**Post date:** [January 3, 2017, 3:51pm UTC](https://discuss.elastic.co/t/logstash-input-rabbitmq-over-8000-unused-threads-on-a-24-cores-machine/70458/1 "2017-01-03T15:51:27Z")

</div>

Hello,

We're using Logstash 5.1.1 on a physical machine with 24 cores (2 sockets \* 6 cores \* 2 hyper threads) and many instances of the input-rabbitmq plugin.

With this setup, Logstash create a lot of unused threads for each connection/instance: about 8000 ! This consume about 2 GB of the RAM just for the thread stacks and slow down the event processing.

Actually, we located the source of this behavior in the march-hare and the RabbitMQ Client libraries: for each connection, a new com.rabbitmq.client.ConnectionFactory is created along with a java.util.concurrent.ExecutorService (used to run the consumers threads). However, each ExecutorService come with a pool of 2\* threads (48 threads on a 24-core machine) but this pool are never shared between the connections.

For example, on our setup we have about 40 queues on 4 RabbitMQ servers : 40 rabbitmq blocs (config) \* 4 \* 24 cores \* 2 = 7680 threads (it doesn't depend on the "workers" parameter).

We can reduce this number by patching the march-hare library like this :  
/usr/share/logstash/vendor/bundle/jruby/1.9/gems/march\_hare-2.20.0-java/lib/march\_hare/session.rb

> def self.connect(options = {})  
> cf = ConnectionFactory.new  
> cf.setSharedExecutor(Executors.newSingleThreadExecutor)

This will create a single thread per AMQP connection. However this is maybe not the best solution. Is there a right solution to do it properly ?

Thank you 🙂

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 5, 2017, 12:31am UTC](https://discuss.elastic.co/t/logstash-input-rabbitmq-over-8000-unused-threads-on-a-24-cores-machine/70458/2 "2017-01-05T00:31:37Z")

</div>

Thanks for raising this, can you please raise an issue against the plugin with this info so we can look into it further?

---

<div class="post-metadata">

**Author:** ![LeMic](https://avatars.discourse-cdn.com/v4/letter/l/91b2a8/32.png) [@LeMic](https://discuss.elastic.co/u/LeMic)\
**Post date:** [January 5, 2017, 8:16am UTC](https://discuss.elastic.co/t/logstash-input-rabbitmq-over-8000-unused-threads-on-a-24-cores-machine/70458/3 "2017-01-05T08:16:32Z")

</div>

Thank you for your response,

I posted a new issue on Github here : [https://github.com/logstash-plugins/logstash-input-rabbitmq/issues](https://github.com/logstash-plugins/logstash-input-rabbitmq/issues)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 2, 2017, 8:16am UTC](https://discuss.elastic.co/t/logstash-input-rabbitmq-over-8000-unused-threads-on-a-24-cores-machine/70458/4 "2017-02-02T08:16:38Z")

</div>

Closed on February 2, 2017, 8:16am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.