# How many redis input plugins Logstash can Handle? **URL:** \ **Category:** Logstash\ **Created:** [January 26, 2016, 10:20pm UTC](https://discuss.elastic.co/t/how-many-redis-input-plugins-logstash-can-handle/40170 "2016-01-26T22:20:14Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![GustavoHoyer1](https://avatars.discourse-cdn.com/v4/letter/g/6a8cbe/32.png) [@GustavoHoyer1](https://discuss.elastic.co/u/GustavoHoyer1)\ **Post date:** [January 26, 2016, 10:20pm UTC](https://discuss.elastic.co/t/how-many-redis-input-plugins-logstash-can-handle/40170/1 "2016-01-26T22:20:14Z")
Hi guys, Is there a limit of how many redis input plugins can be declared on a single instance of Logstash? We've decided some time ago to split the redis keys per client/system so that every system had its own queue on Redis and eventual increase on the amount of logs on one system wouldn't affect indexing the logs from other systems. Ex: Key = Logstash:client:system The problem is that now we have 133 systems on the elk stack and by consequence 133 input declarations on every instance of Logstash. Last month we had about 110 systems and we were indexing about 20000 logs/sec. Now with 133 systems the indexing rate droped to 200 logs/sec. I've tested with only one redis input in one of my indexers and the index rate came back to normal values. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [January 27, 2016, 8:30am UTC](https://discuss.elastic.co/t/how-many-redis-input-plugins-logstash-can-handle/40170/2 "2016-01-27T08:30:59Z")
> We've decided some time ago to split the redis keys per client/system so that every system had its own queue on Redis and eventual increase on the amount of logs on one system wouldn't affect indexing the logs from other systems. But if the same Logstash instance is processing all messages indexing _will_ be affected if a single host spews large amounts of events into Logstash's input queue. You won't see a complete standstill of indexing for other hosts but you're still competing for the same resources. ---
**Author:** ![GustavoHoyer1](https://avatars.discourse-cdn.com/v4/letter/g/6a8cbe/32.png) [@GustavoHoyer1](https://discuss.elastic.co/u/GustavoHoyer1)\ **Post date:** [January 27, 2016, 10:46pm UTC](https://discuss.elastic.co/t/how-many-redis-input-plugins-logstash-can-handle/40170/3 "2016-01-27T22:46:20Z")
We've doscovered that logstash heap wasn't enougth with visualvm and increasing the Heap\_size increased the indexing rate to 3000/s, but now we are getting bulking rejects on the elasticsearch. I think we might simplify our redis keys, specialyze our indexers per group of keys and play with the batch count and flush size values to see with we can get better indexing performance. ---
**Author:** ![GustavoHoyer1](https://avatars.discourse-cdn.com/v4/letter/g/6a8cbe/32.png) [@GustavoHoyer1](https://discuss.elastic.co/u/GustavoHoyer1)\ **Post date:** [January 27, 2016, 10:47pm UTC](https://discuss.elastic.co/t/how-many-redis-input-plugins-logstash-can-handle/40170/4 "2016-01-27T22:47:06Z")
What do you recommend @magnusbaeck? ---
**Author:** ![GustavoHoyer1](https://avatars.discourse-cdn.com/v4/letter/g/6a8cbe/32.png) [@GustavoHoyer1](https://discuss.elastic.co/u/GustavoHoyer1)\ **Post date:** [January 27, 2016, 11:24pm UTC](https://discuss.elastic.co/t/how-many-redis-input-plugins-logstash-can-handle/40170/5 "2016-01-27T23:24:27Z")
> [@magnusbaeck](#): > > But if the same Logstash instance is processing all messages indexing will be affected if a single host spews large amounts of events into Logstash's input queue. You won't see a complete standstill of indexing for other hosts but you're still competing for the same resources. This concept is working fine until now. When we have eventual eventual increase in one system, this backlog is retained on REDIS and the logs from other systems are still delivered near real time. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:14am UTC](https://discuss.elastic.co/t/how-many-redis-input-plugins-logstash-can-handle/40170/6 "2017-07-06T05:14:03Z")
Closed on July 6, 2017, 5:14am UTC