Is there a limit of how many redis input plugins can be declared on a single instance of Logstash?
We've decided some time ago to split the redis keys per client/system so that every system had its own queue on Redis and eventual increase on the amount of logs on one system wouldn't affect indexing the logs from other systems.
Key = Logstash:client:system
The problem is that now we have 133 systems on the elk stack and by consequence 133 input declarations on every instance of Logstash.
Last month we had about 110 systems and we were indexing about 20000 logs/sec.
Now with 133 systems the indexing rate droped to 200 logs/sec.
I've tested with only one redis input in one of my indexers and the index rate came back to normal values.