# Input-redis parameters versus pipeline parameters

**URL:** https://discuss.elastic.co/t/input-redis-parameters-versus-pipeline-parameters/43948
**Category:** Logstash
**Created:** [March 9, 2016, 9:07pm UTC](https://discuss.elastic.co/t/input-redis-parameters-versus-pipeline-parameters/43948 "2016-03-09T21:07:18Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![Bruno\_Lavoie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruno_lavoie/32/8408_2.png) [@Bruno\_Lavoie](https://discuss.elastic.co/u/Bruno_Lavoie)
#### Post date: [March 9, 2016, 9:07pm UTC](https://discuss.elastic.co/t/input-redis-parameters-versus-pipeline-parameters/43948/1 "2016-03-09T21:07:19Z")

</div>

Hello,

As of 2.2, pipeline architecture has changed.

From the docs:

> Each input {} statement in the Logstash configuration file runs in its own thread. Inputs write events to a common Java SynchronousQueue. This queue holds no events, instead transferring each pushed event to a free worker, blocking if all workers are busy. Each pipeline worker thread takes a batch of events off this queue, creating a buffer per worker, runs the batch of events through the configured filters, then runs the filtered events through any outputs.

IIRC, giving that:

- Redis input thread(s) are sleeping when there's no free worker.
- When a pipeline worker is free, he asks to a free Redis input thread _n_ events, where _n_ is the batch size (-b switch)

With this in mind, if I have Logstash running on say 16 core machine:

- with a default batch size of 125
- with a default of 16 workers

Is it a good idea to configure the redis input with:

- The redis batch\_count value the same value of pipeline batch size?
- The redis batch\_count value an inferior multiple of pipeline batch size (in our case, say 25)?
- With a lot of workers, a certain amount of redis-input threads? (in our case, say 4)?
- If I have only 1 input thread for 16 or 32 pipeline workers, is it bad?

I know that's a lot of testing and it depends on our specific case, but I want to find the right balance.

A kind of rule of thumb like:

- number of workers = cores \* 2
- a starting default of 125 for pipeline batch size
- redis-input batch-count at the same value of pipeline batch size (125)
- depending on filter complexity and processing time, a certain amount of redis-input threads like #workers / 4, or 8 depending the case.

The point is to make a certain amount of workers not wait for free input threads.

Am I missing something?

Thanks  
Bruno Lavoie

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 5:07am UTC](https://discuss.elastic.co/t/input-redis-parameters-versus-pipeline-parameters/43948/2 "2017-07-06T05:07:33Z")

</div>


