# Logstash elasticsearch pipeline processing rate

**URL:** <https://discuss.elastic.co/t/logstash-elasticsearch-pipeline-processing-rate/84789>\
**Category:** Logstash\
**Created:** [May 5, 2017, 9:44pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-pipeline-processing-rate/84789 "2017-05-05T21:44:52Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sharat\_Jagannath](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sharat_jagannath/32/24299_2.png) [@Sharat\_Jagannath](https://discuss.elastic.co/u/Sharat_Jagannath)\
**Post date:** [May 5, 2017, 9:44pm UTC](https://discuss.elastic.co/t/logstash-elasticsearch-pipeline-processing-rate/84789/1 "2017-05-05T21:44:52Z")

</div>

i have a basic question on logstash elasticsearch pipeline processing rate.  
When i set flush\_size to a value, does logstash wait until that size is reached before it starts sending it to es? if the size is not reached what happens?  
how do i make it work for bulk and also non bulk requests dynamically based on size of the input.

Thanks

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 8, 2017, 7:26am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-pipeline-processing-rate/84789/2 "2017-05-08T07:26:38Z")

</div>

The documentation of the `idle_flush_time` option should answer your questions.

---

<div class="post-metadata">

**Author:** ![Sharat\_Jagannath](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sharat_jagannath/32/24299_2.png) [@Sharat\_Jagannath](https://discuss.elastic.co/u/Sharat_Jagannath)\
**Post date:** [May 20, 2017, 12:25am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-pipeline-processing-rate/84789/3 "2017-05-20T00:25:35Z")

</div>

does increasing number of logstash consumers increase the indexing speed?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [May 20, 2017, 4:41am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-pipeline-processing-rate/84789/4 "2017-05-20T04:41:40Z")

</div>

That depends on what is limiting performance. If any of the outputs, e.g. Elasticsearch, is not able to handle higher throughput, adding Logstash instances may not help. If however Logstash, which usually is CPU bound, is the bottleneck, scaling up/out the Logstash layer may help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 17, 2017, 4:41am UTC](https://discuss.elastic.co/t/logstash-elasticsearch-pipeline-processing-rate/84789/5 "2017-06-17T04:41:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.