# How to limit filebeat logs to logstash

**URL:** <https://discuss.elastic.co/t/how-to-limit-filebeat-logs-to-logstash/97504>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [August 18, 2017, 3:14am UTC](https://discuss.elastic.co/t/how-to-limit-filebeat-logs-to-logstash/97504 "2017-08-18T03:14:22Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nikhilpawar1985](https://avatars.discourse-cdn.com/v4/letter/n/e68b1a/32.png) [@Nikhilpawar1985](https://discuss.elastic.co/u/Nikhilpawar1985)\
**Post date:** [August 18, 2017, 3:14am UTC](https://discuss.elastic.co/t/how-to-limit-filebeat-logs-to-logstash/97504/1 "2017-08-18T03:14:22Z")

</div>

Hi,  
i am sending our production logs to logstash cluster which has 6 server in cluster. In my filebeat setting i have number of workers as 2 . So 12 pipelines to logstash cluster each prod server.

But sometimes due to production traffic spikes our application logs gets flooded and get very big in size in few minutes. I dont want to crash my logstash cluster as it is used to parse other applications logs too.

I am continuously tailing my log file .

Is there a way that i can restrict limit on sending logs at a time and filebeat keeps sending data in this kind of situations.

Thanks,  
Nikhil

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [August 18, 2017, 12:11pm UTC](https://discuss.elastic.co/t/how-to-limit-filebeat-logs-to-logstash/97504/2 "2017-08-18T12:11:54Z")

</div>

You can not limit event rates from filebeat directly. You are advised to you OS/network tooling to install whatever policies required. See: [https://www.elastic.co/guide/en/beats/filebeat/current/faq.html#bandwidth-throttling](https://www.elastic.co/guide/en/beats/filebeat/current/faq.html#bandwidth-throttling)

As logstash is the server and you want to protect logstash from being overloaded by logs, You should consider to apply QoS rules on the Logstash server. With network or the server having rate limiting in place, this will create back-pressure in filebeat and slow down filebeat. You might even consider a time-scheduled policy, reducing bandwidth for filebeat at peak times even more.

Furthermore, if you add more filebeat instance to your environment, the overall bandwidth used will not change (you don't have to re-balance all filebeat instances). Removing a filebeat instance or having a machine down for maintenance frees up bandwidth for other beats to use.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 15, 2017, 12:11pm UTC](https://discuss.elastic.co/t/how-to-limit-filebeat-logs-to-logstash/97504/3 "2017-09-15T12:11:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.