# Retrying problems - bandwidth increase

**URL:** <https://discuss.elastic.co/t/retrying-problems-bandwidth-increase/124644>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 19, 2018, 9:16pm UTC](https://discuss.elastic.co/t/retrying-problems-bandwidth-increase/124644 "2018-03-19T21:16:13Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![rsk0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rsk0/32/124810_2.png) [@rsk0](https://discuss.elastic.co/u/rsk0)\
**Post date:** [March 19, 2018, 9:16pm UTC](https://discuss.elastic.co/t/retrying-problems-bandwidth-increase/124644/1 "2018-03-19T21:16:13Z")

</div>

I've got a setup of a couple hundred Filebeats sending in to Redis -\> Logstash -\> Elasticsearch, and when Redis fills up and rejects publications the bandwidth being used starts increasing. I'm guessing this is because the Filebeats are still sending whole batches of log lines during their connections, before Redis returns a rejection, and that retrying is happening more frequently than batches are normally ready to be sent.

I'd like to reduce the bandwidth consumed during ingest queue full scenarios. It would be great if there were Filebeat parameters for retry frequency, but it looks like there aren't. I understand there's a feature in development for spooling to disk, which is great, but doesn't address this issue directly as far as I can tell.

Also, the number of connections from Filebeats to the queue host skyrockets during queue full scenarios. I'm guessing this is because publish rejections cause Filebeat to drop the connection and to reconnect for retries. It would be great if I could stop that from happening because it results in a proliferation of brief connections and currently generates an ongoing state of around 12 to 13 thousand TIME\_WAITs. That's not the end of the world (or available network connections), but I'll be adding more clients...

There _is_ a "timeout" option in the Redis output plugin that might be of some use in tweaking network connections, but the docs only say, "The Redis connection timeout in seconds. The default is 5 seconds." and don't clarify what generates timeout conditions or how the software addresses timeouts.

Relevant Filebeat config bits:

> ```
> queue.mem:
> events: 4096
> # wait for at least 512 events to publish
> flush.min_events: 512
> # _or_ at most 10s
> flush.timeout: 10s
> output.redis:
> timeout: 60 # default 5 -- try to keep connection on slow senders
> bulk_max_size: 2048 # default 2048
> 
> ```

Relevant Logstash config bits:

> ```
> input {
> redis {
> ..
> threads => 8
> batch_count => 125 # default 125
> }
> }
> 
> ```

and

> ```
> pipeline.batch.size: 512
> pipeline.workers: 4
> 
> ```

---

<div class="post-metadata">

**Author:** ![rsk0](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rsk0/32/124810_2.png) [@rsk0](https://discuss.elastic.co/u/rsk0)\
**Post date:** [March 21, 2018, 12:01am UTC](https://discuss.elastic.co/t/retrying-problems-bandwidth-increase/124644/2 "2018-03-21T00:01:08Z")

</div>

I've had it recommended to me that we skip Redis entirely and just use Logstash for the ingest, and use Logstash's queueing capability. From what I gather, Logstash handles Filebeat inputs during back pressure by refusing connections, which should eliminate the excessive bandwidth consumption.

Props to Elastic support for the help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 18, 2018, 12:01am UTC](https://discuss.elastic.co/t/retrying-problems-bandwidth-increase/124644/3 "2018-04-18T00:01:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.