# FileBeats with Redis **URL:** \ **Category:** Beats\ **Created:** [December 1, 2015, 9:30am UTC](https://discuss.elastic.co/t/filebeats-with-redis/36009 "2015-12-01T09:30:35Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![mvenkat\_in](https://avatars.discourse-cdn.com/v4/letter/m/58f4c7/32.png) [@mvenkat\_in](https://discuss.elastic.co/u/mvenkat_in)\ **Post date:** [December 1, 2015, 9:30am UTC](https://discuss.elastic.co/t/filebeats-with-redis/36009/1 "2015-12-01T09:30:35Z")
Hi My Current setup is - File Beat --\> Log Stash ---\> Elastic Search The expected log volume is say 10 GB per hour to LS (as current version of FB doesn't have regexp feature and ships all logs). Hence would like to know if FB and Log Stash would be able to handle this log without loss of any messages or network over head !! What is the best practice !! Should we implement FB--\>Any Queue (Redis) --\> Log Stash. If so does File beat support integrating with Fedis. Please suggest. ---
**Author:** ![monica](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/monica/32/3696_2.png) [@monica](https://discuss.elastic.co/u/monica)\ **Post date:** [December 1, 2015, 10:40am UTC](https://discuss.elastic.co/t/filebeats-with-redis/36009/2 "2015-12-01T10:40:38Z")
We are planning to add support in the next release of Filebeat for regexp, that will reduce the number of logs. I'll come back to you when we finish with the implementation, so maybe you want to try one of our nightlies before the release date. ---
**Author:** ![mvenkat\_in](https://avatars.discourse-cdn.com/v4/letter/m/58f4c7/32.png) [@mvenkat\_in](https://discuss.elastic.co/u/mvenkat_in)\ **Post date:** [December 1, 2015, 10:59am UTC](https://discuss.elastic.co/t/filebeats-with-redis/36009/3 "2015-12-01T10:59:25Z")
Hi Monica Thanks. Would wait for the next release. But my question is more on the architecture - whether my current setup could handle the load or need any intermediary Queuing system to ensure the load is balanced or throttled. Not sure if this is the correct category for this query.. ---
**Author:** ![monica](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/monica/32/3696_2.png) [@monica](https://discuss.elastic.co/u/monica)\ **Post date:** [December 1, 2015, 11:41am UTC](https://discuss.elastic.co/t/filebeats-with-redis/36009/4 "2015-12-01T11:41:51Z")
Redis output is deprecated and we encourage our Filebeat users to send data directly to Logstash. In 3.x release, we are planning to add persistent queuing in Logstash ([https://github.com/elastic/logstash/issues/2605](https://github.com/elastic/logstash/issues/2605)) and an additional queuing system will not be needed anymore. ---
**Author:** ![monica](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/monica/32/3696_2.png) [@monica](https://discuss.elastic.co/u/monica)\ **Post date:** [December 1, 2015, 12:33pm UTC](https://discuss.elastic.co/t/filebeats-with-redis/36009/5 "2015-12-01T12:33:11Z")
[Here](https://github.com/elastic/filebeat/issues/78) is the link to the GitHub issue implementing regexp in Filebeat. ---
**Author:** ![Mike\_Wurtz](https://avatars.discourse-cdn.com/v4/letter/m/c0e974/32.png) [@Mike\_Wurtz](https://discuss.elastic.co/u/Mike_Wurtz)\ **Post date:** [June 15, 2016, 11:22pm UTC](https://discuss.elastic.co/t/filebeats-with-redis/36009/7 "2016-06-15T23:22:44Z")
I'm getting this error message in Logstash 2.3: "Beats input: the pipeline is blocked, temporary refusing new connection." I'm assuming this is because I do not have a message queue between filebeat and logstash.. When you say "redis output is depricated, and we should send directly to logstash." What version of logstash does that pertain to? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [June 16, 2016, 9:39am UTC](https://discuss.elastic.co/t/filebeats-with-redis/36009/8 "2016-06-16T09:39:15Z")
totally unrelated to your problem. Please create a new topic. Message queue is optional + redis has been renewed for 5.x release (complete rewrite). Logstash problem due to pipeline in logstash being blocked by output or very slow filter. ---
**Author:** ![Sharath\_Vutpala](https://avatars.discourse-cdn.com/v4/letter/s/9d8465/32.png) [@Sharath\_Vutpala](https://discuss.elastic.co/u/Sharath_Vutpala)\ **Post date:** [March 22, 2017, 8:41am UTC](https://discuss.elastic.co/t/filebeats-with-redis/36009/9 "2017-03-22T08:41:13Z")
Hey @monica: Is the persistent queuing in Logstash available now? I am planning to use Redis in between Filebeat and Logstash. ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [March 22, 2017, 4:44pm UTC](https://discuss.elastic.co/t/filebeats-with-redis/36009/10 "2017-03-22T16:44:31Z")
It's a beta feature in 5.2. [https://www.elastic.co/guide/en/logstash/5.2/persistent-queues.html](https://www.elastic.co/guide/en/logstash/5.2/persistent-queues.html) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 9:49pm UTC](https://discuss.elastic.co/t/filebeats-with-redis/36009/11 "2017-07-05T21:49:47Z")