# Elastic Stack for 100 mn log lines with beat, logstah, elastic, kibana

**URL:** <https://discuss.elastic.co/t/elastic-stack-for-100-mn-log-lines-with-beat-logstah-elastic-kibana/59720>\
**Category:** Logstash\
**Created:** [September 3, 2016, 1:57am UTC](https://discuss.elastic.co/t/elastic-stack-for-100-mn-log-lines-with-beat-logstah-elastic-kibana/59720 "2016-09-03T01:57:16Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![userguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/userguy/32/6224_2.png) [@userguy](https://discuss.elastic.co/u/userguy)\
**Post date:** [September 3, 2016, 1:57am UTC](https://discuss.elastic.co/t/elastic-stack-for-100-mn-log-lines-with-beat-logstah-elastic-kibana/59720/1 "2016-09-03T01:57:16Z")

</div>

Hello team,  
We are getting around 900gb of logs per per day and i want to process on realtime but in logstash i am getting circuit breaker as of now.. We have used 10 servers 4 for logstash and 6 for elasticsearch.. Logstash has 2 grok pattern and location filter.. Elasticsearch has 6 servers where, 3 are master+data and 2are only data and one is neither master no data.. All elasticsearch servers have 31g of ram and no modification is done either at elasticsearch end and logstash end.. But as data is increasing we are getting too much circuit breaker exception and stall slowdown error.. What is best architecture we can have and what configuration we can do at filebeat end and also at elasticsearch and at logstash end #elasticsearch #logstash #kibana #beats:filebeat as of now if i see at marvel i donot see indexing rate more than 8000/sec and also does any one suggest on messaging queue like kafka as in like beat \> logstash with no filter-\>kafka with 2 day retention -\> logstash withh filter and grok-\>elasticsearch -\>kibana

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:40am UTC](https://discuss.elastic.co/t/elastic-stack-for-100-mn-log-lines-with-beat-logstah-elastic-kibana/59720/2 "2017-07-06T04:40:06Z")

</div>

Closed on July 6, 2017, 4:40am UTC