# Need help to find ELK config required for 30 lakh logs in one minute

**URL:** <https://discuss.elastic.co/t/need-help-to-find-elk-config-required-for-30-lakh-logs-in-one-minute/147662>\
**Category:** Elasticsearch\
**Created:** [September 7, 2018, 6:34am UTC](https://discuss.elastic.co/t/need-help-to-find-elk-config-required-for-30-lakh-logs-in-one-minute/147662 "2018-09-07T06:34:19Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sumeet\_dembra](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sumeet_dembra/32/56190_2.png) [@sumeet\_dembra](https://discuss.elastic.co/u/sumeet_dembra)\
**Post date:** [September 7, 2018, 6:34am UTC](https://discuss.elastic.co/t/need-help-to-find-elk-config-required-for-30-lakh-logs-in-one-minute/147662/1 "2018-09-07T06:34:20Z")

</div>

Hello,

We are having servers in our system which are going to produce around 30 lakh log lines every minute. These logs need to be inserted into ES through logstash.  
We have some regex's written in logstash. Logs are pushed to logstash from filebeat.  
Can someone please suggest us how many nodes/replicas/shards will be required for such a project.

Thanks in advance.

-Sumeet

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 7, 2018, 6:56am UTC](https://discuss.elastic.co/t/need-help-to-find-elk-config-required-for-30-lakh-logs-in-one-minute/147662/2 "2018-09-07T06:56:53Z")

</div>

This will depend a lot on the use-case, size and complexity of documents, retention period, the mappings you use as well as the type of hardware you are going to deploy on. There is no way to estimate size based on the data you have provided. The best way to find out is to benchmark, and you may find the following resources useful:

> **[How many shards should I have in my Elasticsearch cluster?
	  	 | Elastic](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster)**
>
> Elasticsearch is a very versatile platform, that supports a variety of use cases, and provides great flexibility around data organisation and replication strategies. This flexibility can however somet...

> **[Sizing Hot-Warm Architectures for Logging and Metrics in the Elasticsearch...](https://www.elastic.co/blog/sizing-hot-warm-architectures-for-logging-and-metrics-in-the-elasticsearch-service-on-elastic-cloud)**
>
> These are exciting times! Elasticsearch Service on Elastic Cloud recently added support for a wide range of hardware choices and deployment templates, which makes it perfectly suited for efficient han...

[https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right](https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right)

[https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing](https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 5, 2018, 6:57am UTC](https://discuss.elastic.co/t/need-help-to-find-elk-config-required-for-30-lakh-logs-in-one-minute/147662/3 "2018-10-05T06:57:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.