# Elastic cluster capacity planning

**URL:** <https://discuss.elastic.co/t/elastic-cluster-capacity-planning/164880>\
**Category:** Elasticsearch\
**Created:** [January 19, 2019, 6:49am UTC](https://discuss.elastic.co/t/elastic-cluster-capacity-planning/164880 "2019-01-19T06:49:56Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![vivektsb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vivektsb/32/49029_2.png) [@vivektsb](https://discuss.elastic.co/u/vivektsb)\
**Post date:** [January 19, 2019, 6:49am UTC](https://discuss.elastic.co/t/elastic-cluster-capacity-planning/164880/1 "2019-01-19T06:49:56Z")

</div>

Hi,

We have requirement to index around 8TB data per day including replica( 4TB per day)

We are planning for 12 nodes cluster each with 8 core, 30TB Hdd,64gb ram out of 5 will be master nodes with SSD.  
Do we need to use jbod or raid? As we have replica jbod is sufficient please correct us if we are wrong?

We have one logstash instance with 16 core,64 GB ram,5 TB Hdd .Each index with 2 primary shards and 1replica.Is that correct configuration for moderate querying.One logstash instance is sufficient or do we need to use redis or kafka for fault tolerance.

Please let us know elastic and logstash configuration is proper.

Regards,  
Vivek

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [January 19, 2019, 9:44am UTC](https://discuss.elastic.co/t/elastic-cluster-capacity-planning/164880/2 "2019-01-19T09:44:01Z")

</div>

Indexing in Elasticsearch is very I/O intensive, so for best performance [it is recommended to use SSDs](https://www.elastic.co/guide/en/elasticsearch/reference/6.5/tune-for-indexing-speed.html#_use_faster_hardware). If you are indexing into spinning disks it is important that you spread out the indexing load across as many disks as possible, e.g. by striping them.

It is difficult to determine exactly how much data a node can index and store, so I would recommend you perform some benchmarks if you have the hardware available. I would also recommend the following resources around sizing and best practices:

> **[Sizing Hot-Warm Architectures for Logging and Metrics in the Elasticsearch...](https://www.elastic.co/blog/sizing-hot-warm-architectures-for-logging-and-metrics-in-the-elasticsearch-service-on-elastic-cloud)**
>
> Want to learn more about the differences between the Amazon Elasticsearch Service and our official Elasticsearch Service? Visit our AWS Elasticsearch comparison page.These are exciting times! Elastics...

> **[How many shards should I have in my Elasticsearch cluster?
	  	 | Elastic](https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster)**
>
> Elasticsearch is a very versatile platform, that supports a variety of use cases, and provides great flexibility around data organisation and replication strategies. This flexibility can however somet...

[https://www.elastic.co/webinars/optimizing-storage-efficiency-in-elasticsearch](https://www.elastic.co/webinars/optimizing-storage-efficiency-in-elasticsearch)

[https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing](https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing)

[https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right](https://www.elastic.co/webinars/using-rally-to-get-your-elasticsearch-cluster-size-right)

---

<div class="post-metadata">

**Author:** ![vivektsb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vivektsb/32/49029_2.png) [@vivektsb](https://discuss.elastic.co/u/vivektsb)\
**Post date:** [January 20, 2019, 7:39am UTC](https://discuss.elastic.co/t/elastic-cluster-capacity-planning/164880/3 "2019-01-20T07:39:24Z")

</div>

Hi Christian,

Thanks for your quick response.Do we need to use redis or kafka as we have only single logstash instance for failover conditions.

Regards,  
Vivek

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 17, 2019, 7:39am UTC](https://discuss.elastic.co/t/elastic-cluster-capacity-planning/164880/4 "2019-02-17T07:39:26Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.