# What Hardware do I need for 100 GB Per day data volume?

**URL:** <https://discuss.elastic.co/t/what-hardware-do-i-need-for-100-gb-per-day-data-volume/138397>\
**Category:** Elasticsearch\
**Created:** [July 3, 2018, 1:05pm UTC](https://discuss.elastic.co/t/what-hardware-do-i-need-for-100-gb-per-day-data-volume/138397 "2018-07-03T13:05:04Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\
**Post date:** [July 3, 2018, 1:05pm UTC](https://discuss.elastic.co/t/what-hardware-do-i-need-for-100-gb-per-day-data-volume/138397/1 "2018-07-03T13:05:04Z")

</div>

I am setting up a cluster and i have following scenario

1. Data Volume 100GB per day (with 3 month retention )
2. Search scenarios (dashboards only mix, of analyze and not analyzed fields max 10 users)
3. I am going to use 8 servers

3 master eligible nodes (6 GB RAM , 2 Core CPU, 50 GB Hard Disk each each)  
2 Data Nodes (64 GB RAM , 8 Core CPU, 10 TB (total to retain data for 3 months) )  
1 coordinate node (6 GB RAM , 2 Core CPU, 60GB Hard Drive)  
1 LS machine (8GB RAM, 2 CPU, 50 GB hard drive)  
1 Kibana (8GB RAM, 2 CPU, 50 GB hard drive)

Is this compute good for my scenario ?

Regards  
VG

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [July 4, 2018, 2:08pm UTC](https://discuss.elastic.co/t/what-hardware-do-i-need-for-100-gb-per-day-data-volume/138397/2 "2018-07-04T14:08:19Z")

</div>

Hi,

your hw pool is much different from mine but here are some thought...

Not sure if you really need the dedicated _coordinating node_ but maybe there is some benefit from having one...

As for the data nodes, that is a lot of storage for one node compared to what I have. I have about 7TB of data and total heap usage is 160GB. I have 20 nodes with about 20GB heap each. That does not mean that what you have will not work. More nodes and more indices/shards increase heap usage and I have a fair bit of indices and shards.

If high availability is not an issue then one LS machine might be enough. How much work does LS do, do you use a lot of filters? I guess that is something for the LS board...

Theoretically you waste the least amount of resources by having as few nodes and indices/shards as possible, so maybe with a good sharding strategy it will work well. Also depends on how complex the documents and queries are.

Just some thought 🙂

---

<div class="post-metadata">

**Author:** ![vikas\_gopal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikas_gopal/32/47661_2.png) [@vikas\_gopal](https://discuss.elastic.co/u/vikas_gopal)\
**Post date:** [July 5, 2018, 9:14am UTC](https://discuss.elastic.co/t/what-hardware-do-i-need-for-100-gb-per-day-data-volume/138397/3 "2018-07-05T09:14:49Z")

</div>

Thanks for the reply , so you mean to say I need more data nodes for this volume ? Sorry for the confusion here 10 TB is the total disk i.e 5 TB each node.  
Now Idea behind to have separate coordinate node is to hit Kibana to this node only as this is the recommendation form Elasticsearch.

Regards  
VG

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [July 5, 2018, 9:53am UTC](https://discuss.elastic.co/t/what-hardware-do-i-need-for-100-gb-per-day-data-volume/138397/4 "2018-07-05T09:53:42Z")

</div>

Hi VG,

your setup is so different from mine so I don't really know if you need more data nodes or not... I guess you will run ES with the max recommended heap of ~32GB? That might be enough for your use case. That will leave the rest for Lucene and os, etc. Someone with a more similar setup can give you better advice on that.

-AB

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 2, 2018, 9:53am UTC](https://discuss.elastic.co/t/what-hardware-do-i-need-for-100-gb-per-day-data-volume/138397/5 "2018-08-02T09:53:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
