# Hardware for ELK

**URL:** <https://discuss.elastic.co/t/hardware-for-elk/126277>\
**Category:** Elasticsearch\
**Created:** [March 30, 2018, 7:39pm UTC](https://discuss.elastic.co/t/hardware-for-elk/126277 "2018-03-30T19:39:03Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mahroon](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@Mahroon](https://discuss.elastic.co/u/Mahroon)\
**Post date:** [March 30, 2018, 7:39pm UTC](https://discuss.elastic.co/t/hardware-for-elk/126277/1 "2018-03-30T19:39:03Z")

</div>

Hi ES Experts,  
I am new to Elasticsearch and have following use case to please suggest me to the right direction.  
Daily Data volume :- 400+ TB Daily.

How many node, cluster, index, shards and replicas do I need. Additionally, our servers will be Linux and it would be great if you advise us to have the correct hardware that can hold and control the big data that we receive CPU, Core, RAM, SSD or HDD

I hope I am clear with my use case. Can you please give me some suggestions?

I look forward to hearing from you in the near future.  
Regards,  
Mahroon

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 31, 2018, 7:08am UTC](https://discuss.elastic.co/t/hardware-for-elk/126277/2 "2018-03-31T07:08:19Z")

</div>

400+ TB of data per day is a lot, so is likely to involve a significant amount of hardware. You have also not described what type of data you have nor how long you need to keep it or how you are going to search/analyze it, which are all important factors.

---

<div class="post-metadata">

**Author:** ![Mahroon](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@Mahroon](https://discuss.elastic.co/u/Mahroon)\
**Post date:** [April 1, 2018, 9:08am UTC](https://discuss.elastic.co/t/hardware-for-elk/126277/3 "2018-04-01T09:08:09Z")

</div>

Thanks for getting back to me Chris.  
Back to your questions:

1. Our data type is structure/unstructured text data in JSON files.
2. Search and Analyze is searching and aggregation.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 1, 2018, 9:31am UTC](https://discuss.elastic.co/t/hardware-for-elk/126277/4 "2018-04-01T09:31:26Z")

</div>

How long do you need to keep data in the cluster?

Is the raw data format JSON? What is the average event size?

How many concurrent users are expected? Will you be using Kibana?

---

<div class="post-metadata">

**Author:** ![Mahroon](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@Mahroon](https://discuss.elastic.co/u/Mahroon)\
**Post date:** [April 1, 2018, 10:14am UTC](https://discuss.elastic.co/t/hardware-for-elk/126277/5 "2018-04-01T10:14:59Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> How long do you need to keep data in the cluster?

Thasnk, can you explain what do u mean by how long do u keep data in the cluster?  
The format has not build yet.  
We will use Kibana but users not yet?.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 1, 2018, 12:09pm UTC](https://discuss.elastic.co/t/hardware-for-elk/126277/6 "2018-04-01T12:09:20Z")

</div>

> [@Mahroon](#):
>
> Thasnk, can you explain what do u mean by how long do u keep data in the cluster?

I am asking about [retention period](https://en.wikipedia.org/wiki/Retention_period).

---

<div class="post-metadata">

**Author:** ![Mahroon](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@Mahroon](https://discuss.elastic.co/u/Mahroon)\
**Post date:** [April 9, 2018, 5:22pm UTC](https://discuss.elastic.co/t/hardware-for-elk/126277/7 "2018-04-09T17:22:54Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> you explain what do u mean by how lo

Hi Chris,

Sorry for the late reply. we are plan to have data for one year.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 9, 2018, 6:05pm UTC](https://discuss.elastic.co/t/hardware-for-elk/126277/8 "2018-04-09T18:05:10Z")

</div>

if you ingest 400TB a day and keep this for a year, that is roughly 143PB of raw data. I doubt you will find anyone here that can give you an estimate for a use case that size, so you probably need to roll up your sleeves and do some benchmarking...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 7, 2018, 6:05pm UTC](https://discuss.elastic.co/t/hardware-for-elk/126277/9 "2018-05-07T18:05:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
