# Storage quantity required for an Elastic SIEM solution

**URL:** <https://discuss.elastic.co/t/storage-quantity-required-for-an-elastic-siem-solution/253582>\
**Category:** Elasticsearch\
**Created:** [October 28, 2020, 3:24pm UTC](https://discuss.elastic.co/t/storage-quantity-required-for-an-elastic-siem-solution/253582 "2020-10-28T15:24:23Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![jelocabral](https://avatars.discourse-cdn.com/v4/letter/j/e0b2c6/32.png) [@jelocabral](https://discuss.elastic.co/u/jelocabral)\
**Post date:** [October 28, 2020, 3:24pm UTC](https://discuss.elastic.co/t/storage-quantity-required-for-an-elastic-siem-solution/253582/1 "2020-10-28T15:24:23Z")

</div>

Hi people, we have an ELK stack for a SIEM solution in our company.

We need to ingest data from 10 devices aproximately, and we need to maintan the data for 3 months. We have one firewall which is a heavy log sender.

Can you tell me how much storage do we need?

Or maybe you can tell me how much storage you use in your implementations, in order to decide to use our corporate storage or to buy a dedicated one.

Thank you very much.

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [October 28, 2020, 3:42pm UTC](https://discuss.elastic.co/t/storage-quantity-required-for-an-elastic-siem-solution/253582/2 "2020-10-28T15:42:08Z")

</div>

How much raw data your devices will generate on a daily basis ? 2To/day for example ?

---

<div class="post-metadata">

**Author:** ![jelocabral](https://avatars.discourse-cdn.com/v4/letter/j/e0b2c6/32.png) [@jelocabral](https://discuss.elastic.co/u/jelocabral)\
**Post date:** [October 29, 2020, 1:50pm UTC](https://discuss.elastic.co/t/storage-quantity-required-for-an-elastic-siem-solution/253582/3 "2020-10-29T13:50:19Z")

</div>

Hi, aproximately 100 GB per day, but in this situation I will need:

30 x 100 GB = 3 TB per month

and 9 TB for a 3-month-period

So we need to buy a new storage a think.

What is yoor situation? Do you work with raw data or do you try to get only relevent fields using a Logstash pipeline or something like this, in order to reduce the data stored?

Thanks again.

---

<div class="post-metadata">

**Author:** ![ylasri](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ylasri/32/86120_2.png) [@ylasri](https://discuss.elastic.co/u/ylasri)\
**Post date:** [October 29, 2020, 2:11pm UTC](https://discuss.elastic.co/t/storage-quantity-required-for-an-elastic-siem-solution/253582/4 "2020-10-29T14:11:19Z")

</div>

You can opt for a [Hot/Warm/Cold](https://www.elastic.co/blog/sizing-hot-warm-architectures-for-logging-and-metrics-in-the-elasticsearch-service-on-elastic-cloud) architecture  
Let's say following config (This is including 1 replicas) :

- 7 days hot data : 2 nodes with 2Tb SSD Storage
- 30 days warm data : 2 nodes with 8Tb SAS Storage
- 60 days cold data : 2 nodes with 16Tb HDD Storage (you can have 0 replica here)

You can refer to this [presentation](https://www.elastic.co/elasticon/conf/2016/sf/quantitative-cluster-sizing) on how to size your cluster

Raw data converted to JSON will double the volume =\> 200 Gb/day of json  
If you clean your data with logstash and you customize your mapping then indexing process will reduce the json size by a factor of 0.5 may be ... then you can say that you will index 100Gb/day of data, considering replicas will double the disk size

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/b/cb8f838ea3b26402d55062c4ded90171cf18fec1.png)

---

<div class="post-metadata">

**Author:** ![jelocabral](https://avatars.discourse-cdn.com/v4/letter/j/e0b2c6/32.png) [@jelocabral](https://discuss.elastic.co/u/jelocabral)\
**Post date:** [October 29, 2020, 6:22pm UTC](https://discuss.elastic.co/t/storage-quantity-required-for-an-elastic-siem-solution/253582/5 "2020-10-29T18:22:03Z")

</div>

Dear, your explanation is fantastic, I have to investigate this way.

Thanks a lot again and maybe I will ask you again if I have any new doubt.

Regards.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 26, 2020, 6:22pm UTC](https://discuss.elastic.co/t/storage-quantity-required-for-an-elastic-siem-solution/253582/6 "2020-11-26T18:22:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
