# What is the ratio between raw data and ingested data that is stored in Elastic cluster

**URL:** <https://discuss.elastic.co/t/what-is-the-ratio-between-raw-data-and-ingested-data-that-is-stored-in-elastic-cluster/327945>\
**Category:** Elasticsearch\
**Created:** [March 17, 2023, 3:03pm UTC](https://discuss.elastic.co/t/what-is-the-ratio-between-raw-data-and-ingested-data-that-is-stored-in-elastic-cluster/327945 "2023-03-17T15:03:17Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![hiruni.insyncit.net](https://avatars.discourse-cdn.com/v4/letter/h/c68b51/32.png) [@hiruni.insyncit.net](https://discuss.elastic.co/u/hiruni.insyncit.net)\
**Post date:** [March 17, 2023, 3:03pm UTC](https://discuss.elastic.co/t/what-is-the-ratio-between-raw-data-and-ingested-data-that-is-stored-in-elastic-cluster/327945/1 "2023-03-17T15:03:17Z")

</div>

Hi,

I want to know what the ratio is between raw data and ingested data that is stored in Elastic cluster.

I know raw logs and ingested logs are not same in size.  
Also is there any way to find the incoming raw log volume to the cluster?

I could calculate ingested log volume using index sizes.

Thank you...!

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [March 17, 2023, 3:12pm UTC](https://discuss.elastic.co/t/what-is-the-ratio-between-raw-data-and-ingested-data-that-is-stored-in-elastic-cluster/327945/2 "2023-03-17T15:12:42Z")

</div>

There is no fixed ratio as it will depend entirely on how your data looks like and your mapping.

You will need to test yourself with the data you are planning to index and the mapping.

This [blog post](https://www.elastic.co/blog/save-space-and-money-with-improved-storage-efficiency-in-elasticsearch-7-10) is a little old, but explains a little how compression works in Elasticsearch, note that things already improved on recent versions.

---

<div class="post-metadata">

**Author:** ![hiruni.insyncit.net](https://avatars.discourse-cdn.com/v4/letter/h/c68b51/32.png) [@hiruni.insyncit.net](https://discuss.elastic.co/u/hiruni.insyncit.net)\
**Post date:** [March 24, 2023, 3:51pm UTC](https://discuss.elastic.co/t/what-is-the-ratio-between-raw-data-and-ingested-data-that-is-stored-in-elastic-cluster/327945/3 "2023-03-24T15:51:12Z")

</div>

Hi Leandro,

Thank you for the reply.  
Actually I don't need exact ratio. I want to get the raw log data volume per day for Elastic On-prem.  
Is there any way to get the raw log volume, which would be great?

I'm using Elasticsearch 8.1.2

Thank you...!  
Hiruni

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [March 24, 2023, 4:20pm UTC](https://discuss.elastic.co/t/what-is-the-ratio-between-raw-data-and-ingested-data-that-is-stored-in-elastic-cluster/327945/4 "2023-03-24T16:20:41Z")

</div>

If you install the [mapper size plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/current/mapper-size.html) you can use this to aggregate on the ingested message size and get an estimate that way.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 21, 2023, 4:21pm UTC](https://discuss.elastic.co/t/what-is-the-ratio-between-raw-data-and-ingested-data-that-is-stored-in-elastic-cluster/327945/5 "2023-04-21T16:21:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.