# Size of an index is very less than the raw document size when ingested via logstash

**URL:** <https://discuss.elastic.co/t/size-of-an-index-is-very-less-than-the-raw-document-size-when-ingested-via-logstash/280812>\
**Category:** Logstash\
**Created:** [August 9, 2021, 1:09pm UTC](https://discuss.elastic.co/t/size-of-an-index-is-very-less-than-the-raw-document-size-when-ingested-via-logstash/280812 "2021-08-09T13:09:44Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![madhanbaskar](https://avatars.discourse-cdn.com/v4/letter/m/f17d59/32.png) [@madhanbaskar](https://discuss.elastic.co/u/madhanbaskar)\
**Post date:** [August 9, 2021, 1:09pm UTC](https://discuss.elastic.co/t/size-of-an-index-is-very-less-than-the-raw-document-size-when-ingested-via-logstash/280812/1 "2021-08-09T13:09:44Z")

</div>

I have a JSON file which has 93k records in it. The size of the JSON file is 413MB. When Indexed via logstash it is just 88MB in the index. I could see all the records ingested.

Index setting :  
I have 50 fields for indexing and 200 fields are not indexed and the same is configured in the index template.

I have \_source configured. No replicas configured. Removed message fields in logstash. I have tried benchmarking with the above settings and could see huge difference in the index size vs raw doc size.

Can someone help me understand how there is a big difference in the size?

Thanks!

---

<div class="post-metadata">

**Author:** ![madhanbaskar](https://avatars.discourse-cdn.com/v4/letter/m/f17d59/32.png) [@madhanbaskar](https://discuss.elastic.co/u/madhanbaskar)\
**Post date:** [August 9, 2021, 1:14pm UTC](https://discuss.elastic.co/t/size-of-an-index-is-very-less-than-the-raw-document-size-when-ingested-via-logstash/280812/2 "2021-08-09T13:14:51Z")

</div>

@warkolm - Could you please help me here?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [August 9, 2021, 1:51pm UTC](https://discuss.elastic.co/t/size-of-an-index-is-very-less-than-the-raw-document-size-when-ingested-via-logstash/280812/3 "2021-08-09T13:51:33Z")

</div>

> [@madhanbaskar](#):
>
> Can someone help me understand how there is a big difference in the size?

Elasticsearch compresses the data on ingest. See [Here](https://www.elastic.co/guide/en/elasticsearch/reference/7.14/index-modules.html#_static_index_settings)  
`index.codec`

Certain types of text data compresses very well.

Elasticsearch is very efficient with the fields indexed.

It looks like you cleaned up / removed non-required field / data.

Are you seeing any issues in the data when it is retrieved? Otherwise sounds like you are in good shape.

---

<div class="post-metadata">

**Author:** ![madhanbaskar](https://avatars.discourse-cdn.com/v4/letter/m/f17d59/32.png) [@madhanbaskar](https://discuss.elastic.co/u/madhanbaskar)\
**Post date:** [August 9, 2021, 2:23pm UTC](https://discuss.elastic.co/t/size-of-an-index-is-very-less-than-the-raw-document-size-when-ingested-via-logstash/280812/4 "2021-08-09T14:23:23Z")

</div>

Thanks Stephen!  
As of now, I dont see any issues in the data when retrieved.  
Great that I'm in a good shape. Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 6, 2021, 2:24pm UTC](https://discuss.elastic.co/t/size-of-an-index-is-very-less-than-the-raw-document-size-when-ingested-via-logstash/280812/5 "2021-09-06T14:24:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.