# \_source field storage (\_source Field Overview)

**URL:** <https://discuss.elastic.co/t/source-field-storage-source-field-overview/340729>\
**Category:** Elasticsearch\
**Created:** [August 14, 2023, 9:37am UTC](https://discuss.elastic.co/t/source-field-storage-source-field-overview/340729 "2023-08-14T09:37:13Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![ksaimohan2k](https://avatars.discourse-cdn.com/v4/letter/k/f4b2a3/32.png) [@ksaimohan2k](https://discuss.elastic.co/u/ksaimohan2k)\
**Post date:** [August 14, 2023, 9:37am UTC](https://discuss.elastic.co/t/source-field-storage-source-field-overview/340729/1 "2023-08-14T09:37:13Z")

</div>

Hello,

We just want to have a clear understanding of the \_source field. Is it going to be indexed or stored? if it is stored where it is going to be stored.

As an experiment, we disabled the \_source using PUT index\_name/\_mapping {enable=false}, and we were unable to see the source in the discovery page or retrieve it using devtools GET index\_name/\_search, but we didn't see any changes in the storage size of the Index. So we want to know if, when we disable the \_source, it will delete the \_source or just stop retrieving it.

In another scenario, I applied the mapper-size plugin to one index and calculated the sum of the \_source field size for all the indexed documents. The result is much bigger than the index file.

Is \_source stored somewhere else or stored within the index?

Thanks In advance !

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 14, 2023, 12:55pm UTC](https://discuss.elastic.co/t/source-field-storage-source-field-overview/340729/2 "2023-08-14T12:55:04Z")

</div>

> [@ksaimohan2k](#):
>
> We just want to have a clear understanding of the \_source field. Is it going to be indexed or stored?

This is explained in the [documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-source-field.html#mapping-source-field), the `_source` field is stored in the index, but it is not indexed.

> [@ksaimohan2k](#):
>
> As an experiment, we disabled the \_source using PUT index\_name/\_mapping {enable=false}, and we were unable to see the source in the discovery page or retrieve it using devtools GET index\_name/\_search, but we didn't see any changes in the storage size of the Index.

What was the size of the index and what was the sample document you used? If you had just a couple of documents or if your sample document is small you will barely see any differences, but as soon as the indice starts to grow (tens of GB) you will start to see the diference.

> [@ksaimohan2k](#):
>
> In another scenario, I applied the mapper-size plugin to one index and calculated the sum of the \_source field size for all the indexed documents. The result is much bigger than the index file.

I do not use the mapper-size plugin, but if I'm not wrong it will store the size of the original `_source` field, but this is without compression, when the data is stored it will be compressed.

If you want to save space I would recommend that first you try to change the `index.codec` to `best_compression` instead of disabling the `_source` field.

---

<div class="post-metadata">

**Author:** ![ksaimohan2k](https://avatars.discourse-cdn.com/v4/letter/k/f4b2a3/32.png) [@ksaimohan2k](https://discuss.elastic.co/u/ksaimohan2k)\
**Post date:** [August 14, 2023, 1:42pm UTC](https://discuss.elastic.co/t/source-field-storage-source-field-overview/340729/3 "2023-08-14T13:42:20Z")

</div>

Thanks, @leandrojmp, for the information; it was really helpful.

Just for my understanding, should I just assume that the 2.17 GB of my index is made up of the fields and the \_source?

For your understading, our index size was 2.17 GB when we deleted the \_source, and we didn't see even a single percent change in the storage size.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 14, 2023, 2:38pm UTC](https://discuss.elastic.co/t/source-field-storage-source-field-overview/340729/4 "2023-08-14T14:38:36Z")

</div>

> [@ksaimohan2k](#):
>
> For your understading, our index size was 2.17 GB when we deleted the \_source, and we didn't see even a single percent change in the storage size.

How did you deleted the `_source`? This is a mapping changing, I'm not sure this can be changed on already existing indices or even if it can, I don't think that this will have any impact to already ingested documents.

Also, 2,17 GB is pretty small, not sure you will see a big difference on the size.

---

<div class="post-metadata">

**Author:** ![ksaimohan2k](https://avatars.discourse-cdn.com/v4/letter/k/f4b2a3/32.png) [@ksaimohan2k](https://discuss.elastic.co/u/ksaimohan2k)\
**Post date:** [August 15, 2023, 9:57am UTC](https://discuss.elastic.co/t/source-field-storage-source-field-overview/340729/5 "2023-08-15T09:57:28Z")

</div>

We just disabled the \_source. PUT index\_name/\_mapping {enable=false}

Thank you for clarifying. Your insights are appreciated.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [August 15, 2023, 10:25am UTC](https://discuss.elastic.co/t/source-field-storage-source-field-overview/340729/6 "2023-08-15T10:25:33Z")

</div>

If you have questions about the disk usage of individual fields in an index, it is simplest to use the [analyze index disk usage API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-disk-usage.html) to investigate more deeply.

---

<div class="post-metadata">

**Author:** ![ksaimohan2k](https://avatars.discourse-cdn.com/v4/letter/k/f4b2a3/32.png) [@ksaimohan2k](https://discuss.elastic.co/u/ksaimohan2k)\
**Post date:** [August 15, 2023, 10:51am UTC](https://discuss.elastic.co/t/source-field-storage-source-field-overview/340729/7 "2023-08-15T10:51:26Z")

</div>

Thanks for the info, @DavidTurner. I will look into that.

---

<div class="post-metadata">

**Author:** ![ksaimohan2k](https://avatars.discourse-cdn.com/v4/letter/k/f4b2a3/32.png) [@ksaimohan2k](https://discuss.elastic.co/u/ksaimohan2k)\
**Post date:** [August 15, 2023, 11:32am UTC](https://discuss.elastic.co/t/source-field-storage-source-field-overview/340729/8 "2023-08-15T11:32:11Z")

</div>

Hello @DavidTurner and @leandrojmp  
Just for the confirmation i am asking,

can I assume that the \_source (stored) and extracted fields ( indexed) are resided in the same index file?

For example an index size is 2GB and it has 60 million events, if I use size plugin, it shows average size of the document is 2KB, when I calculate the size of the index by using the formula of (Index Size in MB/Number of documents\*1024) I get the average size of 0.33 KB. Does that mean the original \_source is compressed from 2KB to 0.33KB?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [August 15, 2023, 11:47am UTC](https://discuss.elastic.co/t/source-field-storage-source-field-overview/340729/9 "2023-08-15T11:47:17Z")

</div>

There's no need to assume anything, just use the API I linked above.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 12, 2023, 11:48am UTC](https://discuss.elastic.co/t/source-field-storage-source-field-overview/340729/10 "2023-09-12T11:48:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
