# Reducing Index Footprint

**URL:** <https://discuss.elastic.co/t/reducing-index-footprint/250318>\
**Category:** Elasticsearch\
**Created:** [September 29, 2020, 8:51am UTC](https://discuss.elastic.co/t/reducing-index-footprint/250318 "2020-09-29T08:51:10Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![shmoolik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shmoolik/32/76360_2.png) [@shmoolik](https://discuss.elastic.co/u/shmoolik)\
**Post date:** [September 29, 2020, 8:51am UTC](https://discuss.elastic.co/t/reducing-index-footprint/250318/1 "2020-09-29T08:51:10Z")

</div>

Hi there!

My team is looking for ways to optimize large elasticsearch indices by reducing the index footprint.

I read this article about [Dynamic Mapping](https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-mapping.html) and was wondering if it can be used in order to reduce index footprint (I know that this is also the default mapping configuration in ElasticSearch).  
By using Dynamic Mapping, we add certain fields only to documents that really use it (rather than adding it to all documents).  
For cases where most documents don't use a certain property we can potentially reduce index size significantly, right?

So if - for example - we may have a basic document mapping that includes "first\_name", "last\_name" and "main\_email". However, some documents (5%) may also include another field "additional\_email".  
So why have 4 fields for all documents where we can same quite a lot of space using dynamic mapping (95% of the documents need only 3 fields), right?

Also, using [Dynamic Templates](https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-templates.html) we can ensure that the mapping type for dynamically added fields is predefined rather than automatically selected by ElasticSearch (used for "additional\_email" in our example above).

Is this use-case valid?  
Is this really a way to reduce index footprint?  
Are there other effective ways to reduce index footprint?

Thanks!  
Shmoolik

---

<div class="post-metadata">

**Author:** ![lusynda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusynda/32/53557_2.png) [@lusynda](https://discuss.elastic.co/u/lusynda)\
**Post date:** [September 29, 2020, 9:11am UTC](https://discuss.elastic.co/t/reducing-index-footprint/250318/2 "2020-09-29T09:11:18Z")

</div>

I don't really know about Dynamic template but you can use logstash to filter out the field that you don't need. But if you are not using logstash then sorry i can't provide any thing helpful.

The way i check are if there are data in the field additional\_email then index, if not then remove the field, quite easy actually.

---

<div class="post-metadata">

**Author:** ![shmoolik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shmoolik/32/76360_2.png) [@shmoolik](https://discuss.elastic.co/u/shmoolik)\
**Post date:** [September 29, 2020, 9:42am UTC](https://discuss.elastic.co/t/reducing-index-footprint/250318/3 "2020-09-29T09:42:25Z")

</div>

Thanks @lusynda for your reply!  
We're not using Logstash so this doesn't seem to be relevant.

Would love to get an "insider" answer for this, since I guess this is more of an elasticsearch internal implementation question... **Can _Dynamic Mapping_ potentially reduce index footprint**.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 29, 2020, 9:48am UTC](https://discuss.elastic.co/t/reducing-index-footprint/250318/4 "2020-09-29T09:48:12Z")

</div>

The mappings and index settings you use [will affect the index size](https://www.elastic.co/guide/en/elasticsearch/reference/7.9/tune-for-disk-usage.html). Dynamic mappings just instructs Elasticsearch how to map new fields so does in itself not affect size although the mapping it is configured to apply does.

---

<div class="post-metadata">

**Author:** ![shmoolik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shmoolik/32/76360_2.png) [@shmoolik](https://discuss.elastic.co/u/shmoolik)\
**Post date:** [September 29, 2020, 10:40am UTC](https://discuss.elastic.co/t/reducing-index-footprint/250318/5 "2020-09-29T10:40:06Z")

</div>

Thanks a lot for your answer and for the link @Christian_Dahlqvist!

I'm not sure I completely understand - please let me rephrase my question using an example.  
Given that I need elasticsearch to perform indexing on all fields, please consider the following use-cases:

**Use-case 1**  
We use **static mapping** and push 1M documents.  
The static mapping defines that each document has the following fields: "first\_name", "last\_name" and "email".  
The index footprint is now X.  
Now, we insert 1 more document.  
It is understood that the index footprint will be X+1 (no re-indexing is required).

**Use-case 2**  
We use **dynamic mapping** and push 1M documents.  
Each of these million documents was inserted with the following fields: "first\_name" and "last\_name" (and only these fields).  
The index footprint is now Y.  
Now, we insert 1 more document that has the "first\_name" and "last\_name" fields, but adds also a new field "email".  
How will this affect the index footprint?  
Will the index footprint increase only by 1 document, or will all 1M documents be updated (re-indexed) to include the new "email" field (with null value)?  
Given that all documents from both use-cases are the same, which one is larger in size - the last footprint of use-case 1 or the last footprint of use-case 2?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 29, 2020, 10:44am UTC](https://discuss.elastic.co/t/reducing-index-footprint/250318/6 "2020-09-29T10:44:56Z")

</div>

Elasticsearch supports sparse documents and only indexes the fields that are present in the document. Whether you use dynamic or static mapping here is irrelevant.

---

<div class="post-metadata">

**Author:** ![shmoolik](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shmoolik/32/76360_2.png) [@shmoolik](https://discuss.elastic.co/u/shmoolik)\
**Post date:** [September 29, 2020, 11:57am UTC](https://discuss.elastic.co/t/reducing-index-footprint/250318/7 "2020-09-29T11:57:04Z")

</div>

Thank you @Christian_Dahlqvist for your prompt response.  
It is much clearer now.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 27, 2020, 11:57am UTC](https://discuss.elastic.co/t/reducing-index-footprint/250318/8 "2020-10-27T11:57:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
