# Allowing unlimited dynamic fields ( dynamic fields based on new data ) into single index . Is this efficient?

**URL:** <https://discuss.elastic.co/t/allowing-unlimited-dynamic-fields-dynamic-fields-based-on-new-data-into-single-index-is-this-efficient/217049>\
**Category:** Elasticsearch\
**Created:** [January 29, 2020, 5:15pm UTC](https://discuss.elastic.co/t/allowing-unlimited-dynamic-fields-dynamic-fields-based-on-new-data-into-single-index-is-this-efficient/217049 "2020-01-29T17:15:21Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sarath\_r\_nair](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sarath_r_nair/32/61664_2.png) [@sarath\_r\_nair](https://discuss.elastic.co/u/sarath_r_nair)\
**Post date:** [January 29, 2020, 5:15pm UTC](https://discuss.elastic.co/t/allowing-unlimited-dynamic-fields-dynamic-fields-based-on-new-data-into-single-index-is-this-efficient/217049/1 "2020-01-29T17:15:21Z")

</div>

In our use case, ( ES 7.4.0 ) we will iterate over multiple data ( text, pdf, etc ). Assume we have  
(name, status, salary, designation, location) are necessary and mandatory fields we require from each data.  
Apart from this each data might have their own fields/information. ( etc doc1 might have extra information like ( hobbies, address ) . doc2 might have ( hobbies, gender, no\_kids ) etc. In the real use case these informations are a lot and vary.

The problem is, now we are indexing all these fields into the same index. The no of fields are getting more in number. But, our core algorithm which uses this index, is only making use of the (name, status, salary, designation, location) this information. Other extra information is not that useful as of now.

Is the current approach of allowing unlimited fields into index efficient?

---

<div class="post-metadata">

**Author:** ![loren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/loren/32/44942_2.png) [@loren](https://discuss.elastic.co/u/loren)\
**Post date:** [January 30, 2020, 12:15am UTC](https://discuss.elastic.co/t/allowing-unlimited-dynamic-fields-dynamic-fields-based-on-new-data-into-single-index-is-this-efficient/217049/2 "2020-01-30T00:15:07Z")

</div>

If you just keep adding in dynamic fields that end up changing the mapping, then no, it's not efficient. Do it enough and you will either bring down your cluster from the mapping explosion, or you will just harmlessly bump into the [`index.mapping.total_fields.limit`](https://www.elastic.co/guide/en/elasticsearch/reference/master/mapping.html#mapping-limit-settings).

Index just the fields you need, and stick everything else under some `object` property with `"enabled": false`

---

<div class="post-metadata">

**Author:** ![sarath\_r\_nair](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sarath_r_nair/32/61664_2.png) [@sarath\_r\_nair](https://discuss.elastic.co/u/sarath_r_nair)\
**Post date:** [January 30, 2020, 4:13am UTC](https://discuss.elastic.co/t/allowing-unlimited-dynamic-fields-dynamic-fields-based-on-new-data-into-single-index-is-this-efficient/217049/3 "2020-01-30T04:13:41Z")

</div>

Thank you so much @loren. I am adding it as a solution for time being. I will update the thread up on my findings and doubts.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 27, 2020, 4:14am UTC](https://discuss.elastic.co/t/allowing-unlimited-dynamic-fields-dynamic-fields-based-on-new-data-into-single-index-is-this-efficient/217049/4 "2020-02-27T04:14:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.