# 12K fields in the mapping

**URL:** <https://discuss.elastic.co/t/12k-fields-in-the-mapping/293340>\
**Category:** Elasticsearch\
**Created:** [January 3, 2022, 10:37am UTC](https://discuss.elastic.co/t/12k-fields-in-the-mapping/293340 "2022-01-03T10:37:21Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![orips](https://avatars.discourse-cdn.com/v4/letter/o/3ab097/32.png) [@orips](https://discuss.elastic.co/u/orips)\
**Post date:** [January 3, 2022, 10:37am UTC](https://discuss.elastic.co/t/12k-fields-in-the-mapping/293340/1 "2022-01-03T10:37:22Z")

</div>

Hi,

We're approaching 12K fields in our mapping. Our cluster does not have many documents (it's about 1 billion, and it's doc one per user, so it's not growing very rapidly).

I'd like to know: how much further can we grow the number of fields before reaching an unusable stage? Which metrics we'll see starting to degrade?

Another question: how can we redesign our cluster once we hit this limit? We are trying to avoid partitioning our customers to several clusters/indexes just to keep the mapping size low.

Thanks

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [January 4, 2022, 5:53am UTC](https://discuss.elastic.co/t/12k-fields-in-the-mapping/293340/2 "2022-01-04T05:53:44Z")

</div>

Welcome to our community! 😃

What's the reasoning behind having so many fields?

---

<div class="post-metadata">

**Author:** ![orips](https://avatars.discourse-cdn.com/v4/letter/o/3ab097/32.png) [@orips](https://discuss.elastic.co/u/orips)\
**Post date:** [January 6, 2022, 8:12am UTC](https://discuss.elastic.co/t/12k-fields-in-the-mapping/293340/3 "2022-01-06T08:12:08Z")

</div>

Hi,

The reason is that the index serves a lot for customers where each can define their own set of fields (up to 100) and we used a dynamic index for this.

---

<div class="post-metadata">

**Author:** ![mayya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mayya/32/83147_2.png) [@mayya](https://discuss.elastic.co/u/mayya)\
**Post date:** [January 9, 2022, 3:02pm UTC](https://discuss.elastic.co/t/12k-fields-in-the-mapping/293340/4 "2022-01-09T15:02:19Z")

</div>

Indeed, mapping explosion is a valid concern, it takes heap memory; it is a part of the cluster state that gets synchronized between nodes, and the bigger the cluster state, the slower synchronization can me. Also a size of mapping affects the speed of indexing of documents, because as you index a new document, this document's mapping is checked agains the existing index mapping.

For an index with many many fields, we recommend using ES [flattened field](https://www.elastic.co/guide/en/elasticsearch/reference/7.16/flattened.html). It has some limitations for search, but from the ES point of view it is a single field with all its subfields with their names and values are stored on disk, which allows to have "unlimited" number of subfields.

---

<div class="post-metadata">

**Author:** ![orips](https://avatars.discourse-cdn.com/v4/letter/o/3ab097/32.png) [@orips](https://discuss.elastic.co/u/orips)\
**Post date:** [January 10, 2022, 1:05pm UTC](https://discuss.elastic.co/t/12k-fields-in-the-mapping/293340/5 "2022-01-10T13:05:24Z")

</div>

Thanks. How does it compare to the method where you create an array like this:

```auto
[
    { "keyAndValue": "priority:urgent" },
    { "keyAndValue": "priority:high" }
]

```

in terms of performance?

---

<div class="post-metadata">

**Author:** ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)\
**Post date:** [January 10, 2022, 1:30pm UTC](https://discuss.elastic.co/t/12k-fields-in-the-mapping/293340/6 "2022-01-10T13:30:08Z")

</div>

it depends on what kind of queries you are using.  
If you filter or search on that field, keyAndValue may not be a good option in most cases and flattened field is better.  
If you filter or search only on other fields and just store and retrieve the data, you can use object field with `enabled: false`.

> **[Object field type | Elasticsearch Guide \[7.16\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/object.html)**

---

<div class="post-metadata">

**Author:** ![orips](https://avatars.discourse-cdn.com/v4/letter/o/3ab097/32.png) [@orips](https://discuss.elastic.co/u/orips)\
**Post date:** [January 11, 2022, 8:35am UTC](https://discuss.elastic.co/t/12k-fields-in-the-mapping/293340/7 "2022-01-11T08:35:28Z")

</div>

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 8, 2022, 8:36am UTC](https://discuss.elastic.co/t/12k-fields-in-the-mapping/293340/8 "2022-02-08T08:36:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.