# Does threshold of 1000 fields per index mean "analyzed fields"? **URL:** \ **Category:** Elasticsearch\ **Created:** [August 2, 2017, 8:49am UTC](https://discuss.elastic.co/t/does-threshold-of-1000-fields-per-index-mean-analyzed-fields/95483 "2017-08-02T08:49:09Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![widhalmt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/widhalmt/32/8237_2.png) [@widhalmt](https://discuss.elastic.co/u/widhalmt)\ **Post date:** [August 2, 2017, 8:49am UTC](https://discuss.elastic.co/t/does-threshold-of-1000-fields-per-index-mean-analyzed-fields/95483/1 "2017-08-02T08:49:09Z")
Hi, I have the problem that we regularly run into the threshold of 1000 fields per per index. Can I prevent this from happening by setting just a few fields I really need to `analyzed` and all others to `not_analyzed`? I found this post ( [Limit of total fields [1000] in index has been exceeded](https://discuss.elastic.co/t/limit-of-total-fields-1000-in-index-has-been-exceeded/87280/4) ) but I'm still not sure if changing the analyzed status of most of my would allow to enter more than 1000? I now, I can raise the threshold but since I know about the danger of "mapping explosion" I'd prefer to stick with it. Thanks in advance. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 2, 2017, 9:44am UTC](https://discuss.elastic.co/t/does-threshold-of-1000-fields-per-index-mean-analyzed-fields/95483/2 "2017-08-02T09:44:31Z")
It's total fields, irrespective if they are text or keyword. ---
**Author:** ![widhalmt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/widhalmt/32/8237_2.png) [@widhalmt](https://discuss.elastic.co/u/widhalmt)\ **Post date:** [August 2, 2017, 9:47am UTC](https://discuss.elastic.co/t/does-threshold-of-1000-fields-per-index-mean-analyzed-fields/95483/3 "2017-08-02T09:47:12Z")
Thanks for your reply! This means I have no way of putting a large number of different fields into Elasticsearch without having overboarding metadata? I talked to the developers and they told me, they need all the data in Elasticsearch for "manual" review in Kibana, but they only need very few fields to be searchable or graphable. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 2, 2017, 9:49am UTC](https://discuss.elastic.co/t/does-threshold-of-1000-fields-per-index-mean-analyzed-fields/95483/4 "2017-08-02T09:49:05Z")
There is a way, increase that limit 🙂 But of course there is a risk there. Why are the documents so large? ---
**Author:** ![Ravi\_Shanker\_Reddy](https://avatars.discourse-cdn.com/v4/letter/r/a5b964/32.png) [@Ravi\_Shanker\_Reddy](https://discuss.elastic.co/u/Ravi_Shanker_Reddy)\ **Post date:** [August 2, 2017, 9:53am UTC](https://discuss.elastic.co/t/does-threshold-of-1000-fields-per-index-mean-analyzed-fields/95483/5 "2017-08-02T09:53:17Z")
> [@warkolm](#): > > But of course there is a risk there. Can you explain what is the risk if my document is too large??? It might help to many others ---
**Author:** ![widhalmt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/widhalmt/32/8237_2.png) [@widhalmt](https://discuss.elastic.co/u/widhalmt)\ **Post date:** [August 2, 2017, 9:53am UTC](https://discuss.elastic.co/t/does-threshold-of-1000-fields-per-index-mean-analyzed-fields/95483/6 "2017-08-02T09:53:56Z")
In this special project developers have to log each and every variable with its value from their software. This is a very special project where policies like this can not be changed. This policy is so fixed that even Elastic Stack might be fully replaced with another tool if that might be able to allow logging all data. I really don't hope that this happens. I didn't want to increase the limit because when I allow for, say, 1500 fields, next months they will hit that limit again, and so on. Therefore I want to have a way which scales to have a lot more than 1000 fields per index. ---
**Author:** ![widhalmt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/widhalmt/32/8237_2.png) [@widhalmt](https://discuss.elastic.co/u/widhalmt)\ **Post date:** [August 2, 2017, 9:54am UTC](https://discuss.elastic.co/t/does-threshold-of-1000-fields-per-index-mean-analyzed-fields/95483/7 "2017-08-02T09:54:48Z")
Hi. The risk is called "mapping explosion". It means that Elasticsearch has to handle so much metadata it can't handle actual data any more. See [https://www.elastic.co/blog/found-crash-elasticsearch#mapping-explosion](https://www.elastic.co/blog/found-crash-elasticsearch#mapping-explosion) for more information. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 2, 2017, 10:00am UTC](https://discuss.elastic.co/t/does-threshold-of-1000-fields-per-index-mean-analyzed-fields/95483/8 "2017-08-02T10:00:52Z")
Basically it adds to the cluster state, which is stored in memory. So a large cluster state can potentially cause excess memory use. It also increases cardinality, aka the diversity of the data. This means that lucene needs more resources to store the same amount of data and Elasticsearch needs more resources to query it. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 30, 2017, 10:01am UTC](https://discuss.elastic.co/t/does-threshold-of-1000-fields-per-index-mean-analyzed-fields/95483/9 "2017-08-30T10:01:07Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.