# ElasticSearch performance problem for dynamic field mapping **URL:** \ **Category:** Elasticsearch\ **Created:** [October 24, 2017, 4:04pm UTC](https://discuss.elastic.co/t/elasticsearch-performance-problem-for-dynamic-field-mapping/105094 "2017-10-24T16:04:11Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![live4forever](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/live4forever/32/23354_2.png) [@live4forever](https://discuss.elastic.co/u/live4forever)\ **Post date:** [October 24, 2017, 4:04pm UTC](https://discuss.elastic.co/t/elasticsearch-performance-problem-for-dynamic-field-mapping/105094/1 "2017-10-24T16:04:11Z")
I have an ES on linux and I index the documents using Nest framework. But the problem is there is one object that containes dynamic field. For exp: there is a class named Product and one of its properties is dynamic which means mapping changes everytime according to the new request. Lets say name of the dynamic property is Data and Data might be an object or string or integer or list of obejct ...etc. it might be everything. I don't know it's normal but there is no other option to do it so far. 1 gb data is stored every day into the es but when I search for a data which was indexed 1 or 2 weeks ago on kibana, it can not find it. It sais "Timeout ..." . So far there are 1800 fields in my index. My question is what is the best practice for **dynamic field mapping**? What should I do? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 24, 2017, 5:38pm UTC](https://discuss.elastic.co/t/elasticsearch-performance-problem-for-dynamic-field-mapping/105094/2 "2017-10-24T17:38:20Z")
What version are you on? > [@live4forever](#): > > So far there are 1800 fields in my index. That is likely causing problems, why is it so high? ---
**Author:** ![live4forever](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/live4forever/32/23354_2.png) [@live4forever](https://discuss.elastic.co/u/live4forever)\ **Post date:** [October 24, 2017, 7:59pm UTC](https://discuss.elastic.co/t/elasticsearch-performance-problem-for-dynamic-field-mapping/105094/3 "2017-10-24T19:59:54Z")
The version is 5.5.0 I am trying to use the index as a log storage. I have like 20 applications and these apps creates logs (request and response). I consume these log data as json in my elastic search consumer application. It means I have json string on my elastic search client application. I convert every json data to jobject and than insert them to elasticsearch index. Thats why there are like 1800 fields, every project has its own type. So what is your recommendation or what is the best practice to do this ? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 24, 2017, 8:26pm UTC](https://discuss.elastic.co/t/elasticsearch-performance-problem-for-dynamic-field-mapping/105094/4 "2017-10-24T20:26:17Z")
Are you putting everything in one index? ---
**Author:** ![live4forever](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/live4forever/32/23354_2.png) [@live4forever](https://discuss.elastic.co/u/live4forever)\ **Post date:** [October 25, 2017, 5:17am UTC](https://discuss.elastic.co/t/elasticsearch-performance-problem-for-dynamic-field-mapping/105094/5 "2017-10-25T05:17:06Z")
Yes but it creates a new index everyday. I mean there is one index per day (index\_name-2017-25-10 or index\_name-2017-26-10). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 25, 2017, 5:30am UTC](https://discuss.elastic.co/t/elasticsearch-performance-problem-for-dynamic-field-mapping/105094/6 "2017-10-25T05:30:33Z")
You should split things out into different indices then. Put things that have similar structures together and you won't run into this problem. ---
**Author:** ![live4forever](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/live4forever/32/23354_2.png) [@live4forever](https://discuss.elastic.co/u/live4forever)\ **Post date:** [October 25, 2017, 7:14am UTC](https://discuss.elastic.co/t/elasticsearch-performance-problem-for-dynamic-field-mapping/105094/7 "2017-10-25T07:14:59Z")
Spliting them into different indices might cause a searching performance ? I mean Kibana search for one index now, but if I split it will start searching for all the indices. Wont this cause performance problems ? ---
**Author:** ![zqc0512](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zqc0512/32/32141_2.png) [@zqc0512](https://discuss.elastic.co/u/zqc0512)\ **Post date:** [October 25, 2017, 7:17am UTC](https://discuss.elastic.co/t/elasticsearch-performance-problem-for-dynamic-field-mapping/105094/8 "2017-10-25T07:17:29Z")
timeout settings in kibana the default timeout of kibana is 30sec ,it small with a large data,can change it. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 25, 2017, 7:03pm UTC](https://discuss.elastic.co/t/elasticsearch-performance-problem-for-dynamic-field-mapping/105094/9 "2017-10-25T19:03:28Z")
It won't cause problems. Searching one index of 10 shards is the same as 10 indices of 1 shard, or 5 of 2 shards etc, ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 22, 2017, 7:03pm UTC](https://discuss.elastic.co/t/elasticsearch-performance-problem-for-dynamic-field-mapping/105094/10 "2017-11-22T19:03:29Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.