Problem due to number of fields limitation in ES 5.5.2

Elastic Stack Elasticsearch
1

Currently in production environment we are using ES-2.4.1 with our own dynamic mapping ( our data in ES is Graph JSON ) . The max document count in Index as-if now is 4,29,890 with n-number of fields.

Now we are planning to Upgrade ES 2.4.1 to ES 5.5.2 , Same mapping is working but it shown error like " Limit of Field Count is 1000 " So I changed the settings to 5000 . Now I can index only 50 Documents in that INDEX .

whats the permanent Fix for this ?? I think this settings is not the good solution as I'm unaware of document count and field count of the data ..

Kindly help me out..

2

The number of fields is only a problem, when all of those fields need to be searchable. If they dont need to be, then you can configure the mapping appropriately.

Elasticsearch is not an arbitrary key/value store that allows you to search in any key - this is the reason why the limit was introduced, because it was an operational issue due to exploding mappings.

Hope this helps.

3

So do you mean to say that ES is mainly a document storage where v can search for document not as fields in that document ( key value pair ) ???

4

Sorry, but I did not understand the question.

You can store any JSON data in Elasticsearch, but you should not make it searchable if you have thousands of different fields, as each field basically needs its own inverted index. Hope this makes sense.

5

Ya thanks Alex. Now I got clear idea ..

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.