Dynamic mappings

Elastic Stack Elasticsearch
1

Hi All,

Thanks in advance for the solution or suggestions.

I have platform where we store user's content(schema based content) as applications such as website data, mobile data, IOT data etc.

To, search on this data we are planning to have ElasticSearch setup. But we have following difficulties.

  • Multiple types can be created by customer in each application, and there are chances that multiple schema might have same field name but with different data type.(Considering this schema as one type will generates different mapping for each one field and conflict of same field with different type will occurs)

  • Customer data is schema based, which will make it dynamic data. Also this schema can be modified by them anytime.(So, if we consider schema as type, any removal of field needs reindexing) - In this situation, all the data of other schemas(types) will also need to be shift(it might affect performance).

Waiting for the reply.

2

You can't work around those limitations, I'm afraid you'll have to apply similar constraints to your customers.

3

Thanks for the quick reply.

I can not enforce such constraint on customers, but I am planning to do the following.

  • Combine all the custom fields in new string field(stringified JSON).
  • Later on schema update(removal of field) will parse the new stringified field and remove the field using scripting in ES(painless) and again redoing above.

It will be weird implementation, can you suggest anything on this approach or any better way of doing it?

Thanks,

4

If exact matching on custom fields is your only requirement, then that would work. However it means that eg. ranges on numeric values will not work.

5

For now, we are just looking for the text search.