Multi tenancy with varying document mappings

Elastic Stack Elasticsearch
1

We have an elasticsearch cluster where we store search data for multiple customers. Right now, we use an index per type of document and filter based on customer ID. This works well for us as all documents are the same.

We now have a requirement that customers should be able to create custom fields on top of the default ones, which are also searchable (customers search their data via a web app so we can control that). The problems around doing what we do now and say prefixing custom fields with the customer ID (way too many fields per index for example) seem like that wouldn't be a good solution.

The only feasible solution I can see is an index per customer with all of the additional overhead that includes (lots of partitions, hot/cold partitions, some large indexes vs some very small).

Are there any other possible ways of doing this that I should investigate?

2

You might be able to define a number of standard fields of different types and then map/translate customer field names to these. This will allow you to reduce the number of fields, but could affect relevancy calculation and would require translation at the application layer.

Having separate indices per customer would work for smaller number of customers but would scale badly.

1 Like
3

Thanks Christian. So you're essentially saying we could have generic fields like custom_string_1 etc then just map each custom string field for a user to one of those?

4

Yes.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.