ElasticSearch : More indices vs More types

(sandeep) #1

Hi All,

We are using elasticsearch for the following usecase.
Elasticsearch Version : 5.1.1
Note: We are using AWS managed ElasticSearch

We have a multi-tenanted system where in each tenant stores data for multiple things and number of tenants will increase day by day.

exa: Each tenant will have following information.

1] tickets
2] sw_inventory
3] hw_inventory
Current indexing stratergy is as follows:

tenant_id (GUID) exa: tenant_xx1234xx-5b6x-4982-889a-667a758499c8


1] tickets
2] sw_inventory
3] hw_inventory
Issues we are facing:

1] Conflicts for mappings of common fields exa: (id,name,userId) in types ( tickets,sw_inventory,hw_inventory )
2] As the number of tenants are increasing number of indices can reach upto 1000 or 2000 also.

Will it be a good idea if we reverse the indexing stratergy ?

exa: index names :

1] tickets
2] sw_inventory
3] hw_inventory

So there will be only 3 huge indices with N number of types as tenants.

So the question in this case is which solution is better?

1] Many small indices and 3 types
2] 3 huge indices and many types


(Christian Dahlqvist) #2

Types are going away, so in order to future-proof your solution you would be better off modelling without types. Also be aware that having lots of small indices per customer can be very inefficient and waste heap space, so the second approach (possibly with a field indicating the customer that you can filter on instead of a type per customer) could be a better approach.

(sandeep) #3

Thanks for reply Christian_Dahlqvist