# How to implement multi tenant environment in Elasticsearch **URL:** \ **Category:** Elasticsearch\ **Created:** [August 24, 2023, 6:42pm UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606 "2023-08-24T18:42:23Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![HARSHAL\_CHAUDHARI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshal_chaudhari/32/36058_2.png) [@HARSHAL\_CHAUDHARI](https://discuss.elastic.co/u/HARSHAL_CHAUDHARI)\ **Post date:** [August 24, 2023, 6:42pm UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/1 "2023-08-24T18:42:23Z")
What is the approach the Elasticsearch community recommends to use in a multi-tenant environment? Is one index Approach good? what are the pros and cons? Thanks, Harshal ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 24, 2023, 7:11pm UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/2 "2023-08-24T19:11:21Z")
It will depend on the use case and specifically how many tenants you need to support and how similar the data is across tenants and how you need to manage retention for different tenants. ---
**Author:** ![HARSHAL\_CHAUDHARI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshal_chaudhari/32/36058_2.png) [@HARSHAL\_CHAUDHARI](https://discuss.elastic.co/u/HARSHAL_CHAUDHARI)\ **Post date:** [August 24, 2023, 7:32pm UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/3 "2023-08-24T19:32:53Z")
Thanks @Christian_Dahlqvist Could you please let me know your thoughts on the below examples? E.g. Approach 1: 4TB of data with 1000 tenants - single index index per tenant Approach 2: 4TB of data with 1000 tenants - 100 index per tenant which could be a better approach for scalability and cost effectiveness. ---
**Author:** ![HARSHAL\_CHAUDHARI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshal_chaudhari/32/36058_2.png) [@HARSHAL\_CHAUDHARI](https://discuss.elastic.co/u/HARSHAL_CHAUDHARI)\ **Post date:** [August 24, 2023, 7:41pm UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/4 "2023-08-24T19:41:07Z")
Thanks @Christian_Dahlqvist !!! Could you please let me know your thoughts on the below examples? E.g. **Approach 1** : 4TB of data with 1000 tenants to implement using a single index per tenant **Approach 2** : 4TB of data with 1000 tenants to implement using 100 index per tenant which could be a better approach for scalability and cost-effectiveness. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 25, 2023, 4:39am UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/5 "2023-08-25T04:39:33Z")
Is that 4TB of data in total or 4TB of data per tenant? The main factors that drive how you handle multi-tenancy in Elasticsearch is however not necessarily just data volume. Could you please answer the following questions? What type of data is it? Is it the same type of data across all tenants? Does each tenant only have one type of data? Do you have control over the format or is this up to the tenants? Do all tenants follow the same data retention pattern? How are the tenants accessing and querying the data? ---
**Author:** ![HARSHAL\_CHAUDHARI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshal_chaudhari/32/36058_2.png) [@HARSHAL\_CHAUDHARI](https://discuss.elastic.co/u/HARSHAL_CHAUDHARI)\ **Post date:** [August 25, 2023, 5:16am UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/6 "2023-08-25T05:16:39Z")
Is that 4TB of data in total or 4TB of data per tenant? > It is 4TB data of all Tenant. The main factors that drive how you handle multi-tenancy in Elasticsearch is however not necessarily just data volume. Could you please answer the following questions? What type of data is it? > Plain text / JSON data of Configure, Price, Quote, Product, Customer Is it the same type of data across all tenants? Does each tenant only have one type of data? > Every tenant has their own types of data (Plain text / JSON data) Do you have control over the format or is this up to the tenants? > This is up to the Customer. > E.g. Single Tenant should have a different index for Product, Category, PriceListItem Do all tenants follow the same data retention pattern? > Yes, all tenants follow the same data retention pattern How are the tenants accessing and querying the data? > 24,000 customers/tenants and over 5 million users are accessing 1.3 million documents every day. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 25, 2023, 5:25am UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/7 "2023-08-25T05:25:26Z")
> [@HARSHAL\_CHAUDHARI](#): > > Do all tenants follow the same data retention pattern? Does the data have a fixed retention period, e.g. deleted after 3 months or never deleted at all? > [@HARSHAL\_CHAUDHARI](#): > > 24,000 customers/tenants and over 5 million users are accessing 1.3 million documents every day. That is not what I meant. Do customers have direct access to the Elasticsearch API? Do they query data through an API that you control? Are they using Kibana to browse data? > [@HARSHAL\_CHAUDHARI](#): > > Every tenant has their own types of data (Plain text / JSON data) How many different types of data can a tenant have? ---
**Author:** ![HARSHAL\_CHAUDHARI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshal_chaudhari/32/36058_2.png) [@HARSHAL\_CHAUDHARI](https://discuss.elastic.co/u/HARSHAL_CHAUDHARI)\ **Post date:** [August 25, 2023, 5:28am UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/8 "2023-08-25T05:28:13Z")
> [@Christian\_Dahlqvist](#): > > How many different types of data can a tenant have? That is not what I meant. Do customers have direct access to the Elasticsearch API? Do they query data through an API that you control? Are they using Kibana to browse data? > Yes we have written our own APIs that make queries using Elasticsearch client. How many different types of data can a tenant have? > Single tenants have 100-200 types ---
**Author:** ![HARSHAL\_CHAUDHARI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshal_chaudhari/32/36058_2.png) [@HARSHAL\_CHAUDHARI](https://discuss.elastic.co/u/HARSHAL_CHAUDHARI)\ **Post date:** [August 25, 2023, 5:29am UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/9 "2023-08-25T05:29:18Z")
> [@Christian\_Dahlqvist](#): > > Does the data have a fixed retention period, e.g. deleted after 3 months or never deleted at all? Does the data have a fixed retention period, e.g. deleted after 3 months or never deleted at all? Yes, all tenant has the same retention period. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 25, 2023, 6:43am UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/10 "2023-08-25T06:43:02Z")
What is the retention period? When tenants query data are you just filtering or do tenants perform free text searched where relevancy is important? Do tenants query across multiple types of data in a single query? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 26, 2023, 11:49am UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/11 "2023-08-26T11:49:03Z")
Elasticsearch does generally not scale or perform well with indices per tenant in a multi-tenant situation, al least not unless the number of tenants is reasonably low. Having lots of very small indices and shards adds overhead and is inefficient. The default limit of 1000 shards per node is there for a very good reason, and it is to prevent users from oversharding. When handling a large number of tenants like in your scenario it is therefore common to group tenants into one or more indices. As you have an API tier you can add a filter on a tenant ID field in your documents in order to prevent one tenant from seeing another tenants data. The second, and possibly most important, aspect to multitenancy is mapping management. Elasticsearch is not schema-fless, so if tenants can ingest data in any form they like you are likely to encounter mapping conflicts where new data can not be indexed. Due to this it is often common to group data based on the type of data being indexed. If you e.g. knew that tenants could each have 200 types of data and that these types were the same across tenants with no schema conflicts, you could create one index per data type and have all tenants share that index for that type of data. As described earlier you would manage access and filtering in your API tier. The amount of data does not really determine how you organise your data and tenants, but rather how you size the different indices with respect to primary and secondary shards. This is why I was asking questions about the tenants and the data types. ---
**Author:** ![HARSHAL\_CHAUDHARI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshal_chaudhari/32/36058_2.png) [@HARSHAL\_CHAUDHARI](https://discuss.elastic.co/u/HARSHAL_CHAUDHARI)\ **Post date:** [August 26, 2023, 12:17pm UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/12 "2023-08-26T12:17:20Z")
thank you @Christian_Dahlqvist for the information. Could you please suggest in the above use case what approach should I go for 1/ Shall I create an Elasticsearch index Per type, per index, per tenant? E.g. For 24000 tenants, 200 types shall I create a 24000\*200 = 4800000 index in Elasticsearch? is it feasible? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 26, 2023, 12:29pm UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/13 "2023-08-26T12:29:46Z")
If the type of data is the same across tenants I would recommend creating one index per type, e.g. 200 indices in total, and have all tenants share these indices. You may group tenants into indices but creating indices per tenant will likely not work nor scale or perform. This does however depend on the nature of your data, which we know nothing about. > [@HARSHAL\_CHAUDHARI](#): > > shall I create a 24000\*200 = 4800000 index in Elasticsearch? is it feasible? No. That is not feasible. ---
**Author:** ![HARSHAL\_CHAUDHARI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshal_chaudhari/32/36058_2.png) [@HARSHAL\_CHAUDHARI](https://discuss.elastic.co/u/HARSHAL_CHAUDHARI)\ **Post date:** [August 27, 2023, 3:50am UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/14 "2023-08-27T03:50:32Z")
Really Appreciate @Christian_Dahlqvist for your time and information. Thanks again. Regards Harshal ---
**Author:** ![HARSHAL\_CHAUDHARI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshal_chaudhari/32/36058_2.png) [@HARSHAL\_CHAUDHARI](https://discuss.elastic.co/u/HARSHAL_CHAUDHARI)\ **Post date:** [August 27, 2023, 3:54am UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/15 "2023-08-27T03:54:10Z")
> [@Christian\_Dahlqvist](#): > > Do tenants query across multiple types of data in a single query? Yes, in some use cases. ---
**Author:** ![HARSHAL\_CHAUDHARI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshal_chaudhari/32/36058_2.png) [@HARSHAL\_CHAUDHARI](https://discuss.elastic.co/u/HARSHAL_CHAUDHARI)\ **Post date:** [August 27, 2023, 4:03am UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/16 "2023-08-27T04:03:17Z")
@Christian_Dahlqvist is there any alternative way, I did not want to keep multiple tenant data in a single type index. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [August 27, 2023, 5:12am UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/17 "2023-08-27T05:12:32Z")
Having an index per tenant does, as I mentioned earlier, sometimes work as long as the number of tenants is low. With a large and potentially growing number of tenants this approach does not work in Elasticsearch. This has come up numerous times before and you can search the forum and see that the advice is the same. One way I have seen suggested in the past, but which Iwould not recommend, is to have many separate small clusters that each host a manageable number of tenants. Given that you have multiple data types with potentially conflicting mappings I believe it makes sense to group indices by data type rather than tenant in this case. ---
**Author:** ![rugenl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rugenl/32/12887_2.png) [@rugenl](https://discuss.elastic.co/u/rugenl)\ **Post date:** [August 31, 2023, 12:20am UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/18 "2023-08-31T00:20:34Z")
What are the aecurity requiremenes? Limiting user access to the appropriate data within a shared index can be done with document level security. 24000 tennents is not trivial for a new elastic designer. ---
**Author:** ![HARSHAL\_CHAUDHARI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshal_chaudhari/32/36058_2.png) [@HARSHAL\_CHAUDHARI](https://discuss.elastic.co/u/HARSHAL_CHAUDHARI)\ **Post date:** [August 31, 2023, 4:38am UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/19 "2023-08-31T04:38:05Z")
> [@rugenl](#): > > 24000 tennents is not trivial for a new elastic designer. Thanks, @rugenl , could you please suggest an appropriate solution? as per ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 28, 2023, 4:38am UTC](https://discuss.elastic.co/t/how-to-implement-multi-tenant-environment-in-elasticsearch/341606/20 "2023-09-28T04:38:43Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.