Multi-Tenant - Shard and Index strategy / Optimizing Elasticsearch configuration?

Elastic Stack Elasticsearch
1

Hi Team,

Could you please suggest the Elasticsearch configuration? (for scale and performance)

My use case is below

  1. I have 1000 - 2000 thousands of customers - each customer has a 100-200 index, How data nodes can be scaled and re-shared?

  2. Efficient use of indexes.

what will it Elasticsearch cluster, node, shard, and index configuration be for the above use cases?

FYI, For the above use case Elasticsearch single cluster with 3 nodes - reached the maximum shard limit, Please guide me in this direction as well.

2

It is pretty hard to suggest anything with just the information you shared, you need to provide more context.

Like, what will the data looks like? It will be the same for each customer? Is it time-series based? What is the average document size? What is the retention needed? What is the event rate per second?

The best way to size an Elasticsearch cluster is doing a Proof of Concept of your use case with a small cluster to see how it will behave and what resources you need, then you can extrapolate with this information.

But assuming 2000 customers and 200 index per customer, you will have something near 400k indices, you will probably need hundreds of nodes for this.

1 Like
3

Thanks @leandrojmp !!!

Could you please let me know your thoughts on the below examples?

E.g.
Approach 1: 4TB of data with 1000 tenants to implement using single index index per tenant
Approach 2: 4TB of data with 1000 tenants to implement using 100 index per tenant

which could be a better approach for scalability and cost-effectiveness.

4

Thanks @leandrojmp !!!

Could you please let me know your thoughts on the below examples?

E.g.
Approach 1: 4TB of data with 1000 tenants to implement using a single index per tenant
Approach 2: 4TB of data with 1000 tenants to implement using 100 index per tenant

which could be a better approach for scalability and cost-effectiveness.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.