Adding shards as the volume of data grows

Elastic Stack Elasticsearch
1

Hi all,

Is there some documentation or group topic that describes when, how
and the impacts of adding new shards to an index, either because the
index is becoming too big or because we need to add additional ES
nodes in the cluster?

We are building a multi-tenants Web app. We have one index per entity
type. We will shard each index as appropriate (volume, update rate and
ratio, update contention, etc) and we will use document routing to map
tenants to shards for each index.

So in short, what is a good strategy to monitor shards performance,
add shards when necessary, possibly re-shard the index and remap
tenant ids to shard ids?

Thank you very much for your insights!

Remy

2

How much a shard can "take you" really depends on several aspects, including number of documents, size, type of search requests against it, facets executed on it and so on. It requires some planning in advance which usually end up with a rough document count per shard.

On Thursday, June 23, 2011 at 6:49 PM, Spring Ninja wrote:

Hi all,

Is there some documentation or group topic that describes when, how
and the impacts of adding new shards to an index, either because the
index is becoming too big or because we need to add additional ES
nodes in the cluster?

We are building a multi-tenants Web app. We have one index per entity
type. We will shard each index as appropriate (volume, update rate and
ratio, update contention, etc) and we will use document routing to map
tenants to shards for each index.

So in short, what is a good strategy to monitor shards performance,
add shards when necessary, possibly re-shard the index and remap
tenant ids to shard ids?

Thank you very much for your insights!

Remy