Adding new node and resharding

Elastic Stack Elasticsearch
1

Hi!

I planned to use 4 nodes for index, but when I created index I had only 2 nodes for start and planned to buy other nodes when I need more storage space for index.

So I created an index with 40 shards with "index.routing.allocation.include" to 4 nodes (2 exist, and 2 not exist) and start to index data. When I had 90% storage space on both nodes, I bought another node and add it cluster.

I suppose that automatic cluster rebalance must happen, so some shards will automatically go to new node, but this was not happen and I wonder why?

Thanks in advance!

2

That's waaaaaay too many shards, you're going to be wasting a lot of resources maintaining this.

Why not just remove the i.r.a.i setting and let ES handle allocation automatically?

3

Why do you think it is too many shards and what resources I am wasting?

How allocation settings depends on resharding mechanics in elasticsearch?
So you think that if I disable allocation options elastic will relocate shards to new nodes?

4

Why did you even bother putting those allocation settings in?

Yes that is too many shards. Unless you expect to grow to 40 servers and can put one shard per node.

Each shard is a lucene instance that takes up resources to maintain, even if empty.