Increase the number of shards in the cluster

Elastic Stack Elasticsearch
1

Hello,

I'm using version 8.6.0 of elastic cloud.

I tried to create an index but got this message:

Validation Failed: 1: this action would add [2] shards, but this cluster currently has [2000]/[2000] maximum normal shards open;

I can't delete or merge indices that already exist. How do I increase the maximum number of shards in my cluster?

2

Hello.

You may increase this in your elasticsearch cluster?
cluster.max_shards_per_node
It is default 1000 shards per node.

3

Be very careful with this - it's tempting to do it to make the problem go away but you may end up with even bigger problems, namely oversharding.

Here's an explainer article about why oversharding is bad and what you can do to avoid it.

Howcome you can't delete / merge indices?

Do you have a lot of indices with not very much data?

You might consider reindexing some or archiving / moving / deleting data you don't need.

4

Thank you for your considerations. I read this article and we are using the Attribute based approach. So we end up having many indexes with not very much data, I can't merge them because I need them to be separated by client_id.

5

Hello, thank you. How do I change this on my elasticsearch cluster?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.