Hello.
My cluster is reaching 2000 opened shards. I have two data nodes now.
I don't want to add another data node and scale up the cluster.
I'm thinking more like changing indexing strategy. Currently logstash is creating 10 indexes per day. Some are 10 GB some 200MB.
The question is what is reasonable index size or documents amount to retain cluster performance and stability ?
Regards.
Hi, i recently answered this discussion on the forum around sizing as well.
Hope this helps you too, if any other questions arise let us know.
1 Like
If you have logs data and metrics, which tend to be immutable, I would recommend an average shard size between 25GB and 50GB. The index size would depend on the number of primary shards.
1 Like
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.