I got the following error when writing docs to Elasticseach cluster. it works till today. I guess I need to increase some limit, but can not find any doc on this yet. Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [3005]/[3000] maximum shards open; C…

If you have 25GB of documents being indexed per hour, resulting in 50GB index size on disk (including replica?) that given 1.2TB of data on disk per day. Having 15-20 shards per daily index in such a case sounds reasonable, but this will depend on your data and queries. 30 days retention period howe…

How to fix hitting maximum shards open error

Elastic Stack Elasticsearch

yuecong (Cong Yue) September 22, 2019, 3:12am 6

I see. thanks for the suggestion.In the case of if I have 25Gb docs ingested hourly and 50GB index storage. Do you think I can still use daily index, which will be 600GB daily index storage size. Also I am using 15 shards for this case, which seems speed up the indexing. Do you think this makes sense?

Topic		Replies	Views
Could not index event to Elasticsearch: Maximum shards open Elasticsearch	3	7209	September 20, 2019
Validation Failed: 1: this action would add [8] total shards, but this cluster currently has [3997]/[4000] maximum shards open Elasticsearch	6	1660	January 15, 2024
Still getting 'max open shards' error after re-indexing and decreasing shards to 850 Elasticsearch	10	1650	August 14, 2020
Increase the number of shards in the cluster Elasticsearch	5	388	August 7, 2023
Could not index event to Elasticsearch..cluster currently has [1000]/[1000] maximum shards open Elasticsearch	3	764	September 24, 2020

How to fix hitting maximum shards open error

Related topics