Could not index event to Elasticsearch

Elastic Stack Elasticsearch
1

hi I am facing issues while trying to create an index by querying microsoft sql server and the error I am facing as follows
Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"index_name", :routing=>nil}, {"@version"=>"1", "column_name"=>000000000000.0, "@timestamp"=>2024-12-03T12:29:53.637323178Z}], :response=>{"index"=>{"_index"=>"index_name", "_id"=>nil, "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Validation Failed: 1: this action would add [2] shards, but this cluster currently has [4999]/[5000] maximum normal shards open;"}}}}

2

It looks like you have hit the built-in limit on the number of shards each data node can hold, which is 1000. I therefore assume you have 5 data nodes. Why do you have this many shards? What is the average shard size?

Which version of Elasticsearch are you using?

3

hi , I am using the 8.6.1 version