Could not index event to elasticsearch maximum shards open

Hi
I have problem with Elasticsearch and Logstash. Here is the detail of log:

Apr 16 13:01:24 elastic7-logstash logstash[980]: [2021-04-16T13:01:24,828][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-7.3.0-2019.08.22", :_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x7e327a87>], :response=>{"index"=>{"_index"=>"winlogbeat-7.3.0-2019.08.22", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"validation_exception", "reason"=>"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;"}}}}
Apr 16 13:01:24 elastic7-logstash logstash[980]: [2021-04-16T13:01:24,828][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-7.3.0-2019.08.22", :_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x5caece52>], :response=>{"index"=>{"_index"=>"winlogbeat-7.3.0-2019.08.22", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"validation_exception", "reason"=>"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;"}}}}
Apr 16 13:01:24 elastic7-logstash logstash[980]: [2021-04-16T13:01:24,828][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-7.3.0-2019.08.22", :_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x1fc5b16f>], :response=>{"index"=>{"_index"=>"winlogbeat-7.3.0-2019.08.22", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"validation_exception", "reason"=>"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;"}}}}
Apr 16 13:01:24 elastic7-logstash logstash[980]: [2021-04-16T13:01:24,828][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-7.3.0-2019.08.22", :_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x1901cbeb>], :response=>{"index"=>{"_index"=>"winlogbeat-7.3.0-2019.08.22", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"validation_exception", "reason"=>"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;"}}}}
Apr 16 13:01:24 elastic7-logstash logstash[980]: [2021-04-16T13:01:24,829][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-7.3.0-2019.08.22", :_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x7e354641>], :response=>{"index"=>{"_index"=>"winlogbeat-7.3.0-2019.08.22", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"validation_exception", "reason"=>"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;"}}}}
Apr 16 13:01:24 elastic7-logstash logstash[980]: [2021-04-16T13:01:24,829][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-7.3.0-2019.08.22", :_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x5624c1f3>], :response=>{"index"=>{"_index"=>"winlogbeat-7.3.0-2019.08.22", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"validation_exception", "reason"=>"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;"}}}}
Apr 16 13:01:24 elastic7-logstash logstash[980]: [2021-04-16T13:01:24,829][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"winlogbeat-7.3.0-2019.08.22", :_type=>"_doc", :routing=>nil}, #<LogStash::Event:0x78945d6f>], :response=>{"index"=>{"_index"=>"winlogbeat-7.3.0-2019.08.22", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"validation_exception", "reason"=>"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;"}}}}

And I see at Stack management > Index management it has many index from the time before I installed the ELK. Image below
image
I dont know why and how to fix this. Plz help me

Welcome!

You can't have more than 1000 shards per node.

Here I think you have only one node.

So either you start 2 more nodes, or you remove some of the existing indices.

Thank you. How to Logstash doesnt collect log from the past.

What is the output of:

GET /
GET /_cat/nodes?v
GET /_cat/health?v
GET /_cat/indices?v

If some outputs are too big, please share them on gist.github.com and link them here.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.