Logs are not showing in kibana - maximum shards open error

Logs stopped suddenly from displaying in kibana. no clue.index pattern also not created.

this is the error logs i am getting from logstash

[2020-02-27T07:24:03,585][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"filebeat-6.5.4-2020.02.27", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x113e8317], :response=>{"index"=>{"_index"=>"filebeat-6.5.4-2020.02.27", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"validation_exception", "reason"=>"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [999]/[1000] maximum shards open;"}}}}

Hey @rvb_n,

This seems to be an elastic search related error.
I'm reassigning it to the correct channel.

Thanks

any response

What is the output of:

GET /
GET /_cat/nodes?v
GET /_cat/health?v
GET /_cat/indices?v

If some outputs are too big, please share them on gist.github.com and link them here.

i am getting this error

bitnami@ip-172-31-31-216:~$ GET /_cat/nodes?v
-bash: GET: command not found

That's something to run from the Kibana dev console. Otherwise transform the commands to actual curl requests.

image

have shared . help on this urgent

Please don't post images of text as they are hard to read, may not display correctly for everyone, and are not searchable.

Instead, paste the text and format it with </> icon or pairs of triple backticks (```), and check the preview window to make sure it's properly formatted before posting it. This makes it more likely that your question will receive a useful answer.

It would be great if you could update your post to solve this.

Read this and specifically the "Also be patient" part.

It's fine to answer on your own thread after 2 or 3 days (not including weekends) if you don't have an answer.

{
"name" : "ip-172-31-38-116",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "S-bJlxKyTmSY4eU1CgiQOw",
"version" : {
"number" : "7.3.1",
"build_flavor" : "oss",
"build_type" : "tar",
"build_hash" : "4749ba6",
"build_date" : "2019-08-19T20:19:25.651794Z",
"build_snapshot" : false,
"lucene_version" : "8.1.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}

=============================

{ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
127.0.1.1 81 97 19 0.28 0.27 0.30 dim * ip-172-31-38-116
}

{epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1583304549 06:49:09 elasticsearch yellow 1 1 500 500 0 0 499 0 - 50.1%
}

As it looks like you only have one node set the number of replicas to 0 for all indices. It looks like the shard count also includes unallocated shards. As replica shards can never be allocated on the same node as the primary you are in effect limited to 500 shards unless you set replicas to 0.

can you please help how to do.

Please don't post unformatted code, logs, or configuration as it's very hard to read.

Instead, paste the text and format it with </> icon or pairs of triple backticks (```), and check the preview window to make sure it's properly formatted before posting it. This makes it more likely that your question will receive a useful answer.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.