How to solve error: "Status changed from green to red - all shards failed"

Elastic Stack Kibana
1

Kibana throws this error message (on a single testing system very often, on production cluster sometimes):

kibana: {"type":"log","@timestamp":"2019-06-13T15:14:28Z","tags":["status","plugin:ml@5.6.16","error"],"pid":5445,"state":"red","message":"Status changed from green to red - all shards failed: [es_rejected_execution_exception] rejected execution of org.elasticsearch.transport.TransportService$7@b47c1ce on EsThreadPoolExecutor[search, queue capacity = 1000, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@21aa6e65[Running, pool size = 7, active threads = 7, queued tasks = 5100, completed tasks = 14922253]]","prevState":"green","prevMsg":"Ready"}

The queue size is already at it's maximum of 1000. How to solve this problem?

2

Do you know what the queued tasks are? Looks like Elasticsearch isn't keeping up with the tasks. If it's a single node cluster you might have to scale it up larger to keep up with those tasks.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.