Courier Fetch: 5 of 5 shards failed on Kibana

Hi , I am running a 3 node elastic search cluster with replica 1 . I am also running two node logstash HA. Problem is that recently i have seen this error "courier fetch : X of X shards failed " . But if i look at my cluster heath it is green and shards are all in started mode. No problem with the cluster whatsoever.

Even on kibana if i look at the dashboards that i have created earlier , they all showing logs. logstash is parsing logs correctly , but kibana still show the problem . i tried to use this configuration and tried rolling restart . Nothing seem helps.

threadpool.search.type: fixed
threadpool.search.size: 200
threadpool.search.queue_size: 10000

I also checked how many threads elasticsearch uses and found in all node , total number of threads are pretty much less than 100. Any idea what is causing it ?

Any help is very much appreciated.

Check your ES logs, it'll explain what happened.