Warning Courier Fetch: shards failed

Hi All,

I am seeing this warning in Kibana 5 with ES 5.0 installed. What can be the reason for the failure and how to fix it.

--
Niraj

@niraj_kumar

that's an internal error from Kibana. It means something went wrong when querying ES.

Is this a persistent problem you have, or did it just show up once?

I almost see this error every 10th or 15th time i query kibana. Annoying

How many nodes do you have in the cluster? How much data do you have in it? How many indices and shards?

This is is a dedicated node cluster with master, data and ingest node.

1198 indices and 7188 shards for less than 20 million documents and 54GB of data (if I am reading it correctly) means a lot of very small shards, which is very inefficient. Depending on how large portion of these you are querying over, it could very well be causing the problems you are seeing. I would recommend rethinking your indexing strategy and drastically reducing the number of indices and shards.

How do you recommend indexing cloudtrail data that is dumped in a single s3 bucket for 150 AWS account. We mark this as a central place to store all the aws account cloudtrail data. We are fairly new to this Elasticsearch world. So a good idea and recommendation would help with this current setup we have.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.