Kibana \ Elasticsearch - Search_phase_execution_exception, all shards failed

Elastic Stack Kibana
1

I'm using Kibana for querying a 3 node Elasticsearch cluster, both version 7.9.2. My Elasticsearch cluster and all of its indexes are healthy and in a green state as reported by _cat/health and _cat/indices?v.

I query an index with a single primary and single replica using KQL with the Kibana Discovery pane. In the query, I modify the date-range so that it will get all of the data in the index, which covers a years worth of data. In this same query, I also added a single filter that checks that a field exists for the returned documents, via the exists operator.

My problem is that I pretty consistently get the error in the bottom right of the Kibana GUI that says:

Error loading data
[status_exception] error while executing search
See the full error

If I click "See the full error" to see the detailed message it states:

Error loading data
Bad Request
search_phase_execution_exception
all shards failed

And in that same detailed message, the stack trace has references to the Kibana URL. Hoping someone can point me in the right direction.

2

I figured out the problem, at least why I was getting this error in my case, I wanted to post my solution in the hopes that it might help someone else out.

The error message was misleading in this case, in fact the problem had nothing to do with the shards for the index. While I was only applying 2 minor query options in the Kibana Discovery pane, the amount of additional crap that Kibana software appends to the query that is run against Elasticsearch behind the scenes is ridiculous. I saved the query that was crapping out in the Kibana Discovery pane as a saved query. I was then able to grab/copy the query that Kibana Discovery pane generated. I then pasted this query in Kibana DevTools. I ran the query in Kibana DevTools and it gave me a more accurate error message. The error message was relating to highlighting on a particular text field that huge amounts of contents in my case. So I modified that query in Kibana DevTools to remove the highlighting code, at that point the query would run in Kibana DevTools without problem. I then had to figure out how to disable highlighting for the Kibana Discovery pane, so that the query would also run there. The option to disable highlighting in the Kibana Discovery page is located at Mgmt > Stack Mgmt > Kibana > Advanced Settings : Highlight Results defaults to true, disable this option. At this point, I no longer get sharding related errors for this particular case.

References:

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.