Cannot Retrieve Search Results

Elastic Stack Kibana
1

Hello,

I have been getting an error message in discover on and off when I run a number of different queries:

Cannot Retrieve Search Results

Then w/in the text box there is a base64 encoded string. When I decode the string it is made up of the task id of a task as long as the node that it is running on. There is another string that precedes the nodeid:task_id part of the string that I haven't been able to figure out to this point.

Then I take the json response and run it in dev tools successfully. Finally, this error happens sometimes and it doesn't at other times. I have tried to search specific time ranges to see if it is a specific date and time. These issues are all happening on time series data, so I have been trying to ID if there is an index or shard that is somehow corrupted or not working properly.

Unfortuantely, that is all that I have to go on at this point. I am unable to share screen shots from kibana as the system is air-gapped. I am curious if anyone has heard or seen of anything similar to this before.

Thanks,
Alex

2

Hi @alaine , I'm thinking this is likely an Elasticsearch error (you can see that Kibana shows that message when it detects an Elasticsearch error here).

Does it work consistently in Dev Tools? Do you see anything in the Elasticsearch logs?

3

Andrew,

Thanks for getting back to me and yes that is the error it includes a base64 encoded string made up of the task id and something else.

The search always works in dev tools. We haven't been able to find anything in the elastic logs yet at this point, but we will look again. We will also look through the logs again when it happens again.

It started a day or two before I originally made the post here. It was really bad at first, but it has since become less prevalent. It was weird because there weren't any changes made to the environment.

I am a little confused, because the query always works in dev tools. Wouldn't that indicate it wasn't an elasticsearch error. I was thinking it was an issue with kibana not being able to display the data properly, but if elasticsearch isn't able to get the data to the discover app properly it could be an elasticsearch issue.

Is it an issue sending the data from the elastic nodes to the discover app? It seems like it is able to get the right information to kibana, because it displays fine in dev tools.

Thanks,
Alex

4

@alaine

Yes, this is strange, but all sorts of factors could be at play. Did your data change? Did the number of people viewing dashboards change? Are you managing your own infrastructure or are we talking about a cloud environment that could have been subject to the noisy-neighbor problem? All these things could affect a change to the load the cluster is able to bear.

Yes, if the exact query created by Discover app truly always worked in dev tools it would point to an issue in Kibana, but it's really hard to prove that since this error is so intermittent. The fact that the error's response body mentioned a node ID also points to an Elasticsearch origination.

We know Kibana is getting a response from Elasticsearch since the node ID is being reported (Kibana keeps no record of what node a request is routed to). If this comes up again, I would recommend checking those logs and reaching out on the Elasticsearch topic. Sorry that I can't be more helpful!

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.