View stuck queries in ES

Elastic Stack Elasticsearch
1

We have been seeing an issue that ES gets stuck with queries and the client doesn't return results. If we kill the client the queries are still stuck with ES and any further queries don't yield any results. The only solution is restart of the server.

Is there a way to view the queries stuck in ES? Is there a way to timeout the queries so that restart in not needed? Is there a way to timeout the client? Can we see the queries being executed in the logs?

2

jstack or hot_threads. But you need to understand the implementation some to make heads or tails of the output. There is an issue to implement something nicer but its not anyone's top priority.

There are timeouts but they are best effort. Have a look at this doc. Its for the rest layer but I imagine the transport client can do it to. Still, this won't always stop the threads. There just isn't a way to add hard timeouts to java. So if you are seeing a bug then its likely the timeout won't work.

All the http based clients should support a client side timeout. Set it to higher than the server side timeout to give it a chance.

There is a slow search log that you can enable but it only triggers when the query finishes. Its more for figuring out which searches your application is doing are expensive or so you'll notice when you start querying cold disk blocks and stuff. Not so much for hung queries.

Odds are good you've hit some kind of bug.

1 Like