Search Cancellation mechanism on timeout before v.6.4.0

Elastic Stack Elasticsearch
1

The 6.4.0 version of "Search Request Body" reference (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html#_parameters_4) says that "Search Cancellation" mechanism is used on search timeout.

I am running v.6.3.0 and curious if the same mechanism is used when searches timeout or it was updated between versions 6.3.0 and 6.4.0. Would appreciate any information on what exactly happens when search hits timeout and what does it mean for a search to hit a timeout.

Thank you!

2

The same "Search Request Body" reference in prev. versions (for example: v.6.3.0) doesn't mention "Search Cancellation" mechanism for the timeout parameter.

3

I'm not familiar with the specific timeout changes between those versions but it's true to say that the implementation is constantly improving/evolving.
The challenge is adding "timed out yet?" checks into all the various code loops that execute as part of search without significantly slowing down the search execution. These loops exist in multiple places, including the Lucene library.
If we miss a loop eg some regex expansion logic*, then it means that certain queries may overrun and take longer than the timeout setting defines. If we add a check into a very tight loop then we slow down search. This is why we say it is a "best effort" approach and is evolving.

* (we may check regex expansion logic. I only use it as an example.)

4

Hi @Mark_Harwood, thank you for the answer.

The cluster that I'm running has a lot of long-running queries. I am after building a query killer that periodically checks the list of search tasks (using Task Management API) and cancelling the ones that pass some pre-defined time-threshold (for example, 1 min).

Looks like, if I was running a newer version, 6.5.0, I wouldn't need to implement anything, since the same cancellation mechanism is used for search timeouts. If v.6.3.0 uses the same cancellation mechanism, it'd save me some time. Any suggestions on what'd be a good way to find out if Task Management API (i.e. task cancellation from it) is used in v.6.3.0 when search times out?

5

That sentence was added to the docs in #33354 to resolve #31263 which arose from a question from a user about how search cancellation worked in 6.2.4. I don't think anything significant changed, it was just a clarification to the docs that wasn't backported because we don't normally backport this kind of thing.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.