Is Kibana EQL Rule Using Async Search?

[davendu]

Hi everyone,

Similar to this thread, I also was notified by error Request timed out like the following debug log says:

{ ... "message":"Changing rule status to \"failed\". An error occurred during rule execution: message: \"Request timed out\"  ... }

There is a load-balancer between my Kibana and Elasticsearch server, which has a ~60s max connection timeout. I'm not sure if this is a probable cause. However, rather than changing this config, I know EQL search can be async called, and want to use this feature to solve the root cause.

I enabled the debugging log, and seems all EQL queries are not carrying argument wait_for_completion_timeout, which means all searches are sync query. I also checked v8.5.0 kibana source, and I'm not sure if my findings are correct.

So could anyone check if Kibana is using sync or async mode by default when executing EQL rules?

[Venkata_Raja]

Hi,

By default, EQL search requests are synchronous and wait for complete results before returning a response. However, complete results can take longer for searches across large data sets data.

To avoid long waits, run an async EQL search. Set wait_for_completion_timeout to a duration you’d like to wait for synchronous results.

EQL search | Elasticsearch Guide [master] | Elastic

[davendu]

Hi Venkata,

Thanks for reply. As I have mentioned, I know wait_for_completion_timeout is a feature in Elasticsearch to control if the EQL query should be executed sync or async.

However, as I have said in the title, I would like to focus on if a EQL based rule in Kibana (or, to be more specifically, Elastic Security) is running in sync mode or not by default.

[Venkata_Raja]

Hi Davendu,

As EQL rules are also based on EQL queries only , i can say they default run in sync as per documentation

By default, EQL search requests are synchronous

[davendu]

Hi Venkata,

It's not that simple. For instance, if you use the EQL preview function, the query is actually executed in async way, but you have never got a chance to select how it is executed. Search about EQL_SEARCH_STRATEGY (case sensitive) to know how the strategy is registered, and then used in x-pack/plugins/security_solution/public/common/hooks/eql/use_eql_preview.ts when previewing the rule.

I think a more sounding evidence is required if we want to say the EQL rules in Elastic Security is run in the sync/async way.