Throttling search queries on Kibana or via API

Elastic Search
1

Hi,

I am facing issues where users query the ES (version 8.8.0) via the API using a script at high frequency (i.e. one immediately after the previous one completes), and as a result, the JVM heap usage was constantly at a high level (60+%), and when it was around 75%, I started getting the error below.

circuit_breaking_exception: [parent] Data too large, data for [<http_request>] would be [32620387928/30.3gb], which is larger than the limit of [31621696716/29.4gb], real usage: [32620387928/30.3gb], new bytes reserved: [0/0b], usages [eql_sequence=0/0b, fielddata=24272253421/22.6gb, request=0.0b, inflight_requests=0/0b, model_inference=0.0b]

I was wondering if there is a way to throttle/limit the queries (be it on Kibana or via API), or some buffering mechanism, so as to not overwhelm the ES with too many queries.

Thank you.

2

Hey there @hjazz6, my suggestion would be to implement client side throttling if possible.

3

Is client-side throttling on an app-by-app basis, i.e. not something I can configure on ES?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.