ELK query throtteling

Elastic Stack Elasticsearch
1

Hello,

we are using the ELK setup for our log management. The document count is currently around 1.5 Billion.
We are facing the problem when users making queries by the elasticsearch API or by the Kibana Dashboard it makes the API unresponsive and its just recoverable by restarting the elasticsearch service.

Is there a way to prevent searches on to many documents or skip a search when it takes to long.

Any help appreciated!

Cheers,
Chris

2

There's not. ES doesn't handle process management (in this context) very well. It's something we're going to improve.

3

HI @warkolm - wouldn't circuit breaker feature of ES be helpful in this matter?

4

Only if the query would cause an OOM.
It doesn't currently handle (say) a thousand users running a bunch of smaller queries and causing the same issue.

5

Though you could kind of control it by setting appropriate search thread pool queue size...

Otis

Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Elasticsearch Consulting & Support * http://sematext.com/

6

Yeah true :slight_smile: