How exactly can the _search be overwhelmed?

[abhinav_chourasia]

hello folks,

we have been using elasticsearch for many products in our organization. And one recommendation that we have always given our developers is to make sure that the elasticsearch is definitely not exposed directly to the internet. Gladly, the devs have adhered to it.

However, recently we came across a situation where we have search functionality on a website which speaks to a service in the backend. This service then directly searches somethings in the underlying ES.

We have had similar situations in the past, where we used to simply ask the devs to rate limit the search functionality as it may overload the searches and make the ES itself slow for all the other services interacting with it and we were good with it.

However, now I am more inclined towards understanding

1. How exactly would the ES instance get slowed down if the searches are not really rate limited ?
2. How is this any different from a huge number of non rate limited queried on any RDMS ?

I did find this ES security guidelines
which says:

Understand that it is quite possible to write a _search that overwhelms Elasticsearch and brings down the cluster.

Would be great if someone could please help me understand how exactly is the above possible.

[Yogesh_Gaikwad]

It depends on different factors ranging from the system configuration for ES handling search requests to the data that has been indexed into ES.
Things like large documents and queries which are doing highlighting or retrieving large data sets without scroll.
Depending on the number of queries, kind of data (high cardinality) it might put memory pressure on the ES nodes there by slowing down the responses.

Following resources might help you further:
https://www.elastic.co/guide/en/elasticsearch/reference/master/tune-for-search-speed.html

Query Planning for Range Queries in Elasticsearch

Plan ahead, benchmark, and keep monitoring your ES.
Hope this is helpful.

Regards,
Yogesh Gaikwad