Performance problem with custom sort

Elastic Stack Elasticsearch
1

Hello!
My search page needs to have a custom sorting type.
To do this sorting i need to return about 5k items from elasticsearch

The problem is that this causes a serious performance problem, the application ends up crashing all the time.
I need the 5k items to go through each one and decide how to display in the listing
If I use the "size" parameter with the value "30" for example, I can't use my custom rule

Is there any way to not have this performance problem?

Thanks.

2

Why do you need 5K results returned and sorted?

3

What kind of performance problems are you encountering? Is it Elasticsearch or your application that is crashing?

4

I have a business rule that needs to be respected

for every page with 30 products, i need to show a certain percentage of products from specific stores

example:

page 1 - 30 products (10 products from store 1, 10 products from store 2, 10 products from store 3)
page 2 - 30 products (10 products from store 1, 10 products from store 2, 10 products from store 3)
page 3 - 30 products (10 products from store 1, 10 products from store 2, 10 products from store 3)

and so on

I have a business rule in ruby ​​that does this sorting
but it doesn't work if the size parameter is equal to "30"
because I need to run the sort on top of all products returned by elasticsearch
but when I set the "size" to 5k for example, the performance is terrible

5

elasticsearch gets very high response time until application crashes
the elasticsearch support says that is because the big query

6

How are you sorting the results related to a single store?

If this is simpler, might an option perhaps be to run three queries in parallel (one per store) using the multi search API and let Elasticsearch sort the queries and then let your application merge them?

If you still need to return 5k results you may need to look into adding resources to your cluster.

It might also help if you show the query that is slow, which version of Elasticsearch you are using and what the specification of your cluster is.

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.