Limit results count when using sliced scroll api


(Josef Hák) #1

Hi,
I use sliced scroll api to fetch results from Elasticsearch in parallel (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html#sliced-scroll). But I am not able to set Elasticsearch to return first milion only. Results count cab be reduced using parameter "min_score", but it's not way I want. Is it possible to set results count even if sliced scroll method is used?
Thanks.


(Zachary Tong) #2

terminate_after (bottom of this page: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-uri-request.html) might work with scrolling, I'm not sure

But why not just record the number of hits that you've scrolled client-side and stop when you get to 1m?


(Josef Hák) #3

Thanks, I will try "terminate_after".

Yes, I tried this method. But when using "sliced scroll" results seems to be received in random order among slices. So when I stop after 1m received results, it's not sure I received 1m highest score results.

Example: I'm fetching results using 5 slices with size 1000. I supposed that slice 0 will receive 1000 highest score results. But it's not sure. I tested it.


(Josef Hák) #4

Ok, I solved this problem. I control results count fetched with each slice and I stop fetching when defined count per slice is reached.

I don't know how to use "terminate_after" param exactly. It stops fetching when defined count is reached, but received results have not highest score. So it's useless for me.


(system) #5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.