ES Pagination - best way

Elastic Stack Elasticsearch
1

I was looking into making pagination in my app.

at first, i used the "From", "size" options and it all worked great, until i reached the 10,000, results, which was 2 large to process as each shard process 10,010 results in order to give me the 10,001-10,010...

plus, i understood its hard to do as each search might return a different order of results (scheduling prob?) then how can we know whats the 10,007 results? so it needs to somehow save my search entirely, then go through the results of the current search with pagination ids or something like that.

So I looked farther and found out about the Scan and Scroll, so thats the way to do it!
But then i found out i can only go forward and not backward so basically thats not good enough.

so first of all, is that correct that scan and scroll can only go forward and not backwards, if i send an old scroll_id, wouldnt i get the same results as i got previously ( i didnt fully understood the scan and scroll API)??
Secondly, Is there a new/what is the best way to use pagination?

Lets say i want 10 results per pages, and i have 1,000,000 pages.
i want to go back and forth seamlessly in my app ( ES to return 10 results each request) including going backwards

Whats the best way to do so?

Thanks in advance

2

If your user has to go to the 1000th page to find relevant results then you probably need to think your search differently.

I'm basically never going more than page number 3 on google. Do you?

2 Likes
3

Thats fair enough, you are correct, Thank you.
But still, I read that the from&size method is very inefficient( resource wise) as it loads the from+size results to the machines memory than sends only the last size

last say i will assume my user scrolls only 3-5 pages forward, AND i want my paging to be efficient as in using the scan and scroll.

Will i be able to let my user move to the previous page?

if i use scroll to get into the 3rd page (20-30), now the user wants to go back to the second page (10-20)

Will sending the first page (0-10) "scroll id" return the second page results ? (20-10)

I will try it out in the meantime.

Thanks!

P.S - its not the end of the world using the From&Size i just think the scan&scroll can be more fitting.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.