How to retrieve the last page of my documents even if it is more than 10,000th document?

[Noctis17 (Van Tristan Calimlim)]

Hello and good day! As the title says, we're expecting that we're able to get the last page of our search. But unfortunately, we're encountering an error message:

Things beforehand:

We're aware the with the relationship of the "page" and "page_size" parameter of the search query. Typically, we're just showing 30 documents per page, but it just so happened that one Use Case that we have is, someone was able to click on the last page, here's the parameters

    page_size: 30,
    page: 334

at page: 333, we know that we're displaying documents at the 9,990th document so the query isn't showing any errors yet, but at page: 334 and above, it's now on 10,020th and the error message above is now appearing. After reading some ElasticSearch documentations, ES is really limiting to display only up to 10 thousand documents...

Is there a technique to easily perform this the desired or target output without the use of scroll? As for the "search_after", it's not recommended because for the search_after we're supposed to do a query on a series behavior, but our web app that uses ElasticSearch, should be able to select which page they like any time...

thank you very much in advance!

[warkolm (Mark Walkom)]

The main issue here is what happens when you have 12000 documents, or that increases to 13000, 14000, 18000, 90000. Do you keep increasing the limit to factor that in for the edge case you run into?

Otherwise;

@Noctis17

Is there a technique to easily perform this the desired or target output without the use of scroll?

Nope, sorry.

I would suggest that you may want to reconsider the use case that needs the last document, and then adapt your approach to use scroll.