We are currently using from and size pagination in our Elasticsearch results of ~1,000 records on average. We recently ran into some instances of ~15,000 records and expect some more of similar magnitude, resulting in the 10,000 record cap causing issues.
Users can navigate to specific pages and change the size of the pages in real time, which search after and scroll search don't seem to support. Would increasing the value for index.max_result_window to 20,000 or 30,000 and continue to use from and size pagination be our best option in this case, or is there a more efficient method?
Paginate search results | Elasticsearch Guide [8.17] | Elastic