Hello,
I would like to use HighLevelRestClient to fetch all results matching a given query using pagination. Now, checking on the rest api I can found that scroll api have been replaced by search_after.
So I checked the HighLevelRestClient documentation in order to understand how to implement pagination but there I found documentation only for the scroll api. So which is the right implementation? I am not performing this query in order to show results to the user but to process a batch of data. Of course while paginating I can still receive data to write or to remove from the index so I would like to keep a coherent view while paginating. Can you please tell me which is the bast way to paginate? Also, I made an experiment with search_after (using the HighLevelRestClient) and it´s working but I don´t see any PIT (point in time) parameter.
Thanks a lot!
This sounds unrelated to our Enterprise Search or App Search products. Reassigning to elasticsearch.
Thanks @JasonStoltz, sorry my bad, I chose the wrong category
Yeah. True. See also the discussion at How to make Point In Time API request using JAVA REST High Level Client
You can use:
- the
size
andfrom
parameters to display by default up to 10000 records to your users. If you want to change this limit, you can changeindex.max_result_window
setting but be aware of the consequences (ie memory). - the search after feature to do deep pagination.
- the Scroll API if you want to extract a resultset to be consumed by another tool later.
The same applies to the rest client
Hello @dadoonet, thanks for your reply. Not sure I understood the part related to PIT, the link points to this thread again. Did you mean another thread? In that case, could you please fix it?
Also, @dadoonet which is your advice? Which is the best practice?
I need to go over all the results for a query (it's a percolate query, just to provide you one more detail) and I would like the pagination to be consistent even if in the meantime changes happen. Should I use scroll or search after from the HighLevelRestClient?
Thanks!
Fixed
I'd use the scroll API.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.