I would like to use HighLevelRestClient to fetch all results matching a given query using pagination. Now, checking on the rest api I can found that scroll api have been replaced by search_after.
So I checked the HighLevelRestClient documentation in order to understand how to implement pagination but there I found documentation only for the scroll api. So which is the right implementation? I am not performing this query in order to show results to the user but to process a batch of data. Of course while paginating I can still receive data to write or to remove from the index so I would like to keep a coherent view while paginating. Can you please tell me which is the bast way to paginate? Also, I made an experiment with search_after (using the HighLevelRestClient) and it´s working but I don´t see any PIT (point in time) parameter.
Thanks a lot!
This sounds unrelated to our Enterprise Search or App Search products. Reassigning to elasticsearch.
Thanks @JasonStoltz, sorry my bad, I chose the wrong category
Yeah. True. See also the discussion at How to make Point In Time API request using JAVA REST High Level Client
You can use:
fromparameters to display by default up to 10000 records to your users. If you want to change this limit, you can change
index.max_result_windowsetting but be aware of the consequences (ie memory).
- the search after feature to do deep pagination.
- the Scroll API if you want to extract a resultset to be consumed by another tool later.
The same applies to the rest client
Hello @dadoonet, thanks for your reply. Not sure I understood the part related to PIT, the link points to this thread again. Did you mean another thread? In that case, could you please fix it?
Also, @dadoonet which is your advice? Which is the best practice?
I need to go over all the results for a query (it's a percolate query, just to provide you one more detail) and I would like the pagination to be consistent even if in the meantime changes happen. Should I use scroll or search after from the HighLevelRestClient?
I'd use the scroll API.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.