How to use search_after with match all query and without sorting?

Elastic Stack Elasticsearch
1

Hi,
I had billions of document in an index. I read half of them with java-high-level-client until my code got an exception. so I need to skip these documents. I found search_after as a solution but I have no idea how it is working on the match-all query and without sorting.
here is the simplified version of my search request :

SearchRequest searchRequest = new SearchRequest("Index");
searchRequest.types("type");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.matchAllQuery());
searchSourceBuilder.size(5000);
// searchSourceBuilder.searchAfter() // need to skip x documents or a given uid
searchRequest.source(searchSourceBuilder);
sourceBuilder.timeout(new TimeValue(10, TimeUnit.MINUTES));
searchRequest.scroll(new TimeValue(10, TimeUnit.MINUTES));
SearchResponse searchResponse = highLevelClient.search(searchRequest);
....

thank you.

2

It needs a sort key.
The problem is that with match all you don't have anything else than _score: 1 which can't help.

I'd recommend starting again from start but sort by _uid which is anyway the most efficient.

BTW you should use scroll API instead.

1 Like
3

Thank you so much
I didn't sort the match all query before and my code did its work on the billions document. so I start the project again with scroll API (read and skip 1 million document per minute on my VPS ).

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.