ElasticSearch scroll over all documents always returns same 10 results

Elastic Stack Elasticsearch
1

I have index and type with, lets say, 1000 documents. I would like to retrieve all documents using SCROLL API and then perform some action on them.

I created query to get scroll id. Then in the loop, I am using scroll id from previous request to get next 10 elements. The problem is that for some reason, my code retrieving only 10 SAME documents all the time. And my loop is never ending. Break condition (listOfHitsJson.size() < 1) is never return TRUE.

However, request to ES seems to be correct, every time it generates NEW scroll id and passes it to the next request.

I am not sure, what is wrong with this code, why it continue retrieving the same 10 elements in each cycle of the loop? Did I miss something?

JsonArray listOfHitsJson = null;

String query = "{\"query\":{\"filtered\":{\"query\":{\"match_all\":{}}}}}";
Search.Builder searchBuilder = new Search.Builder(query).setParameter("scroll", "1m").addIndex("filterbasedsummarydataindex").addType("filterbasedsummarydata");
SearchResult scrollResult = clientForScrollRequest.execute(searchBuilder.build());

while (true)
{
    String scrollId = scrollResult.getJsonObject().get("_scroll_id").getAsString();

    listOfHitsJson = scrollResult.getJsonObject().getAsJsonObject("hits").getAsJsonArray("hits");

    if (listOfHitsJson.size() > 0)
    {
       // DO SOMETHING WITH HITS
    }

    // Create new scroll request using scrollId from previous scroll request
    searchBuilder = new Search.Builder(query).setParameter("scroll", "1m").setParameter("scroll_id", scrollId).addIndex("filterbasedsummarydataindex").addType("filterbasedsummarydata");
    Search search = searchBuilder.build();
    scrollResult = clientForScrollRequest.execute(search);
    listOfHitsJson = scrollResult.getJsonObject().getAsJsonObject("hits").getAsJsonArray("hits");   

   //BREAK CONDITION: No hits are returned:         
    if (listOfHitsJson.size() < 1) 
    {
        break;
    }
}
System.out.println("Done!");

}

2

The second and following request should be SearchScroll not Search. Please check this example in documentation.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.