[No code for] Example of using scrollAsync with the java High Level Rest Client

Hi there,

Need an example of using "searchScrollAsync" with the High Level Rest Client 5.6 version

The documentation is quite light for that:
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-search-scroll.html#java-rest-high-search-scroll-async

Thank you in advance

J.

Hi,
there is a complete example below in the same page: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-search-scroll.html#java-rest-high-search-scroll-example . Could you let us know what you are missing there please?

Cheers
Luca

Hi Luca,

Thank you for the reply.

Sure, this is a complete example of secuential scroll. I’m looking for one showing the use of an asynchronous scroll. I need to optimize the reading.

Ex:

client.scrollAsync

Thank you in advance

J.

Hi Lucas,

Maybe you can help us giving a better to understand example considering this code:

Thank you in advance,
For sure that will help a lot to the community

J.

Hi, the logic needs to be moved to the listener in that case. Could you be more specific on what you are struggling with?

Hi there,

Yes, sure. For me is more easy to understand an example like the normal search scroll you have here: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-search-scroll.html

At the same time the link I posted is not very easy to understand for somebody who just started using the HL Rest Client.

Why Elastic Search don't have just a simple example of how to do that in the documentation? Why is to hard to get an example of this? Come on

Thank you in advance

J.

Hi Lucas [@javanna ]

Finally, I have produced an initial code to take advantage of searchScrollAsync functionality, which I consider very useful.

I want to share the code in order to get some help, because what I'm putting inside the listener's methods is not working:

	//Script Query
    final ScriptQueryBuilder scriptQuery = scriptQuery(new Script(ScriptType.INLINE,"painless", queryBuilder(), getParameters()));

    //Builder
    final BoolQueryBuilder bool = new BoolQueryBuilder().must(scriptQuery);
    final SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.size(10000);
    searchSourceBuilder.query(bool);


    //Indices
    System.out.println("INDICES: " + INDICES_10);// list of 10 indices, 


    //Search Request
    final SearchRequest searchRequest   = new SearchRequest(INDICES_10);
    searchRequest.source(searchSourceBuilder);
    searchRequest.scroll("120s");
	
    System.out.println("QUERY: "+ searchRequest.toString());

    //High Level Rest client
    final RestHighLevelClient client = new CustomRestHighLevelClient(getRestClients());

    //First search
    SearchResponse initialSearchResponse = client.search(searchRequest);
    String scrollId = initialSearchResponse.getScrollId();
    System.out.println("Initial SearchResponse total hits: "+ initialSearchResponse.getHits().getTotalHits());

    //Scroll
    SearchScrollRequest scrollRequest = new SearchScrollRequest();
    scrollRequest.scrollId(scrollId);
   /* scrollRequest.scroll(TimeValue.timeValueSeconds(120L));*/

    //Execute asynchronous
    client.searchScrollAsync(scrollRequest, new ActionListener<SearchResponse>() {

        @Override
        public void onResponse(SearchResponse searchResponse) {
            System.out.println("onResponse");
        }

        @Override
        public void onFailure(Exception e) {
            System.out.println("We had an error. Description:" + e.getStackTrace());
        }
    });

The only output after printing the indices and query is:
...
Initial SearchResponse total hits: 140220

Please, any help will be really appreciated. I saw you were involved in the development of this PR #25086. I guess you are the right person.

Thank you in advance,

J.

Here is a full code that works well with 6.6.0:

public class App {
    public static void main(String[] args) throws IOException, InterruptedException {
        RestHighLevelClient client = new RestHighLevelClient(
                RestClient.builder(HttpHost.create("http://localhost:9200")));

        client.indices().delete(new DeleteIndexRequest("test"), RequestOptions.DEFAULT);
        for (int i = 0; i < 100; i++) {
            client.index(new IndexRequest("test", "_doc").source("foo", "bar"), RequestOptions.DEFAULT);
        }
        client.indices().refresh(new RefreshRequest("test"), RequestOptions.DEFAULT);

        SearchRequest searchRequest = new SearchRequest("test").scroll(TimeValue.timeValueSeconds(30L));
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        String scrollId = searchResponse.getScrollId();

        System.out.println("response = " + searchResponse);

        SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId)
                .scroll(TimeValue.timeValueSeconds(30));

        client.scrollAsync(scrollRequest, RequestOptions.DEFAULT, new ActionListener<SearchResponse>() {
            public void onResponse(SearchResponse searchResponse) {
                System.out.println("response async = " + searchResponse);
            }

            public void onFailure(Exception e) {

            }
        });


        Thread.sleep(2000);

        client.close();
    }
}
1 Like

Hi David,

Thank you so much for your help. You are my hero.
For the 6.6 version the code is quite similar. It was enough to realize I have to wait (Thread.sleep) for the results.

I just made a little change to the code:

.....
final CountDownLatch countDownLatch = new CountDownLatch(1);
//Execute asynchronous
client.searchScrollAsync(scrollRequest, new ActionListener<SearchResponse>() {

    @Override
    public void onResponse(SearchResponse searchResponse) {
        System.out.println("onResponse");
        countDownLatch.countDown();
    }

    @Override
    public void onFailure(Exception e) {
        System.out.println("We had an error. Description:" + e.getStackTrace());
    }
});

countDownLatch.await();

........
1 Like

Yeah I did a quick and dirty solution :wink:

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.