[No code for] Example of using scrollAsync with the java High Level Rest Client

Elastic Stack Elasticsearch
1

Hi there,

Need an example of using "searchScrollAsync" with the High Level Rest Client 5.6 version

The documentation is quite light for that:
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-search-scroll.html#java-rest-high-search-scroll-async

Thank you in advance

J.

2

Hi,
there is a complete example below in the same page: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-search-scroll.html#java-rest-high-search-scroll-example . Could you let us know what you are missing there please?

Cheers
Luca

3

Hi Luca,

Thank you for the reply.

Sure, this is a complete example of secuential scroll. I’m looking for one showing the use of an asynchronous scroll. I need to optimize the reading.

Ex:

client.scrollAsync

Thank you in advance

J.

4

Hi Lucas,

Maybe you can help us giving a better to understand example considering this code:

Thank you in advance,
For sure that will help a lot to the community

J.

5

Hi, the logic needs to be moved to the listener in that case. Could you be more specific on what you are struggling with?

6

Hi there,

Yes, sure. For me is more easy to understand an example like the normal search scroll you have here: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-search-scroll.html

At the same time the link I posted is not very easy to understand for somebody who just started using the HL Rest Client.

Why Elastic Search don't have just a simple example of how to do that in the documentation? Why is to hard to get an example of this? Come on

Thank you in advance

J.

7

Hi Lucas [@javanna ]

Finally, I have produced an initial code to take advantage of searchScrollAsync functionality, which I consider very useful.

I want to share the code in order to get some help, because what I'm putting inside the listener's methods is not working:

	//Script Query
    final ScriptQueryBuilder scriptQuery = scriptQuery(new Script(ScriptType.INLINE,"painless", queryBuilder(), getParameters()));

    //Builder
    final BoolQueryBuilder bool = new BoolQueryBuilder().must(scriptQuery);
    final SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.size(10000);
    searchSourceBuilder.query(bool);


    //Indices
    System.out.println("INDICES: " + INDICES_10);// list of 10 indices, 


    //Search Request
    final SearchRequest searchRequest   = new SearchRequest(INDICES_10);
    searchRequest.source(searchSourceBuilder);
    searchRequest.scroll("120s");
	
    System.out.println("QUERY: "+ searchRequest.toString());

    //High Level Rest client
    final RestHighLevelClient client = new CustomRestHighLevelClient(getRestClients());

    //First search
    SearchResponse initialSearchResponse = client.search(searchRequest);
    String scrollId = initialSearchResponse.getScrollId();
    System.out.println("Initial SearchResponse total hits: "+ initialSearchResponse.getHits().getTotalHits());

    //Scroll
    SearchScrollRequest scrollRequest = new SearchScrollRequest();
    scrollRequest.scrollId(scrollId);
   /* scrollRequest.scroll(TimeValue.timeValueSeconds(120L));*/

    //Execute asynchronous
    client.searchScrollAsync(scrollRequest, new ActionListener<SearchResponse>() {

        @Override
        public void onResponse(SearchResponse searchResponse) {
            System.out.println("onResponse");
        }

        @Override
        public void onFailure(Exception e) {
            System.out.println("We had an error. Description:" + e.getStackTrace());
        }
    });

The only output after printing the indices and query is:
...
Initial SearchResponse total hits: 140220

Please, any help will be really appreciated. I saw you were involved in the development of this PR #25086. I guess you are the right person.

Thank you in advance,

J.

8

Here is a full code that works well with 6.6.0:

public class App {
    public static void main(String[] args) throws IOException, InterruptedException {
        RestHighLevelClient client = new RestHighLevelClient(
                RestClient.builder(HttpHost.create("http://localhost:9200")));

        client.indices().delete(new DeleteIndexRequest("test"), RequestOptions.DEFAULT);
        for (int i = 0; i < 100; i++) {
            client.index(new IndexRequest("test", "_doc").source("foo", "bar"), RequestOptions.DEFAULT);
        }
        client.indices().refresh(new RefreshRequest("test"), RequestOptions.DEFAULT);

        SearchRequest searchRequest = new SearchRequest("test").scroll(TimeValue.timeValueSeconds(30L));
        SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
        String scrollId = searchResponse.getScrollId();

        System.out.println("response = " + searchResponse);

        SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId)
                .scroll(TimeValue.timeValueSeconds(30));

        client.scrollAsync(scrollRequest, RequestOptions.DEFAULT, new ActionListener<SearchResponse>() {
            public void onResponse(SearchResponse searchResponse) {
                System.out.println("response async = " + searchResponse);
            }

            public void onFailure(Exception e) {

            }
        });


        Thread.sleep(2000);

        client.close();
    }
}
1 Like
9

Hi David,

Thank you so much for your help. You are my hero.
For the 6.6 version the code is quite similar. It was enough to realize I have to wait (Thread.sleep) for the results.

I just made a little change to the code:

.....
final CountDownLatch countDownLatch = new CountDownLatch(1);
//Execute asynchronous
client.searchScrollAsync(scrollRequest, new ActionListener<SearchResponse>() {

    @Override
    public void onResponse(SearchResponse searchResponse) {
        System.out.println("onResponse");
        countDownLatch.countDown();
    }

    @Override
    public void onFailure(Exception e) {
        System.out.println("We had an error. Description:" + e.getStackTrace());
    }
});

countDownLatch.await();

........
1 Like
10

Yeah I did a quick and dirty solution :wink:

1 Like
11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.