How to wait for asynch requests to finish

Elastic Stack Elasticsearch
1

I am using bulk request to insert/update data in Elastic, I send a bunch of asynch requests with 10k documents in each one.

What I want to do is at the end of the process to wait for each request to finish to make sure all the docs came through.

What would be the best way to do this? Is there something I can call to see if there is something still pending?

2

I send a bunch of asynch requests with 10k documents in each one.

How do you send these requests? Why not just wait for the response from these request?

3

I assume that it's faster not to wait and send asynch and let the server process requests.

I can probably wait, but why have asynch then? There should be a way to check if the things finished processing after all the requests are submitted, otherwise you don't even know what happened to them.

4

I am not quite sure what you mean by async. Could you explain?

5

We send the request with data and as one of the parameters we give a callback class. The request returns almost immediately and when it succeeds or fails the callback is called.

Actually I think I found my answer here:
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/master/java-rest-low-usage-requests.html#_multiple_parallel_asynchronous_actions

it uses CountDownLatch

6

Makes sense now. So you can use the same pattern with countdown latch. Just check the resUlt to make sure no errors occurred. When count down latch goes to 0 and you don't have no errors that means all your documents are indexed on primary and all currently available replicas.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.