# How to use BulkProcessor in synchronous processing?

**URL:** <https://discuss.elastic.co/t/how-to-use-bulkprocessor-in-synchronous-processing/283968>\
**Category:** Elasticsearch\
**Tags:** language-clients\
**Created:** [September 12, 2021, 8:56am UTC](https://discuss.elastic.co/t/how-to-use-bulkprocessor-in-synchronous-processing/283968 "2021-09-12T08:56:18Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![avocato](https://avatars.discourse-cdn.com/v4/letter/a/e480ec/32.png) [@avocato](https://discuss.elastic.co/u/avocato)\
**Post date:** [September 12, 2021, 8:56am UTC](https://discuss.elastic.co/t/how-to-use-bulkprocessor-in-synchronous-processing/283968/1 "2021-09-12T08:56:18Z")

</div>

Hello,

I am trying to figure out how to use **BulkProcessor** [[Bulk API | Java REST Client [7.14] | Elastic](https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-document-bulk.html#java-rest-high-document-bulk-processor)] during synchronous processing (`bulk` method) so that I could write something like this:

```auto
    val processor = BulkProcessor.builder(
        { request, _ ->
            client.bulk(
                request,
                RequestOptions.DEFAULT
            )
        })

```

Unfortunately, documentation only mentions async substitutes.

I know that BulkProcessor has **concurrentRequests** [[BulkProcessor.Builder - elasticsearch 6.0.1 javadoc](https://www.javadoc.io/doc/org.elasticsearch/elasticsearch/6.0.1/org/elasticsearch/action/bulk/BulkProcessor.Builder.html#setConcurrentRequests-int-)] property which I could set to **0** but is it enough to mimic synchronous behavior?

Has anyone had a similar problem?

Thank you in advance.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [September 12, 2021, 1:48pm UTC](https://discuss.elastic.co/t/how-to-use-bulkprocessor-in-synchronous-processing/283968/2 "2021-09-12T13:48:31Z")

</div>

The bulk processor has never been synchronous. It has always used a listener.  
If you want to use synchronous bulk api use [Bulk API | Java REST Client [7.14] | Elastic](https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-document-bulk.html)

---

<div class="post-metadata">

**Author:** ![avocato](https://avatars.discourse-cdn.com/v4/letter/a/e480ec/32.png) [@avocato](https://discuss.elastic.co/u/avocato)\
**Post date:** [September 12, 2021, 2:39pm UTC](https://discuss.elastic.co/t/how-to-use-bulkprocessor-in-synchronous-processing/283968/3 "2021-09-12T14:39:37Z")

</div>

@dadoonet thank you for the response. **BulkProcessor** allows me to configure **backoffpolicy** and **bulksize**. Is it possible to configure mentioned properties using the synchronous `bulk` method?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [September 12, 2021, 3:36pm UTC](https://discuss.elastic.co/t/how-to-use-bulkprocessor-in-synchronous-processing/283968/4 "2021-09-12T15:36:25Z")

</div>

Why would you need this as BulkProcessor makes by design asynchronous calls?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [September 12, 2021, 4:50pm UTC](https://discuss.elastic.co/t/how-to-use-bulkprocessor-in-synchronous-processing/283968/5 "2021-09-12T16:50:53Z")

</div>

I also wonder the same thing, but regardless of the "why" it's always simple to turn an asynchronous call into a synchronous one: simply make the async call and then block until the listener completes. For instance, create a `new java.util.concurrent.CountDownLatch(1)` and a listener that calls `CountDownLatch#countDown()` on completion, then pass the listener into the async call and block the calling thread by calling `CountDownLatch#await()` on it.

---

<div class="post-metadata">

**Author:** ![avocato](https://avatars.discourse-cdn.com/v4/letter/a/e480ec/32.png) [@avocato](https://discuss.elastic.co/u/avocato)\
**Post date:** [September 13, 2021, 6:46am UTC](https://discuss.elastic.co/t/how-to-use-bulkprocessor-in-synchronous-processing/283968/6 "2021-09-13T06:46:56Z")

</div>

@DavidTurner thank you for your answer. Maybe my question was wrong. I would like to set properties like **backoffpolicy** and **bulksize**. I know that I can set them using BulkProcessor (hence my question).

If BulkProcessor is designed for async calls then is there any other way to set mentioned properties using sync calls (bulk method)?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [September 13, 2021, 7:48am UTC](https://discuss.elastic.co/t/how-to-use-bulkprocessor-in-synchronous-processing/283968/7 "2021-09-13T07:48:00Z")

</div>

The parameters you are asking about are parameters of the bulk processor, so it doesn't really make sense to ask about using them without using the bulk processor. Why not just use the bulk processor (modifying it to work synchronously as I described)? If you don't want that, you'll need to write some code to retry with backoff and to size the bulks according to your needs.

---

<div class="post-metadata">

**Author:** ![avocato](https://avatars.discourse-cdn.com/v4/letter/a/e480ec/32.png) [@avocato](https://discuss.elastic.co/u/avocato)\
**Post date:** [September 13, 2021, 7:50am UTC](https://discuss.elastic.co/t/how-to-use-bulkprocessor-in-synchronous-processing/283968/8 "2021-09-13T07:50:38Z")

</div>

Ok, thank you. I just thought there is some other solution that I am not aware of.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 11, 2021, 7:50am UTC](https://discuss.elastic.co/t/how-to-use-bulkprocessor-in-synchronous-processing/283968/9 "2021-10-11T07:50:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
