# Bulk Processor doesn't auto flush after flushInterval duration

**URL:** https://discuss.elastic.co/t/bulk-processor-doesnt-auto-flush-after-flushinterval-duration/95030
**Category:** Elasticsearch
**Created:** [July 29, 2017, 4:08am UTC](https://discuss.elastic.co/t/bulk-processor-doesnt-auto-flush-after-flushinterval-duration/95030 "2017-07-29T04:08:08Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![hugo53](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hugo53/32/24588_2.png) [@hugo53](https://discuss.elastic.co/u/hugo53)
#### Post date: [July 29, 2017, 4:08am UTC](https://discuss.elastic.co/t/bulk-processor-doesnt-auto-flush-after-flushinterval-duration/95030/1 "2017-07-29T04:08:08Z")

</div>

Hi there,

I have a problem with bulk indexing using Bulk Processor. I setup like this:  
.setBulkActions(5000)  
.setBulkSize(new ByteSizeValue(50, ByteSizeUnit.MB))  
.setFlushInterval(TimeValue.timeValueSeconds(5))  
.setConcurrentRequests(3)  
.setBackoffPolicy(BackoffPolicy.exponentialBackoff(TimeValue.timeValueMillis(100), 5))  
.build();

In my application, I need to keep bulk processor to listen for new coming index request, then BulkProcessor is not allowed to close (by BulkProcessor.close). However, Bulk Processor doesn't auto flush remaining request after flushInterval duration, see below.

Use BulkProcessor.close, indexed 4156 documents  
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size  
yellow open test-index Y4Y-HRwpT8SM4Uv3V3v2OA 5 1 4156 0 3.7mb 3.7mb

Without BulkProcessor.close, only indexed 3469 documents  
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size  
yellow open test-index J\_zoR04SRtKZO4p1c-UAWw 5 1 3469 0 2.3mb 2.3mb

So, I would like to know how to index whole documents while no need to close bulk processor. Any help would be appreciated!

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [July 29, 2017, 5:45am UTC](https://discuss.elastic.co/t/bulk-processor-doesnt-auto-flush-after-flushinterval-duration/95030/2 "2017-07-29T05:45:07Z")

</div>

Are you stopping your application? Do you run that in the context of a test?

Can you try also with concurrent requests set to 1?

---

<div class="post-metadata">

### Author: ![hugo53](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hugo53/32/24588_2.png) [@hugo53](https://discuss.elastic.co/u/hugo53)
#### Post date: [July 29, 2017, 2:34pm UTC](https://discuss.elastic.co/t/bulk-processor-doesnt-auto-flush-after-flushinterval-duration/95030/3 "2017-07-29T14:34:57Z")

</div>

I just check my code and seems the application stopped right after finish inserting index request to bulk, then the last request batch is ignored. Thank you so much for your suggestion @dadoonet!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 26, 2017, 2:35pm UTC](https://discuss.elastic.co/t/bulk-processor-doesnt-auto-flush-after-flushinterval-duration/95030/4 "2017-08-26T14:35:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
